If this json output looks like this
{
"Bank_Name":"This is bank name",
"ACC_Name":"Tummy",
"ACC_No":"1122XXXX115",
"Date_Active":"Jan 31 2019 2:16PM",
"Date_Expired":"Nov 17 2020 1:14PM",
"Bank_Status":"Expired",
"email_Notif":[
{"Verification":[{"User_Email":"tfe.master#gmail.com","Send_DateTime":"2020-11-03T13:30:59.7036152"},
{"User_Email":"the.user#outlook.com","Send_DateTime":"2020-11-03T13:31:02.1563596"}]
},{"Verified":[{"DateTime": "2020-11-03T13:31:02.1563596"}]
},
{ "Updating":[{"User_Email":"the.spv#gmail.com","Send_DateTime":"2020-11-03T13:30:59.7036152"},
{"User_Email":"the.officer#outlook.com","Send_DateTime":"2020-11-03T13:31:02.1563596"}]
}
],
"rejection_Statuses":[
{"Verification":"Nov 3 2020 01:31:02 PM"} ,
{"Verified":"Nov 7 2020 01:12:03 PM"} ,
{"Updating":"Nov 17 2020 01:18:03 PM"} ,
{"Re_run":"Nov 27 2020 05:18:03 PM"}
]
}
Questions:
How do I use "JSON_Modifiy" in SQL Server to insert email_Notif (as object of array) ? if JSON input looks like this:
{"Bank_Name": "BPD SULAWESI SELATAN",
"ACC_Name": "Tutik",
"ACC_No": "1122000115",
"Date_Active": "Jan 31 2019 2:16PM",
"Date_Expired": "Nov 17 2020 1:14PM",
"Bank_Status": "Expired",
"rejection_Statuses":[
{"Verification":"Nov 3 2020 01:31:02 PM"} ,
{"Verified":"Nov 7 2020 01:12:03 PM"} ,
{"Updating":"Nov 17 2020 01:18:03 PM"} ,
{"Re_run":"Nov 27 2020 05:18:03 PM"}]
}
How to get value from "email_Notify" as JSON format in SQL Server query by using select statement ? (Verification, Verified and Updating)
If I understand correctly, one way to achieve this is:
JSON:
DECLARE
#email_Verfication nvarchar(max) = N'{
"Verification":[
{"User_Email":"tfe.master#gmail.com", "Send_DateTime":"2020-11-03T13:30:59.7036152"},
{"User_Email":"the.user#outlook.com", "Send_DateTime":"2020-11-03T13:31:02.1563596"}
]
}',
#email_Verified nvarchar(max) = N'{
"Verified":[
{"DateTime":"2020-11-03T13:31:02.1563596"}
]
}',
#email_Updating nvarchar(max) = N'{"Updating":[
{"User_Email":"the.spv#gmail.com", "Send_DateTime":"2020-11-03T13:30:59.7036152"},
{"User_Email":"the.officer#outlook.com", "Send_DateTime":"2020-11-03T13:31:02.1563596"}
]
}',
#detail NVARCHAR (MAX) = N'{
"Bank_Name":"BPD SULAWESI SELATAN",
"ACC_Name":"Tutik",
"ACC_No":"1122000115",
"Date_Active":"Jan 31 2019 2:16PM",
"Date_Expired":"-",
"Bank_Status":"Active",
"rejection_Statuses":[
{"Verification":"Nov 3 2020 01:31:02 PM"},
{"Verified":"Nov 7 2020 01:12:03 PM"},
{"Updating":"Nov 17 2020 01:18:03 PM"},
{"Re_run":"Nov 27 2020 05:18:03 PM"}
]
}'
Modify JSON:
SET #detail = JSON_MODIFY (#detail, 'append $.email_Notif', JSON_QUERY(#email_Verfication, '$'))
SET #detail = JSON_MODIFY (#detail, 'append $.email_Notif', JSON_QUERY(#email_Verified, '$'))
SET #detail = JSON_MODIFY (#detail, 'append $.email_Notif', JSON_QUERY(#email_Updating, '$'))
Parse JSON:
SELECT j2.[key], j2.[value]
FROM OPENJSON(#json, '$.email_Notif') j1
CROSS APPLY OPENJSON(j1.[value], '$') j2
Related
I have a json and in my example values for years 1996 to 2012 were left out. Is there any way to insert the missing values (years)? If yes, how? Finally, the json should be completely filled in (1988 to 2021) and insert the value "0" for the years where the year is missing.
Kind Regards and thank you
{
"data": {
"ArrayValue": [
0.0030350000000000004,
0.003661,
0.0080348532,
0.0053554275,
0.004284,
0.008569710000000001,
0.008569710000000001,
0.007498282499999999,
0.189286,
0.42142999999999997,
0.461429,
0.5075000000000001,
0.5575,
0.615,
0.705,
0.76,
0.8075,
0.865,
0.89
],
"ArrayYears": [
"1988",
"1989",
"1990",
"1991",
"1992",
"1993",
"1994",
"1995",
"2012",
"2013",
"2014",
"2015",
"2016",
"2017",
"2018",
"2019",
"2020",
"2021",
"TTM"
]
}
}
2012 is there. Filling in 1996 - 2011:
'use strict';
const fs = require('fs');
let rawdata = fs.readFileSync('input.json');
let obj = JSON.parse(rawdata);
const len = 16;
const insertPos = 8;
const totSize = 35;
const tempArrData = new Array(len).fill(0);
const tempArrYears = new Array(len);
let i = 0;
for (let y = 1996; y < 2012; y += 1) {
tempArrYears[i] = y + '';
i += 1;
}
const arrV = obj.data.ArrayValue;
arrV.splice(insertPos, 0, ...tempArrData);
const arrY = obj.data.ArrayYears;
arrY.splice(insertPos, 0, ...tempArrYears);
for (let i = 0; i < totSize; i += 1) {
console.log(`${obj.data.ArrayYears[i]} , ${obj.data.ArrayValue[i]}`);
}
Output:
1988 , 0.0030350000000000004
1989 , 0.003661
1990 , 0.0080348532
1991 , 0.0053554275
1992 , 0.004284
1993 , 0.008569710000000001
1994 , 0.008569710000000001
1995 , 0.007498282499999999
1996 , 0
1997 , 0
1998 , 0
1999 , 0
2000 , 0
2001 , 0
2002 , 0
2003 , 0
2004 , 0
2005 , 0
2006 , 0
2007 , 0
2008 , 0
2009 , 0
2010 , 0
2011 , 0
2012 , 0.189286
2013 , 0.42142999999999997
2014 , 0.461429
2015 , 0.5075000000000001
2016 , 0.5575
2017 , 0.615
2018 , 0.705
2019 , 0.76
2020 , 0.8075
2021 , 0.865
TTM , 0.89
The JSON string we have stored in the table has this format to it:
[
{
"itemid" : "18726539",
"duration" : "19:28",
"title" : "Flight Plan for Trading: Market Lessons from My Pilot Dad",
"description" : "<p>Learning to fly an airplane can build an excellent structured thought process for navigating markets</p>",
"pubDate" : "Wed, 14 Apr 2021 18:04:19 +0000"
}
]
When I test it with IS JSON, all the data comes back. So, Oracle is seeing this as JSON. However, I'm not sure how to get the "itemid" and "description" items. I tried this query, but it comes up empty:
SELECT JSON_QUERY(json_str, '$.itemid' RETURNING VARCHAR2(30) WITH ARRAY WRAPPER) AS overview FROM json_podcast_data;
I'm thinking that the leading bracket might be causing an issue?
Any help is greatly appreciated.
Rather than using multiple JSON_VALUE functions, you can use a single JSON_TABLE:
SELECT j.itemid,
j.duration,
j.title,
j.description,
TO_TIMESTAMP_TZ(
j.pubdate,
'Dy, DD Mon YYYY HH24:MI:SS TZHTZM',
'NLS_DATE_LANGUAGE=American'
) AS pubdate
FROM json_podcast_data d
CROSS JOIN JSON_TABLE(
d.json_str,
'$[*]'
ERROR ON ERROR
COLUMNS(
itemid VARCHAR2(20) PATH '$.itemid',
duration VARCHAR2(10) PATH '$.duration',
title VARCHAR2(200) PATH '$.title',
description VARCHAR2(4000) PATH '$.description',
pubdate VARCHAR2(50) PATH '$.pubDate'
)
) j
Which, for the sample data:
CREATE TABLE json_podcast_data (json_str CLOB CHECK (json_str IS JSON ) );
INSERT INTO json_podcast_data ( json_str ) VALUES (
'[
{
"itemid" : "18726539",
"duration" : "19:28",
"title" : "Flight Plan for Trading: Market Lessons from My Pilot Dad",
"description" : "<p>Learning to fly an airplane can build an excellent structured thought process for navigating markets</p>",
"pubDate" : "Wed, 14 Apr 2021 18:04:19 +0000"
}
]' );
Outputs:
ITEMID
DURATION
TITLE
DESCRIPTION
PUBDATE
18726539
19:28
Flight Plan for Trading: Market Lessons from My Pilot Dad
Learning to fly an airplane can build an excellent structured thought process for navigating markets
2021-04-14 18:04:19.000000 +00:00
You can get the value of the itemid and description using JSON_VALUE. See example below
WITH
json_podcast_data (json_str)
AS
(SELECT '[
{
"itemid" : "18726539",
"duration" : "19:28",
"title" : "Flight Plan for Trading: Market Lessons from My Pilot Dad",
"description" : "<p>Learning to fly an airplane can build an excellent structured thought process for navigating markets</p>",
"pubDate" : "Wed, 14 Apr 2021 18:04:19 +0000"
}
]'
FROM DUAL)
SELECT json_value (json_str, '$[0].itemid') AS itemid,
json_value (json_str, '$[0].description') AS description
FROM json_podcast_data;
ITEMID DESCRIPTION
___________ ______________________________________________________________________________________________________________
18726539 <p>Learning to fly an airplane can build an excellent structured thought process for navigating markets</p>
If your JSON array has multiple items in it, you can use the JSON_TABLE function to list all items in the array.
/* Formatted on 27-Jul-2021 5:51:38 PM (QP5 v5.354) */
WITH
json_podcast_data (json_str)
AS
(SELECT '[
{
"itemid" : "18726539",
"duration" : "19:28",
"title" : "Flight Plan for Trading: Market Lessons from My Pilot Dad",
"description" : "<p>Learning to fly an airplane can build an excellent structured thought process for navigating markets</p>",
"pubDate" : "Wed, 14 Apr 2021 18:04:19 +0000"
},
{
"itemid" : "23456",
"duration" : "10:15",
"title" : "Test Title",
"description" : "<p>Some cool podcast</p>",
"pubDate" : "Wed, 14 Apr 2021 18:04:19 +0000"
}
]'
FROM DUAL)
SELECT j.itemid, j.description
FROM json_podcast_data
CROSS JOIN
JSON_TABLE (
json_str,
'$[*]'
COLUMNS itemid NUMBER PATH '$.itemid', description VARCHAR2 PATH '$.description') j;
ITEMID DESCRIPTION
___________ ______________________________________________________________________________________________________________
18726539 <p>Learning to fly an airplane can build an excellent structured thought process for navigating markets</p>
23456 <p>Some cool podcast</p>
I have a json data as below:
{
"X": "abc",
"Y": 1,
"Z": 4174,
"t_0":
{
"M": "bm",
"T": "sp",
"CUD": 4,
"t_1": '
{
"CUD": "1",
"BBC": "09",
"CPR": -127
},
"EVV": "10.7000",
"BBC": -127,
"CMIX": "25088"
},
"EYR": "sp"
}
The problem is converting to python data-frame creates two columns of same name CUD. One is under t_0 and another is under t_1. But both are different events. How can I append json tag name to column names so that I can differentiate two columns of same name. Something like t_0_CUD , t_1_CUD.
My code is below:
df = pd.io.json.json_normalize(json_data)
df.columns = df.columns.map(lambda x: x.split(".")[-1])
If use only first part of solution it return what you need, only instead _ are used .:
df = pd.io.json.json_normalize(json_data)
print (df)
X Y Z EYR t_0.M t_0.T t_0.CUD t_0.t_1.CUD t_0.t_1.BBC t_0.t_1.CPR \
0 abc 1 4174 sp bm sp 4 1 09 -127
t_0.EVV t_0.BBC t_0.CMIX
0 10.7000 -127 25088
If need _:
df.columns = df.columns.str.replace('\.','_')
print (df)
X Y Z EYR t_0_M t_0_T t_0_CUD t_0_t_1_CUD t_0_t_1_BBC t_0_t_1_CPR \
0 abc 1 4174 sp bm sp 4 1 09 -127
t_0_EVV t_0_BBC t_0_CMIX
0 10.7000 -127 25088
How to query result like these picture.
First column select is this month plus next column by condition field (SelectColumn)
yellow background is the select column for sum
my example code.
declare #myDate date = getdate(),#qry varchar(max)
set #qry = 'select case v.SelectColumn
when 0 then (SELECT '+DATENAME(month,#myDate)+')
when 2 then (SELECT '+DATENAME(month,#myDate)+'+'+DATENAME(MONTH,DATEADD(MONTH,1,#myDate))+')
when 1 then (SELECT '+DATENAME(month,#myDate)+'+'+DATENAME(MONTH,DATEADD(MONTH,1,#myDate))+'+'+DATENAME(MONTH,DATEADD(MONTH,2,#myDate))+')
end
as SumColumn
from vwQC12Month v'
exec (#qry)
This problem requires the monthly values to be accessible as rows such that selected aggregation can be applied. Typically, you can use UNPIVOT or VALUES to rotate columns into rows. Below is a working example using UNPIVOT.
In the query below, you'd have to change the short form month names to match your column names for it to work. Here is a working demo of this: http://sqlfiddle.com/#!18/094e4/2
Also you might want to consider how you want to deal with year end wrapping and adjust the aggregation accordingly.
-- setup sample data
create table Test (
id int, Jan int, Feb int, Mar int, Apr int, May int,
Jun int, Jul int, Aug int, Sep int, Oct int, Nov int, Dec int,
SelectColumn int)
insert Test values
(1, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 1),
(2, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 2),
(3, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 3),
(4, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 4)
-- query
DECLARE #currentMonthNumber int = MONTH(getdate())
-- CTE that rotates the data into rows
; WITH RowsByMonth(id, SelectColumn, monthNumber, val) AS
(
SELECT id, SelectColumn, CONVERT(int, month) AS monthNumber, val
FROM
(SELECT id, SelectColumn,
Jan AS [1], Feb AS [2], Mar AS [3], Apr AS [4],
May AS [5], Jun AS [6], Jul AS [7], Aug AS [8],
Sep AS [9], Oct AS [10], Nov AS [11], Dec AS [12]
FROM Test) AS Source
UNPIVOT
(val FOR month IN
([1], [2], [3], [4],
[5], [6], [7], [8],
[9], [10], [11], [12])
) AS asRows
) -- aggregation below
SELECT id, SUM(val) AS SumAcrossSelectedMonths
FROM RowsByMonth
WHERE
monthNumber >= #currentMonthNumber
AND monthNumber - #currentMonthNumber < SelectColumn
GROUP BY id
-- Results
| id | SumAcrossSelectedMonths |
|----|-------------------------|
| 1 | 40 |
| 2 | 90 |
| 3 | 150 |
| 4 | 220 |
I am trying to write a data.frame from R into a JSON file, but in a hierarchical structure with child nodes within them. I found examples and JSONIO but I wasn't able to apply it to my case.
This is the data.frame in R
> DF
Date_by_Month CCG Year Month refYear name OC_5a OC_5b OC_5c
1 2010-01-01 MyTown 2010 01 2009 2009/2010 0 15 27
2 2010-02-01 MyTown 2010 02 2009 2009/2010 1 14 22
3 2010-03-01 MyTown 2010 03 2009 2009/2010 1 6 10
4 2010-04-01 MyTown 2010 04 2010 2010/2011 0 10 10
5 2010-05-01 MyTown 2010 05 2010 2010/2011 1 16 7
6 2010-06-01 MyTown 2010 06 2010 2010/2011 0 13 25
In addtion to writing the data by month, I would also like to create an aggregate child, the 'yearly' one, which holds the sum (for example) of all the months that fall in this year. This is how I would like the JSON file to look like:
[
{
"ccg":"MyTown",
"data":[
{"period":"yearly",
"scores":[
{"name":"2009/2010","refYear":"2009","OC_5a":2, "OC_5b": 35, "OC_5c": 59},
{"name":"2010/2011","refYear":"2010","OC_5a":1, "OC_5b": 39, "OC_5c": 42},
]
},
{"period":"monthly",
"scores":[
{"name":"2009/2010","refYear":"2009","month":"01","year":"2010","OC_5a":0, "OC_5b": 15, "OC_5c": 27},
{"name":"2009/2010","refYear":"2009","month":"02","year":"2010","OC_5a":1, "OC_5b": 14, "OC_5c": 22},
{"name":"2009/2010","refYear":"2009","month":"03","year":"2010","OC_5a":1, "OC_5b": 6, "OC_5c": 10},
{"name":"2009/2010","refYear":"2009","month":"04","year":"2010","OC_5a":0, "OC_5b": 10, "OC_5c": 10},
{"name":"2009/2010","refYear":"2009","month":"05","year":"2010","OC_5a":1, "OC_5b": 16, "OC_5c": 7},
{"name":"2009/2010","refYear":"2009","month":"01","year":"2010","OC_5a":0, "OC_5b": 13, "OC_5c": 25}
]
}
]
},
]
Thank you so much for your help!
Expanding on my comment:
The jsonlite package has a lot of features, but what you're describing doesn't really map to a data frame anymore so I doubt any canned routine has this functionality. Your best bet is probably to convert the data frame to a more general list (FYI data frames are stored internally as lists of columns) with a structure that matches the structure of the JSON exactly, then just use the converter to translate
This is complicated in general but in your case should be fairly simple. The list will be structured exactly like the JSON data:
list(
list(
ccg = "Town1",
data = list(
list(
period = "yearly",
scores = yearly_data_frame_town1
),
list(
period = "monthly",
scores = monthly_data_frame_town1
)
)
),
list(
ccg = "Town2",
data = list(
list(
period = "yearly",
scores = yearly_data_frame_town2
),
list(
period = "monthly",
scores = monthly_data_frame_town2
)
)
)
)
Constructing this list should be a straightforward case of looping over unique(DF$CCG) and using aggregate at each step, to construct the yearly data.
If you need performance, look to either the data.table or dplyr packages to do the looping and aggregating all at once. The former is flexible and performant but a little esoteric. The latter has relatively easy syntax and is similarly performant, but is designed specifically around building pipelines for data frames so it might take some hacking to get it to produce the right output format.
Looks like ssdecontrol has you covered... but here's my solution. Need to loop over unique CCG and Years to create the entire data set...
df <- read.table(textConnection("Date_by_Month CCG Year Month refYear name OC_5a OC_5b OC_5c
2010-01-01 MyTown 2010 01 2009 2009/2010 0 15 27
2010-02-01 MyTown 2010 02 2009 2009/2010 1 14 22
2010-03-01 MyTown 2010 03 2009 2009/2010 1 6 10
2010-04-01 MyTown 2010 04 2010 2010/2011 0 10 10
2010-05-01 MyTown 2010 05 2010 2010/2011 1 16 7
2010-06-01 MyTown 2010 06 2010 2010/2011 0 13 25"), stringsAsFactors=F, header=T)
library(RJSONIO)
to_list <- function(ccg, year){
df_monthly <- subset(df, CCG==ccg & Year==year)
df_yearly <- aggregate(df[,c("OC_5a", "OC_5b", "OC_5c")] ,df[,c("name", "refYear")], sum)
l <- list("ccg"=ccg,
data=list(list("period" = "yearly",
"scores" = as.list(df_yearly)
),
list("period" = "monthly",
"scores" = as.list(df[,c("name", "refYear", "OC_5a", "OC_5b", "OC_5c")])
)
)
)
return(l)
}
toJSON(to_list("MyTown", "2010"), pretty=T)
Which returns this:
{
"ccg" : "MyTown",
"data" : [
{
"period" : "yearly",
"scores" : {
"name" : [
"2009/2010",
"2010/2011"
],
"refYear" : [
2009,
2010
],
"OC_5a" : [
2,
1
],
"OC_5b" : [
35,
39
],
"OC_5c" : [
59,
42
]
}
},
{
"period" : "monthly",
"scores" : {
"name" : [
"2009/2010",
"2009/2010",
"2009/2010",
"2010/2011",
"2010/2011",
"2010/2011"
],
"refYear" : [
2009,
2009,
2009,
2010,
2010,
2010
],
"OC_5a" : [
0,
1,
1,
0,
1,
0
],
"OC_5b" : [
15,
14,
6,
10,
16,
13
],
"OC_5c" : [
27,
22,
10,
10,
7,
25
]
}
}
]
}