I have a SQL variable which stores the following JSON object:
{"name": "Dan", "country": "Japan", "job": "manager"}
My goal is to sort the object based on the keys alphabetically and return it
{"country": "Japan", "job": "manager", "name": "Dan"}. The keys will vary for different json objects.
Unfortunately, I am unable to use OpenJson because of compatibility issue and it may stay that way. So I tried using JSON_TABLE command. My overall goal is to:
Create a table with two columns, keys and values, keys will hold all the keys in the json object, whereas values will hold the corresponding value for the key in that row.
Use order by key to return the result in sorted way
Return the result in a json object format
However, I am getting stuck at the first step. I created two variables to store the json arrays for the keys and values separately using JSON_KEYS and JSON_EXTRACT. Then, I combined them into another variable, which has the following structure:
{key: ['name', 'country', 'job'], value:['Dan', 'Japan', 'manager']}. Finally, I used JSON_Table on it with the hope of completing step 1. But since they are sibling arrays, JSON_TABLE produced two columns where total number of rows are 6 instead of 3. I do not know how to get past this.
Is there any simpler solution than this?
Thanks in advance.
This is because of you set type JSON of column. So it converts automatically in alphabetically order. If you want to save data of json array as you insert(not rearrange in alphabetically). you need to change column type JSON to TEXT. it will work.
Thank you.
Related
I have a field in a mysql table that stores events in JSON. This JSON has inside it the ID of a file that the event is about. Is there a way to have another field auto-populate with that file ID? Something like a formula field in Excel? Format inside the JSON is "item_id": "1234567"
New to SQL so help is appreciated. :)
Example JSON in column "event":
{"video_proc_producer_ver": 2,"mp_event":{"project_name": "some
project","project_account": "some customer","mp_notes": "No playable
combo ids found for item_id: 1234abcd.\\n\n","item_name":
"file.mov","mp_m_stamp": "2020-03-09 02:27:50","mp_c_stamp":
"2020-03-09 02:22:14","mp_processing_mask": "4","c_user_id":
"123456","mp_export_time": "2020-03-09 02:24:22","item_id":
"0987654","project_id": "1234","mp_complete_time": "2020-03-09
02:27:50"}}
result in column file_id: 0987654
I know I can use some process outside the table to pull that info and insert it, but I was wondering if there is the equivalent of a calculated column like in except that I can use inside the table to have that auto-populate. If so, what kind of column do I need to create and what formula would I use?
It looks like you want to access the content of the json object. In MySQL, you can use json_extract(), or the ->> operator, like so:
select event, event ->> '$.mp_event.item_id' file_id from mytable
I have a dataset with a column containing arrays of JSON data that looks like:
[{"name":"aaa","type":"yyy"},{"name":"bbb","type":"ccc"}]
or more specifically:
dataset with JSON array column
Is there any straight forward method of extracting the JSON data from the column using something like JSON_QUERY, so that I can use it in a report
As far as I can tell, the existing JSON array format is not usable with any of the T-SQL JSON functions.
The array in the column "jsonCol" needs to be in the form of:
{ "tag": [{"name":"aaa","type":"yyy"},{"name":"bbb","type":"ccc"}]}
and then I can extract each array element individually with:
SELECT JSON_QUERY(jsonCol, '$.tag[0]') as tag
FROM
So I could add a prefix and suffix string to the select statement to fix this as long as no one else will see it.
I have an Azure Data Factory v2 pipeline that's pulling data from a Cosmos DB collection. This collection has a property that's an array.
I want to, at the least, be able to dump that entire property's value into a column in SQL Azure. I don't need it parsed (although that would be great too), but ADF lists this column as "Unsupported Type" in the dataset definition and listed it in the Excluded Columns section.
Here is an example of the JSON I'm working with. The property I want is "MyArrayProperty":
{
"id": "c4e2012e-af82-4c48-8960-11e0436e6d3f",
"Created": "2019-06-14T16:04:13.9572567Z",
"Updated": "2019-06-14T16:04:14.1920988Z",
"IsActive": true,
"MyArrayProperty": [
{
"SomeId": "a4427015-ca69-4958-90d3-0918fd5dcac1",
"SomeName": "BlahBlah"
}
]
}
}
I've tried manually specifying a column in the ADF data source like "MyArrayProperty" and using a string data type, but the value always comes across as null.
please check this document about schema mapping example between MongoDB and Azure SQL. Basically you should define your collectionReference that will iterate through your nested array of objects and do cross apply.
There may be a better way to solve this problem, but I ended up creating a second copy activity which uses a query against Cosmos rather than a collection based capture. The query flattened the array like so:
SELECT m.id, c.SomeId, c.SomeName
FROM myCollection m join c in m.MyArrayProperty
I then took this data set and dumped it into a table in SQL then did my other work inside SQL Azure itself. You could also use the new Join pipeline task to do this in memory before it gets to the destination.
As part of a tool I am creating for my team I am connecting to an internal web service via PowerQuery.
The web service returns nested JSON, and I have trouble parsing the JSON data to the format I am looking for. Specifically, I have a problem with extracting the content of records in a column to a comma separated list.
The data
As you can see, the data contains details related to a specific "race" (race_id). What I want to focus on is the information in the driver_codes which is a List of Records. The amount of records varies from 0 to 4 and each record is structured as id: 50000 (50000 could be any 5 digit number). So it could be:
id: 10000
id: 20000
id: 30000
As requested, an example snippet of the raw JSON:
<race>
<race_id>ABC123445</race_id>
<begin_time>2018-03-23T00:00:00Z</begin_time>
<vehicle_id>gokart_11</vehicle_id>
<driver_code>
<id>90200</id>
</driver_code>
<driver_code>
<id>90500</id>
</driver_code>
</race>
I want it to be structured as:
10000,20000,30000
The problem
When I choose "Extract values" on the column with the list, then I get the following message:
Expression.Error: We cannot convert a value of type Record to type
Text.
If I instead choose "Expand to new rows", then duplicate rows are created for each unique driver code. I now have several rows per unique race_id, but what I wanted was one row per unique race_id and a concatenated list of driver codes.
What I have tried
I have tried grouping the data by the race_id, but the operations allowed when grouping data do not include concatenating rows.
I have also tried unpivoting the column, but that leaves me with the same problem: I still get multiple rows.
I have googled (and Stack Overflowed) this issue extensively without luck. It might be that I am using the wrong keywords, however, so I apologize if a duplicate exists.
UPDATE: What I have tried based on the answers so far
I tried Alexis Olson's excellent and very detailed method, but I end up with the following error:
Expression.Error: We cannot convert the value "id" to type Number. Details:
Value=id
Type=Type
The error comes from using either of these lines of M code (one with a List.Transform and one without):
= Table.Group(#"Renamed Columns", {"race_id", "begin_time", "vehicle_id"},
{{"DriverCodes", each Text.Combine([driver_code][id], ","), type text}})
= Table.Group(#"Renamed Columns", {"race_id", "begin_time", "vehicle_id"},
{{"DriverCodes", each Text.Combine(List.Transform([driver_code][id], each Number.ToText(_)), ","), type text}})
NB: if I do not write [driver_code][id] but only [id] then I get another error saying that column [id] does not exist.
Here's the JSON equivalent to the XML example you gave:
{"race": {
"race_id": "ABC123445",
"begin_time": "2018-03-23T00:00:00Z",
"vehicle_id": "gokart_11",
"driver_code": [
{ "id": "90200" },
{ "id": "90500" }
]}}
If you load this into the query editor, convert it to a table, and expand out the Value record, you'll have a table that looks like this:
At this point, choose Expand to New Rows, and then expand the id column so that your table looks like this:
At this point, you can apply the trick #mccard suggested. Group by the first columns and aggregate over the last using, say, max.
This last step produces M code like this:
= Table.Group(#"Expanded driver_code1",
{"Name", "race_id", "begin_time", "vehicle_id"},
{{"id", each List.Max([id]), type text}})
Instead of this, you want to replace List.Max with Text.Combine as follows:
= Table.Group(#"Changed Type",
{"Name", "race_id", "begin_time", "vehicle_id"},
{{"id", each Text.Combine([id], ","), type text}})
Note that if your id column is not in the text format, then this will throw an error. To fix this, insert a step before you group rows using Transform Tab > Data Type: Text to convert the type. Another options is to use List.Transform inside your Text.Combine like this:
Text.Combine(List.Transform([id], each Number.ToText(_)), ",")
Either way, you should end up with this:
An approach would be to use the Advanced Editor and change the operation done when grouping the data directly there in the code.
First, create the grouping using one of the operations available in the menu. For instance, create a column"Sum" using the Sum operation. It will give an error, but we should get the starting code to work on.
Then, open the Advanced Editor and find the code corresponding to the operation. It should be something like:
{{"Sum", each List.Sum([driver_codes]), type text}}
Change it to:
{{"driver_codes", each Text.Combine([driver_codes], ","), type text}}
I have a dynamically created jqGrid with a custom formatter. I need to be able to access some information that is not in a grid column within the formatter. My problem is that the rowObject seems to contain only the information from the columns and nothing else, even though the JSON data has an extra attribute.
My JSON looks like:
{"rows":
[
{"cell":["637","Alice","Test","01\/01\/1980",""],"id":"637","patient_id":"637"},
...
{"cell":["635","Alice","Test","01\/01\/1980",""],"id":"635","patient_id":"635"},
],
"form_id":"3",
"records":"35",
"total":2,
"goto_patient_sidebar":"1",
"totalRecords":"35",
"dynamicFilters":"",
"listName":"Patients",
"page":1,
"recordsReturned":25,
"columns":
[
{"id":"75","field_id":"2","zmr_list_id":"13","alias":"Last Name",
"sortorder":"0","reference_field_id":"90","show_date":"0","show_time":"0"},
{"id":"76","field_id":"1","zmr_list_id":"13","alias":"First Name",
"sortorder":"1","reference_field_id":"90","show_date":"0","show_time":"0"},
{"id":"77","field_id":"25","zmr_list_id":"13","alias":"DOB",
"sortorder":"2","reference_field_id":"90","show_date":"1","show_time":"0"},
{"id":"78","field_id":"47","zmr_list_id":"13","alias":"Phone Number",
"sortorder":"3","reference_field_id":"90","show_date":"0","show_time":"0"}
]
}
For each row, I need to access the patient_id from the JSON. The data for the columns of each row is contained in the 'cell' for each row. The rowObject seems to contain only the information in each cell attribute of the JSON. How can I access patient_id without it being rendered in the column?
In the data the values of patient_id are always the same as the id. I suppose that there can be different (if patient_id are equal to id you can see the information in the custom formatter already). You can implement what you need in at least three ways:
You add an hidden column (having the property hidden:true) which will represent the patient_id. You move the information about the patient_id in the JSON input inside the "cell" array.
You place the information about the mapping between id and patient_id as the part of userdata which you will include in the JSON (see here for more information).
You use data parameter of the loadComplete event handle. The data parameter will contain full JSON data posted from the server. You can get the information which you need from the data and save it somewhere. Then you get the saved information from the custom formatter.
One more small remark about your JSON data. Currently you use items like
{
"cell": [
"637",
"Alice",
"Test",
"01\/01\/1980",
""
],
"id": "637",
}
(if we forget about the patient_id). It means that you send id information twice: one as the part of the column and second time for the first column of the grid. If you would include key:true setting in the definition of the first grid column and add jsonReader: {cell:""} you could make the JSON data in more compact where the row item would be in the array form
[
"637",
"Alice",
"Test",
"01\/01\/1980",
""
]
Add a hidden column for the extra data.