I am trying to convert JSON API response into a table in Excel using the Power Query functionality. I am currently getting the error below when I try to put the customerOrderHistory which is a list into a delimited list as I don't want to create extra rows for a list if I can avoid it.
If possible I would like to either
Just print the customerOrderHistory list in the JSON format that it is in already into the cell
OR
Create a delimited list of the values as the lists only contain one entry at the moment
The JSON test file looks like this:
{
"computerid": "1",
"total": 1,
"results": [
{
"computerid": "1",
"customerOrderHistory": [
{
"orderId": "1",
"channelId": null,
"agentId": null,
"orderItems": 1
}
]
}
]
}
Thanks
You can use Json.FromValue to convert the list into a Binary value, and then use Text.FromBinary to get back the string representation. If you can dig down deep enough to get to the customerOrderHistory field it would look like Text.FromBinary(Json.FromValue(result[customerOrderHistory])).
Related
I've defined a default crawler on the data directory of an export from dynamodb. I'm trying to get it to give me a structured table instead of a table with a single column of type struct. What do I have to do make get the actual column names in there? I've tried adding custom classifiers and different path expressions but nothing seems to work, and I feel like I'm missing something really obvious.
I'm using the crawler builder inside of glue, which doesn't seem to offer much customization.
Here's the schema from the table generated by the default crawler:
And here's one of the items that I've exported from dynamo:
{
"Item": {
"the_url": {
"S": "/2021/07/06/****redacted****.html"
},
"as_of_when": {
"S": "2021-09-01"
},
"user_hashes": {
"SS": [
"****redacted*****"
]
},
"user_id_hashes": {
"SS": [
"u3MeXDcpQm0ACYuUv6TMrg=="
]
},
"accumulated_count": {
"N": "1"
},
"today_count": {
"N": "1"
}
}
}
The way Athena interprets JSON data means that your data has only a single column, Item. Athena doesn't have any mechanism to map arbitrary parts of a JSON object to columns, it can only map top-level attributes to columns.
If you want other parts of the objects as columns you will either have to create a new table with transformed data, or create a view with the attributes as columns, e.g.
CREATE OR REPLACE VIEW attributes_as_top_level_columns AS
SELECT
item.the_url.S AS the_url,
CAST(item.as_of_when.S AS DATE) AS as_of_when,
item.user_hashes.SS AS user_hashes,
item.user_id_hashes.SS AS user_id_hashes,
item.accumulated_count.N AS accumulated_count,
item.today_count.N AS today_count
FROM items
In the example above I've also flattened the data type keys (S, SS, N) and I converted the date string to a date.
I'm trying to convert a CSV (or XLS) file into JSON. I've come across a number of closed/resolved posts about this, but I am yet to find a solution to what I'm specifically trying to do - which is as follows.
If a column is called 'Colour' then I want to export a single JSON property, but if the column is called 'Features_Count' then I want to create a 'Features' property that is an object instead of a string, and that object contains a property called 'Count'. In other words, I want to be able to have nested properties. The JSON should therefore look something like this:
[
{
"id": 1,
"colour": "blue",
"features":{
"count": 1
}
}
{
"id": 2,
"colour": "red",
"features":null
}
]
Does anyone have any ideas how this can be done? Please do bear in mind I'm pretty much a beginner when it comes to this...
I have a large JSON file with many objects with many properties. Simplified structure looks like this:
"allGadgets": [
{
"Model Code": "nokia1",
"Top Category": "Mobile Phones",
"Category": "non-iPhone",
"Brand": "Nokia",
"Device": "1",
"Price": "£ 11"
},
{
"Model Code": "nokia2",
"Top Category": "Mobile Phones",
"Category": "non-iPhone",
"Brand": "Nokia",
"Device": "2",
"Price": "£ 17",
},
{
"Model Code": "nokia3",
"Top Category": "Mobile Phones",
"Category": "non-iPhone",
"Brand": "Nokia",
"Device": "3",
"Price": "£ 10",
}] ... plus a few hundreds more of different brands and models
I'm extracting from this json list of maps a list of Strings for a search panel for the user to look up their device. The Strings are made of two of the values from the json, i.e.: "${item['Brand']} - ${item['Device']}"
Once the user has selected the relevant model from the dropdown search panel, I need to use this string value to give them the price from the json file. The question is how do I achieve that in dart/flutter? If it was html/css, I would have added an extra hidden field of model code and/or price itself and then just made it visible.
In flutter/dart, however, a search panel plugin I found only accepts Strings, which the user selects and which then have to be used to look up the the corresponding price value in the json file.
Complicating the lookup is the fact that my Strings are now composed of two field values with spaces and a hyphen in between so I probably need to convert them back into how they had been prior to the string conversion and then use both for the lookup... which sounds quite convoluted...
Any thoughts on how to solve the above task would be welcome!
What I guess would help a lot is an example - looking up an object using a String (formed from two values from the objects) within a json with many objects. User is presented with a subset of those objects, but just sees a couple of fields from them. Then the user effectively selects a query using the String shown to him based off the two fields. That String then allows to look up the object and find another value (price) in that corresponding object...
Having decoded your json, you have a List of Maps. Make a new data structure which is a Map of Maps (i.e. Map<String, Map<String, dynamic>>). Populate the new Map by adding each member of the List, keyed by the brand/device name. Now you can directly look up the device details by that composite name.
List<Map<String, dynamic>> original;
Map<String, Map<String, dynamic>> data = {};
original.forEach((item) {
String brandDeviceName = '${item['Brand']} - ${item['Device']}';
data[brandDeviceName] = item;
});
As far as I know, Apache spark requires json file to have one record in exactly one string. I have a splitted by fields json file like this:
{"id": 123,
"name": "Aaron",
"city": {
"id" : 1,
"title": "Berlin"
}}
{"id": 125,
"name": "Bernard",
"city": {
"id" : 2,
"title": "Paris"
}}
{...many more lines
...}
How can I parse it using Spark? Do I need a preprocessor or can I provide custom splitter?
Spark uses splitting by newline to distinguish records. This mean that when using the standard json reader you would need to have one record per line.
You can convert by doing something like in this answer: https://stackoverflow.com/a/30452120/1547734
The basic idea would be to read as a wholeTextFiles and then load it to a json reader which would parse it and flatmap the results.
Of course this assumes the files are big enough to be in memory and parsed one at a time. Otherwise you would need more complicated solutions.
i have a big json object with a list of "tickets". schema looks like below
{
"Artist": "Artist1",
"Tickets": [
{
"Id": 1,
"Attr2Array": [
{
"Att41": 1,
"Att42": "A",
"Att43": null
},
{
"Att41": 1,
"Att42": "A",
"Att43": null
},
],
.
.
.
(more properties)
"Price": "20",
"Description": "I m a ticket"
},
{
"Id": 4,
"Attr2Array": [
{
"Att41": 1,
"Att42": "A",
"Att43": null
},
{
"Att41": 1,
"Att42": "A",
"Att43": null
},
],
.
.
.
.
(more properties)
"Price": "30",
"Description": "I m a ticket"
}
]
}
each item in the list has around 25-30 properties (some simple types, and others complex array as nested objects)
i have to read the object from an api endpoint and extract only "ID" and "Description" but they need to be sorted by "Price" which is an int for example
In what order shall i proceed with this data manipulation
Shall i use the json object, deserialised it into another object with just those 2 properties (which i need) and THEN perform sort "asc" on the "Price"?
Please note that after i have the sorted list i will have to convert it back to a json list because the front end consumes a json after all.
What i dont like about this approach is the cycle of serialisation and deserialisation that happens
or
I perform a sort on the json object first (using for example a binary/bubble sort) and then use the object to create a strongly typed (deserialised) object with just those 2 properties and then serialise it back to pass to the front end
I dont know how performant the bubble sort will be and if at all i will get any gain in performance for large chunks of data processing.
I also need to keep in mind that this implementation can take into account other properties like "availabilitydate" because at a later date, this front end could add one more filter like "availabilitdate" asc
any help is much appreciated
thanks
You can deserialize your JSON string (or file) using the Microsoft System.Web.Extensions and JavaScriptSerializer.DeserializeObject.
First, you must have classes associated to your JSON. To create classes, select your JSON sample data and, in Visual Studio, go to Edit / Paste Special / Paste JSON As Classes.
Next, use this sample to deserialize a JSON string to typed objects, and to sort all Tickets by Price property using Linq.
String json = System.IO.File.ReadAllText(#"C:\Data.json");
var root = new System.Web.Script.Serialization.JavaScriptSerializer().Deserialize<Rootobject>(json);
var sortedTickets = root.Tickets.OrderBy(t => t.Price);