I have a csv table which I converted into a json and want to parse this json in flutter based on conditional statements.
For example, first condition would be if the type is a planet and if the name matches the provided value then show the rest of the details in a list view like Charcs, Color and Distance.
Input would be just the type and name, so based on that I need to parse the json and extract the correct data.
Either I can have like 6 separate jsons for each type but it does not seem to be a good idea to me.
Any suggestions on how I can use conditional statements or maybe convert this json into a different format so that I can parse using indexes or elements in Flutter.
[
{
"Type": "Planet",
"Name": "Jupiter",
"Charcs": "Largest",
"Color": "blue",
"distance": "near"
},
{
"Type": "Planet",
"Name": "Earth",
"Charcs": "Inhabited",
"Color": "green",
"distance": "nearby"
},
{
"Type": "Planet",
"Name": "Mars",
"Charcs": "Red Planet",
"Color": "red",
"distance": "very near"
},
{
"Type": "Star",
"Name": "Polaris",
"Charcs": "abcddw",
"Color": "casc",
"distance": "far"
},
{
"Type": "Star",
"Name": "Sirius",
"Charcs": "qwqwd",
"Color": "asfaf",
"distance": "very far"
},
{
"Type": "Star",
"Name": "Betelgeuse",
"Charcs": "qwdwqeq",
"Color": "asfasfa",
"distance": "far far away"
}
]
Because this is an array, you can use a loop that filters each item into a separate JSON key.
var array = [
{
"Type": "Planet",
"Name": "Jupiter",
"Charcs": "Largest",
"Color": "blue",
"distance": "near"
},
{
"Type": "Planet",
"Name": "Earth",
"Charcs": "Inhabited",
"Color": "green",
"distance": "nearby"
},
{
"Type": "Planet",
"Name": "Mars",
"Charcs": "Red Planet",
"Color": "red",
"distance": "very near"
},
{
"Type": "Star",
"Name": "Polaris",
"Charcs": "abcddw",
"Color": "casc",
"distance": "far"
},
{
"Type": "Star",
"Name": "Sirius",
"Charcs": "qwqwd",
"Color": "asfaf",
"distance": "very far"
},
{
"Type": "Star",
"Name": "Betelgeuse",
"Charcs": "qwdwqeq",
"Color": "asfasfa",
"distance": "far far away"
}
]
var json = {}
array.forEach(e => {
json[e.Type] ||= {}
json[e.Type][e.Name] = {Charcs: e.Charcs,Color: e.Color,distance:e.distance}
})
console.log(json)
The definition of the code above is the following:
json[e.Type] ||= {}: This creates a key from the object type if it does not already exist
json[e.Type][e.Name] = {Charcs: e.Charcs,Color: e.Color,distance:e.distance}: This puts the data of the object in its own sub-key
I have a file that has multiple nested jsons, I would like topush to redshift, i figured to start off with I can just save the entire row as one column string then save
I am new to both spark and scala so excuse the ignorance please, but it seems that reading the file as json will automatically infer a schema which I cant save to redshift because the connector im using cant save the schema for nested jsons
So my question is this, how can I read in a file that has a nested json, save each row completely in a table with one column "message"
If it helps here is a sample json that I want to break down, maybe to different tables?
Assuming I have a sample json as
{
"OrderId": "foo",
"LegacyOrderId": 123,
"NotifiyCustomer": true,
"OrderResolutionStatus": "OkByBox",
"CustomerId": 123,
"RestaurantId": 132111,
"OrderContainer": {
"Id": "foo",
"LegacyId": 123,
"ApplicationInfo": {
"UserAgent": "Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13F69 Safari/601.1",
"ClientIp": "0.0.0.0",
"JeFeature": "sadsad",
"ApplicationName": "Resadasdsab",
"ApplicationVersion": "1.0.10444.0"
},
"Order": {
"foo": "Fasdsaasdsdse. ",
"foo": "asdasdsads",
"fooDate": "2016-07-06T10:58:40.3579409+00:00",
"InitialDueDate": "2016-07-06T12:10:00+00:00",
"InitialDueDateWithUtcOffset": "2016-07-06T13:10:00+01:00",
"DueDate": "2016-07-06T12:10:00+00:00",
"DueDateWithUtcOffset": "2016-07-06T13:10:00+01:00",
"foo": true,
"foo": true
},
"RestaurantInfo": {
"Id": "34421159",
"Name": "Casdadasdase",
"PhoneNumber": "0fdsfddsf2",
"AddressLines": ["5asdasddasad"],
"City": "London",
"Postcode": "SE1 3BR",
"Longitude": 0.012155,
"Latitude": 51.920357,
"DispatchMethod": "JctV3",
"SeoName": "caadssadasdas6",
"TempOffline": false,
"Offline": false
},
"PaymentInfo": {
"OrderId": "foo",
"PaymentLines": [{
"Type": "Cash",
"Value": 18.8,
"CardFee": 0.0
}],
"DriverTipValue": 0.0,
"Total": 18.8,
"TotalComplementary": 0.0,
"PaidDate": "2016-07-06T10:58:51.7863535+00:00"
},
"CustomerInfo": {
"Id": "7604asdsa879",
"Email": "oasdasd#rest.com",
"Name": "Lysadsadasi",
"Address": "3sadsadsadsdsaasde",
"City": "London",
"Postcode": "E11 3sG",
"PhoneNumber": "asdsads",
"TimeZone": "GMT Standard Time",
"PreviousJeOrderCount": 22,
"PreviousRestuarantOrderCount": 0
},
"BasketInfo": {
"BasketId": "asdsdas",
"MenuId": 119949,
"Items": [{
"ProductId": 1392310,
"ProductTypeId": 366,
"MenuCardNumber": "96",
"Name": "Casdasdsaef",
"Description": "In batter<br />\r\n<strong>Spicy</strong>",
"Synonym": "",
"UnitPrice": 52.5,
"CombinedPrice": 52.5,
"MealParts": [],
"OptionalAccessories": [],
"RequiredAccessories": [],
"Discounts": [],
"MultiBuyDiscounts": []
}, {
"ProductId": 12233,
"ProductTypeId": 44218,
"MenuCardNumber": "418",
"Name": "Aasdsaddsadsadsadsa",
"Description": "",
"Synonym": "",
"UnitPrice": 7.5,
"CombinedPrice": 7.5,
"MealParts": [],
"OptionalAccessories": [],
"RequiredAccessories": [],
"Discounts": [],
"MultiBuyDiscounts": []
}, {
"ProductId": 13917482,
"ProductTypeId": 365,
"MenuCardNumber": "129",
"Name": "Szasdsadsaadssad",
"Description": "asdsaddsadssaddas",
"Synonym": "",
"UnitPrice": 5.8,
"CombinedPrice": 5.8,
"MealParts": [],
"OptionalAccessories": [],
"RequiredAccessories": [],
"Discounts": [],
"MultiBuyDiscounts": []
}],
"Discounts": [],
"SubTotal": 18.8,
"ToSpend": 0.0,
"MultiBuyDiscount": 0.0,
"Discount": 0.0,
"DeliveryCharge": 0.0,
"Total": 18.8
},
"IsLocked": true
},
"Id": "c35d48c1-268f-4e46-81d9-700afc22f2d9",
"TimeStamp": "2016-07-06T10:59:22.982145Z",
"RaisingComponent": "Leasdsadasdasw",
"Tenant": "uk"
}
You're correct that Redshift requires the JSON structure to be isomorphic to the top level keys in the JSON source. There are a number of other limitations, e.g., can't load nested arrays, can't load more than 2MB per file, etc.
My suggestion (and the solution I have just implemented personally) is:
Load the data into a Spark dataframe sqlContext.read.format("com.databricks.spark.csv")
Let Spark infer the schema if needed .option("inferSchema", "true")
Manipulate it as a sequence of dataframes val nextDf = df.[filter|explode|select|…etc]
Write the final flat dataframe(s) to Redshift finalDf.write.format("com.databricks.spark.redshift").option(…
I am using QuickBooks Online Plus, which means i can edit the item. I would like to use the API to change the on hand # for an item. I've created an item "test1" through QuickBooks Online Plus. When i read the item from API i got the following in the attached json file. things looks great as follow:
{
"QueryResponse": {
"Item": [
{
"Name": "test1",
"Active": true,
"FullyQualifiedName": "test1",
"Taxable": false,
"UnitPrice": 3,
"Type": "Inventory",
"IncomeAccountRef": {
"value": "60",
"name": "Sales of Product Income"
},
"PurchaseCost": 1,
"ExpenseAccountRef": {
"value": "61",
"name": "Cost of Goods Sold"
},
"AssetAccountRef": {
"value": "62",
"name": "Inventory Asset"
},
"TrackQtyOnHand": true,
"QtyOnHand": 6,
"InvStartDate": "2015-12-02",
"domain": "QBO",
"sparse": false,
"Id": "19",
"SyncToken": "1",
"MetaData": {
"CreateTime": "2015-12-01T14:38:23-08:00",
"LastUpdatedTime": "2015-12-01T14:38:42-08:00"
}
}
],
"startPosition": 1,
"maxResults": 1
},
"time": "2015-12-02T09:59:29.936-08:00"
}
But when i tried to update that item using the json object below:
{
"Name": "test1",
"Active": true,
"Taxable": false,
"UnitPrice": 3,
"Type": "Inventory",
"IncomeAccountRef": {
"value": "60",
"name": "Sales of Product Income"
},
"PurchaseCost": 1,
"ExpenseAccountRef": {
"value": "61",
"name": "Cost of Goods Sold"
},
"AssetAccountRef": {
"value": "62",
"name": "Inventory Asset"
},
"TrackQtyOnHand": true,
"QtyOnHand": 16,
"InvStartDate": "2015-12-02",
"domain": "QBO",
"sparse": false,
"Id": "19",
"SyncToken": "2"
}
, i got an error like this:
{"Fault":{"Error":[{"Message":"Stale Object Error","Detail":"Stale Object Error : You and seven.li#hotschedules.com were working on this at the same time. seven.li#hotschedules.com finished before you did, so your work was not saved.","code":"5010","element":""}],"type":"ValidationFault"},"time":"2015-12-02T10:42:52.466-08:00"}
I would like to update the on hand quantity. Does anyone know what's wrong and how to do it? Thanks.
The problem is with the "SyncToken". I believe when you try to update the new item, you should not increment the "SyncToken", you should keep it the same. QBO will update the "SyncToken" for you.
I need to create a 3D model of a complex geometry (nuclear physics particle detectors) using ThreeJS.
In one example I have about 100.000 geometries and 4.5 Million logical volumes.
I import the geometry using JSON format but then I'm facing two problems.
the JSON file is huge (3.5 Gb)
obviously the file cannot be imported using the editor available at http://threejs.org/editor/. Probably because of memory problems.
For example, in the JSON file I'm using a new UUID when I position a volume. I'm woundering if I could use the UID of the volume that I have already positioned once whithout redefining all of its childs and this should be enough to reposition the full tree.
Just for clarification I create a small example of JSON file where "Box 1" contains others volumes and then I cloned "Box 1" and repositioned it inside "Container 1". You can see that that all childs of "Box 1" have also been copied (indeed only information about mesh.
{
"metadata": {
"version": 4.3,
"type": "Object",
"generator": "ObjectExporter"
},
"geometries": [
{
"uuid": "0F1A7764-83AC-41AE-BB49-AE74A475CF80",
"type": "BoxGeometry",
"width": 500,
"height": 500,
"depth": 500,
"widthSegments": 1,
"heightSegments": 1,
"depthSegments": 1
},
{
"uuid": "E69365CA-206A-4286-BB0D-940A804B7DEB",
"type": "TorusGeometry",
"radius": 100,
"tube": 40,
"radialSegments": 16,
"tubularSegments": 64,
"arc": 6.28
},
{
"uuid": "B2B3E2FD-17C5-42B3-8732-0918F4196199",
"type": "SphereGeometry",
"radius": 75,
"widthSegments": 32,
"heightSegments": 16,
"phiStart": 0,
"phiLength": 6.283185307179586,
"thetaStart": 0,
"thetaLength": 3.141592653589793
},
{
"uuid": "CCB3F065-632D-4AD4-AB16-6DC65D3F9163",
"type": "CylinderGeometry",
"radiusTop": 20,
"radiusBottom": 20,
"height": 100,
"radialSegments": 32,
"heightSegments": 1,
"openEnded": false
},
{
"uuid": "D3905F27-8D63-447B-A4D2-D4901FAB00E2",
"type": "BoxGeometry",
"width": 100,
"height": 100,
"depth": 100,
"widthSegments": 1,
"heightSegments": 1,
"depthSegments": 1
},
{
"uuid": "EE1DB156-C18F-41B1-B1B5-E4B2619F6FC2",
"type": "BoxGeometry",
"width": 100,
"height": 100,
"depth": 100,
"widthSegments": 1,
"heightSegments": 1,
"depthSegments": 1
}],
"materials": [
{
"uuid": "6B15E58A-F80D-4BC4-A1DE-185A846B6BBD",
"type": "MeshPhongMaterial",
"color": 16777215,
"emissive": 0,
"specular": 1118481,
"shininess": 30,
"side": 2,
"opacity": 0.5,
"transparent": true
},
{
"uuid": "BD2913CC-3A1F-45A7-8498-F90988426445",
"type": "MeshLambertMaterial",
"color": 16777215,
"emissive": 16723858
},
{
"uuid": "135C8BD4-895B-42C8-A72D-3FB356248D7A",
"type": "MeshLambertMaterial",
"color": 16777215,
"emissive": 63744
},
{
"uuid": "033485A5-CADB-4240-B01B-8EF94148626C",
"type": "MeshLambertMaterial",
"color": 16777215,
"emissive": 9605120
},
{
"uuid": "DD696C09-0225-446A-A040-CA9917C49D6A",
"type": "MeshLambertMaterial",
"color": 16777215,
"emissive": 275455
},
{
"uuid": "7E04EF43-1516-433E-ABD1-42E303AA0F19",
"type": "MeshLambertMaterial",
"color": 16777215,
"emissive": 9705297
}],
"object": {
"uuid": "6BE54385-5498-42D2-B3D4-2377AE039F70",
"type": "Scene",
"name": "Scene",
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1],
"children": [
{
"uuid": "C900FDBD-6DC9-41FD-8536-65AAEFD4CC13",
"type": "Mesh",
"name": "Box 1",
"geometry": "0F1A7764-83AC-41AE-BB49-AE74A475CF80",
"material": "6B15E58A-F80D-4BC4-A1DE-185A846B6BBD",
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,0,0,-44.368385314941406,1],
"children": [
{
"uuid": "E04152A2-AA7D-4D36-AAF7-63CF6A109D42",
"type": "Mesh",
"name": "Torus 2",
"geometry": "E69365CA-206A-4286-BB0D-940A804B7DEB",
"material": "BD2913CC-3A1F-45A7-8498-F90988426445",
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,16.59000015258789,0,108,1]
},
{
"uuid": "37414A51-B50E-4944-AFC5-A15EA7B0AF58",
"type": "Mesh",
"name": "Sphere 3",
"geometry": "B2B3E2FD-17C5-42B3-8732-0918F4196199",
"material": "135C8BD4-895B-42C8-A72D-3FB356248D7A",
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,0,0,-169.27000427246094,1]
},
{
"uuid": "6B0D91CC-C760-4311-A2BF-7A83C2E86306",
"type": "Mesh",
"name": "Cylinder 4",
"geometry": "CCB3F065-632D-4AD4-AB16-6DC65D3F9163",
"material": "033485A5-CADB-4240-B01B-8EF94148626C",
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1]
}]
},
{
"uuid": "280B1F69-8E93-442E-A49D-42BF45004313",
"type": "PointLight",
"name": "PointLight 1",
"color": 16777215,
"intensity": 1,
"distance": 0,
"decay": 1,
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,-165.60302734375,0,-170.2801971435547,1]
},
{
"uuid": "44E76258-B0C2-43B1-99FB-0BE25D31972A",
"type": "Mesh",
"name": "Container 1",
"geometry": "D3905F27-8D63-447B-A4D2-D4901FAB00E2",
"material": "DD696C09-0225-446A-A040-CA9917C49D6A",
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,501.0995178222656,0,-383.1794738769531,1],
"children": [
{
"uuid": "A9FE3E04-1AE7-4A13-ADAF-706CE9361109",
"type": "Mesh",
"name": "Box 1",
"geometry": "0F1A7764-83AC-41AE-BB49-AE74A475CF80",
"material": "6B15E58A-F80D-4BC4-A1DE-185A846B6BBD",
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,0,0,-111.47900390625,1],
"children": [
{
"uuid": "3C7CB934-95EF-4327-9595-52A64AE40D2D",
"type": "Mesh",
"name": "Torus 2",
"geometry": "E69365CA-206A-4286-BB0D-940A804B7DEB",
"material": "BD2913CC-3A1F-45A7-8498-F90988426445",
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,16.59000015258789,0,108,1]
},
{
"uuid": "7B2F4D0F-D8C3-46EF-9600-EF39D9DFE590",
"type": "Mesh",
"name": "Sphere 3",
"geometry": "B2B3E2FD-17C5-42B3-8732-0918F4196199",
"material": "135C8BD4-895B-42C8-A72D-3FB356248D7A",
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,0,0,-169.27000427246094,1]
},
{
"uuid": "D6741C5F-2B08-432A-B671-11F538EC73DF",
"type": "Mesh",
"name": "Cylinder 4",
"geometry": "CCB3F065-632D-4AD4-AB16-6DC65D3F9163",
"material": "033485A5-CADB-4240-B01B-8EF94148626C",
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1]
}]
}]
},
{
"uuid": "426C5B44-3DA4-4451-81B9-9CA53780411E",
"type": "Mesh",
"name": "Container 2",
"geometry": "EE1DB156-C18F-41B1-B1B5-E4B2619F6FC2",
"material": "7E04EF43-1516-433E-ABD1-42E303AA0F19",
"matrix": [1,0,0,0,0,1,0,0,0,0,1,0,488.5225524902344,0,548.1283569335938,1]
}]
}
}
This issue has been raised in the past by BuschnicK
I repeat the question here because if I use the clone operation, this created a copy in memory of the volume and its childs. Althaugh geometries and materials are copied but only the mesh and and positioning, it created a huge memory problem.
I would like to know if there is a way to position a volume by creating only a link to that volume whithout copying in memory all the information about the volume and its childs. This would save space on disk by a factor of 500-1000 and the same savings in memory.
I would appreciate any suggestion,
Thank you in advance,
Ilias
var temp = Box1.children;
Box1.children = [];
var newBox = Box1.clone();
Box1.children = temp;
??