Setting up postgres database for recipes - json

I am new to PostgreSQL and trying to plan a database that will allow me to query recipes based on id, ingredients etc from a provided dataset, example below.
I am getting a bit thrownoff by the nested ingredients and how to lay out my tables.
I was initially thinking two tables one for instructions and image and another for the ingredients.
i was now wondering if i am on the right lines and if so how to loop through the nested ingredients to produce a useful second table
enter code here
[ {
"id": "recipe-88",
"imageUrl": "http://www.images.com/12",
"instructions": "blend with oat milk and ice, sprinkle with
salt",
"ingredients": [
{ "name": "blueberries", "grams": 114 },
{ "name": "coffee", "grams": 20 },
{ "name": "kale", "grams": 48 }
]
},
{
"id": "recipe-74",
"imageUrl": "http://www.images.com/2",
"instructions": "crush ingredients with mortar and pestle, mix
with whole milk, serve in bowl",
"ingredients": [
{ "name": "coffee", "grams": 25 },
{ "name": "lime", "grams": 140 },
{ "name": "strawberries", "grams": 3 },
{ "name": "apricots", "grams": 24 },
{ "name": "kale", "grams": 50 }
]
},
{
"id": "recipe-77",
"imageUrl": "http://www.images.com/25",
"instructions": "blend with oat milk and ice, sprinkle with
salt",
"ingredients": [
{ "name": "coconut", "grams": 14 },
{ "name": "coconut", "grams": 57 },
{ "name": "lime", "grams": 153 },
{ "name": "oat milk", "grams": 31 }
]
},]

I have parsed the json you provided and loaded it in a table.
Check the dbfiddle link.
Here is the parsed content loaded into a table.
postgres=# select * from parsedjj;
id | recipe_id | imageurl | instructions | ingredients
----+-----------+-------------------------+-------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 | recipe-88 | http//www.images.com/12 | blend with oat milk and ice, sprinkle with salt | [{"name": "blueberries", "grams": 114}, {"name": "coffee", "grams": 20}, {"name": "kale", "grams": 48}]
2 | recipe-74 | http//www.images.com/2 | crush ingredients with mortar and pestle, mix with whole milk, serve in bowl | [{"name": "coffee", "grams": 25}, {"name": "lime", "grams": 140}, {"name": "strawberries", "grams": 3}, {"name": "apricots", "grams": 24}, {"name": "kale", "grams": 50}]
3 | recipe-77 | http//www.images.com/25 | blend with oat milk and ice, sprinkle with salt | [{"name": "coconut", "grams": 14}, {"name": "coconut", "grams": 57}, {"name": "lime", "grams": 153}, {"name": "oat milk", "grams": 31}]
(3 rows)
Note: I had changed the JSON key value from "imageUrl" to "image" because possibly "url" which is a keyword for postgres might have been impeding the parsing.

Related

Selecting in JQ with Contains in a Array

I want to select the particular item from the array using contains and get the first item using JQ.
JQ:
.amazon.items[] | select(.name | contains ("shoes"))
JSON:
{
"amazon": {
"activeitem": 2,
"items": [
{
"id": 1,
"name": "harry potter",
"state": "sold"
},
{
"id": 2,
"name": "adidas shoes",
"state": "in inventory"
},
{
"id": 3,
"name": "watch",
"state": "returned"
},{
"id": 4,
"name": "adidas shoes",
"state": "in inventory"
}
]
}
}
Expected Result:
{
"activeitem": 2,
"item": {
"id": 2,
"name": "adidas shoes",
"state": "in inventory"
}
}
Actual :
Tried various options like but not getting the Intended response .
.amazon.items[] | select(.name | contains ("shoes"))
.amazon.items | select(.[].name | contains ("shoes")) | .[0]
Also when I try to combine activeitem and item, I get something like this, which is also wrong.
{
"activeitem": 2,
"item": {
"id": 2,
"name": "adidas shoes",
"state": "in inventory"
}
},
{
"activeitem": 2,
"item": {
"id": 2,
"name": "adidas shoes",
"state": "in inventory"
}
}
To edit "in-place" you could write:
.amazon
| .items |= map(select(.name | contains ("shoes")))[0]
If you really want to change the name 'items' to 'item', you could tweak the above as follows:
.amazon
| .item = (.items | map(select(.name | contains ("shoes")))[0])
| del(.items)

How to insert fields into a JSON file using jq?

Let's say I have a JSON file recipe.json containing this:
{
"name": "Curried Lentils and Rice",
"ingredients": [
{
"quantity": "1 quart",
"name": "beef broth",
"type": "Misc"
},
{
"quantity": "1 cup",
"name": "dried green lentils",
"type": "Misc"
},
{
"quantity": "1/2 cup",
"name": "basmati rice",
"type": "Misc"
},
{
"quantity": "1 tsp",
"name": "curry powder",
"type": "Condiments"
},
{
"quantity": "1 tsp",
"name": "salt",
"type": "Condiments"
}
],
"steps": [
"Bring broth to a low boil.",
"Add curry powder and salt.",
"Cook lentils for 20 minutes.",
"Add rice and simmer for 20 minutes.",
"Enjoy!"
],
"timers": [
0,
0,
20,
20,
0
],
"imageURL": "http://dagzhsfg97k4.cloudfront.net/wp-content/uploads/2012/05/lentils3.jpg"
}
And I'm trying to insert new JSON fields into the file at specific positions in the file, using jq such as:
"cuisine": "meditaranean"
will become the 2nd entry, and
"meal": "lunch"
will become the 4th entry. So the file after the command is like this:
{
"name": "Curried Lentils and Rice",
"cuisine": "meditaranean"
"ingredients": [
{
"quantity": "1 quart",
"name": "beef broth",
"type": "Misc"
},
{
"quantity": "1 cup",
"name": "dried green lentils",
"type": "Misc"
},
{
"quantity": "1/2 cup",
"name": "basmati rice",
"type": "Misc"
},
{
"quantity": "1 tsp",
"name": "curry powder",
"type": "Condiments"
},
{
"quantity": "1 tsp",
"name": "salt",
"type": "Condiments"
}
],
"meal": "lunch"
"steps": [
"Bring broth to a low boil.",
"Add curry powder and salt.",
"Cook lentils for 20 minutes.",
"Add rice and simmer for 20 minutes.",
"Enjoy!"
],
"timers": [
0,
0,
20,
20,
0
],
"imageURL": "http://dagzhsfg97k4.cloudfront.net/wp-content/uploads/2012/05/lentils3.jpg"
}
My question is how to do this with jq?
Note: This other question addresses performing updates of single fields, while the current question is about inserts. They are as different as orange and yellow bell peppers, which is different! (Don't make me add a picture of bell peppers to this post, I swear I'll do it if I have to.)
With the help of a helper function, the task becomes trivial:
def insert_kv($key; $value; $ix):
to_entries
| .[0:$ix] + [{key: $key, value: $value}] + .[$ix:]
| from_entries;
insert_kv("cuisine"; "mediterranean"; 1)
| insert_kv("meal"; "lunch"; 3)
You could (alternatively or in addition) define:
def insert_kv($object; $ix):
to_entries
| .[0:$ix] + ($object|to_entries) + .[$ix:]
| from_entries;
Where your json is saved to meals.json, executing the following command will result into exactly what you need.
jq '.cusine="mediteranean"' meals.json

How to extract a JSON field that has a random string with the jq command?

Please bear with me I'm still learning about JSON and programming in general. So I have this JSON file:
{
"root_200888546292191": {
"fields": {
"buyerId": "31392191"
},
"id": "200718546292191",
"tag": "root",
"type": "biz"
},
"shippingInfo_#package#OF04472002179150#WAREHOUSE_ACCEPTED": {
"fields": {
"delivery": {
"createdAt": "Sen 09 Apr - Rab 11 Apr",
"desc": "Standar",
"email": null,
"method": "Standard",
"status": "info"
},
"statusMap": {
"active": "Dalam proses",
"all": ["Dalam proses", "Dalam pengiriman", "Telah diterima"]
},
"trackingList": [{
"info": "Status One",
"updatedAt": "05 Apr 2018 - 11:00"
}, {
"info": "Status Two",
"updatedAt": "05 Apr 2018 - 11:00"
}]
},
"id": "#package#OF04472002179150#WAREHOUSE_ACCEPTED",
"tag": "shippingInfo",
"type": "biz"
},
"shippingInfo_#package#AAAAAAAAAAAAA#NOT_WAREHOUSE_ACCEPTED": {
"fields": {
"delivery": {
"createdAt": "Sen 09 Apr - Rab 11 Apr",
"desc": "Standar",
"email": null,
"method": "Standard",
"status": "info"
},
"statusMap": {
"active": "Dalam proses",
"all": ["Dalam proses", "Dalam pengiriman", "Telah diterima"]
},
"trackingList": [{
"info": "Status Three",
"updatedAt": "05 Apr 2018 - 11:00"
}, {
"info": "Status Four",
"updatedAt": "05 Apr 2018 - 11:00"
}]
},
"id": "#package#AAAAAAAAAAAAA#NOT_WAREHOUSE_ACCEPTED",
"tag": "shippingInfo",
"type": "biz"
},
"login_200718577292191": {
"fields": {
"buyerEmail": "myemail#gmail.com",
"buyerName": "myname"
},
"id": "200718522292191",
"tag": "login",
"type": "biz"
}
}
And I want to extract Info in shippingInfo_ > fields > trackingList So the output that I want is like this:
Status One
Status Two
Status Three
Status Four
The string after shippingInfo_ is always random, how do I extract it with jq?
This is as far as I've got jq '.shippingInfo_*.fields.trackingList.info'
Direct approaches
There are many direct approaches, e.g.:
Using paths
paths as $p
| select( $p|length == 5 and
($p[0] | startswith("shippingInfo_")) and
$p[1:3] == ["fields", "trackingList"] and
$p[4] == "info")
| getpath($p)
Using to_entries
to_entries[]
| select(.key | startswith("shippingInfo_"))
| .value
| .fields.trackingList[]
| .info
Indirect approaches
There are also some indirect approaches that are worth mentioning, e.g.
Using a helper function
def dot(s):
to_entries[] | select(.key|test(s)) | .value ;
dot("^shippingInfo_")
| .fields.trackingList[]
| .info
The last-mentioned filter can be abbreviated to:
dot("^shippingInfo_").fields.trackingList[].info
Relaxed requirements
If it is acceptable to ignore the "^shippingInfo_" requirement, the following may be worth considering as well:
.[].fields.trackingList[]?.info
or even:
.. | objects.fields.trackingList[]?.info

Multi Object JSON containing static and dynamic data

I have more than 1 million rows of data in excel and I want to convert it to JSON so i can visualize it using D3js and other web based applications. Data is comprised of two subsets:
General information of each data points, including geographical location, ID.... (static data not changing once it is written)
Monthly measurements at each data point. This data set updates monthly once new data arrives
This is how data looks like:
ID: 2411976, State: Texas, County: DEWITT, Latitude: 29 Longitude:-96,
Data: 11/1/2013 27.516; 12/1/2013 15.3566; 1/1/2014 27.6418;
2/1/2014 13.45; 3/1/2014 11.21; 4/1/2014 20
ID: 2321771, State: Texas, County: DEWITT, Latitude: 29 Longitude:-96,
Data: 11/1/20134 19; 12/1/2014 21; 1/1/2015 30; 2/1/2015 50; 3/1/2015 10;
4/1/2015 5
.....
Is it possible to wrap all data points in one JSON document that contains both temporal data and static data?
This is indeed possible, as you can represent arrays/objects in a nested structure, like this:
{
"locations": [{
"id": 2411976,
"state": "Texas",
"county": "DEWITT",
"latitude": 29,
"longitude":-96,
"data": [{
"date": "2013-11-01T00:00:00.000Z",
"value": 27.516
}, {
"date": "2013-12-01T00:00:00.000Z",
"value": 15.3566
}, {
"date": "2014-01-01T00:00:00.000Z",
"value": 27.6418
}, {
"date": "2014-02-01T00:00:00.000Z",
"value": 13.45
}, {
"date": "2014-03-01T00:00:00.000Z",
"value": 11.21
}, {
"date": "2014-04-01T00:00:00.000Z",
"value": 20
}]
}, {
"id": 2321771,
"state": "Texas",
"county": "DEWITT",
"latitude": 29,
"longitude":-96,
"data": [{
"date": "2014-11-01T00:00:00.000Z",
"value": 19
}, {
"date": "2014-12-01T00:00:00.000Z",
"value": 21
}, {
"date": "2015-01-01T00:00:00.000Z",
"value": 30
}, {
"date": "2015-02-01T00:00:00.000Z",
"value": 50
}, {
"date": "2015-03-01T00:00:00.000Z",
"value": 10
}, {
"date": "2015-04-01T00:00:00.000Z",
"value": 5
}]
}]
}
This is just one way of doing it. Depending on what the consumer of this data expects as input, you could adapt accordingly.

Using common_schema library not able to parse the json array values

We are using this common_schema library in mysql 5.6 to extract the values from json array. The format is given below. But it returns the NULL value. So, can you please help us out how to parse the json array using common_schema.
select common_schema.extract_json_value('"batter":
[
{ "id": "1001", "type": "Regular" },
{ "id": "1002", "type": "Chocolate" },
{ "id": "1003", "type": "Blueberry" },
{ "id": "1004", "type": "Devils Food" }
]','/id');
Expected output should be saved in table as
id type
1001 Regular
1002 Chocolate
1003 Blueberry
1004 Devils Food
Please let us know how we can achieve this parsing.
Thanks
Kalyan
Directly it seems not so easy to get what you need.
An option to obtain a single value is:
SET #`json` := '
{
"batter":
[
{ "id": "1001", "type": "Regular" },
{ "id": "1002", "type": "Chocolate" },
{ "id": "1003", "type": "Blueberry" },
{ "id": "1004", "type": "Devils Food" }
]
}
';
SELECT
`common_schema`.`extract_json_value`(#`json`,'descendant-or-self::id[1]') `id`,
`common_schema`.`extract_json_value`(#`json`,'descendant-or-self::type[1]') `type`;
+------+---------+
| id | type |
+------+---------+
| 1001 | Regular |
+------+---------+
1 row in set (0,04 sec)