I want to return a psql table, but I want to return it in json format.
Let's say the table looks like this...
id
name
value
1
joe
6
2
bob
3
3
joey
2
But I want to return it as an object like this...
{
"1": {
"name": "joe",
"value": 6
},
"2": {
"name": "bob",
"value": 3
},
"3": {
"name": "joey",
"value": 2
}
}
So if I were doing this with pandas and the table existed as a dataframe, I could transform it like this...
df.set_index('id').to_dict('index')
But I want to be able to do this inside the psql code.
The closest I've gotten is by doing something like this
select
json_build_object (
id,
json_build_object (
'name', name,
'value', value
)
)
from my_table
But instead of aggregating this all into one object, the result is a bunch of separate objects separated by rows at the key level... that being said, it's kinda the same idea...
Any ideas?
You want jsonb_object_agg() to get this:
select jsonb_object_agg(id, jsonb_build_object('name', name, 'value', value))
from my_table
But this is not going to work well for any real-world sized tables. There is a limit of roughly 1GB for a single value. So this might fail with an out-of-memory error with larger tables (or values inside the columns)
Related
I have a table where each row has a json column named data_object. I want to get the data_object for each row and create a json array that represents all rows in the table.
Example:
id | data_object
----------------
1 | { "someKey": "someValue", etc. }
2 | { "someKey": "someOtherValue", etc. }
Desired Result:
[
{ "someKey": "someValue", etc. },
{ "someKey": "someOtherValue", etc. }
]
Some of the keys per row are known (a standard set), but some of the the keys per row will vary, I just want a json array that represents all rows in the table for the data_object json column.
I appreciate your help in advance!
You can use
select array_agg(data_object) from table_name;
jsonb_agg is what you're looking for:
SELECT jsonb_agg(data_object) FROM mytable
Demo: db<>fiddle
Consider the following JSON object,
[
{
"id": 5964460916832,
"name": "Size",
"value": "Small",
"position": 1,
"product_id": 4588516409440
},
{
"id": 5964460916833,
"name": "Size",
"value": "Medium",
"position": 2,
"product_id": 4588516409440
},
{
"id": 5964460916834,
"name": "Size",
"value": "Large",
"position": 3,
"product_id": 4588516409440
}
]
This is a value present in a table field called custom_attrs of JSON data type in a MySQL 8.0 table. I wanted to search the JSON data to match with multiple fields in the same object.
For example,
I wanted to see if there's a match for name "Size" and value "Medium" within the same object. It should not match the name in the first object and the value in the second object.
While we can always use JSON table, I don't prefer that due to the complexities it brings during the JOINs.
JSON_SEARCH supports LIKE operator, but it cannot ensure if it's from the same object
JSON_CONTAINS supports multiple fields but not LIKE as follows,
SET #doc = CAST('[{"id":5964460916832,"name":"Size","value":"Small","position":1,"product_id":4588516409440},{"id":5964460916833,"name":"Size","value":"Medium","position":2,"product_id":4588516409440},{"id":5964460916834,"name":"Size","value":"Large","position":3,"product_id":4588516409440}]' AS JSON);
SELECT JSON_CONTAINS(#doc, '{"name":"Size", "value":"Small"}')
Is there any way to get the same JSON_CONTAINS like functionality with partial search like, {"name":"Size", "value":"%sma%"}
Any help on this would be greatly helpful.
JSON_CONTAINS() only works with equality, not with pattern matching.
The JSON_TABLE() function is the solution intended to address the task you are trying to do. But you said you don't want to use it.
You can simulate JSON_TABLE() using other functions.
select * from (
select
json_unquote(json_extract(col, concat('$[',n.i,'].id'))) as `id`,
json_unquote(json_extract(col, concat('$[',n.i,'].name'))) as `name`,
json_unquote(json_extract(col, concat('$[',n.i,'].value'))) as `value`
from (select #doc as col) j
cross join (select 0 as i union select 1 union select 2 union select 3 union select 4 union select 5 ...) as n
) as t
where t.`id` is not null
order by id, `name`;
Output:
+---------------+------+--------+
| id | name | value |
+---------------+------+--------+
| 5964460916832 | Size | Small |
| 5964460916833 | Size | Medium |
| 5964460916834 | Size | Large |
+---------------+------+--------+
You could then easily add a condition like AND value LIKE '%sma%'.
As you can see, this query is even more complex than if you had used JSON_TABLE().
Really, any solution is going to be complex when you store your data in JSON format, then try to use SQL expressions and relational operations to query them as if they are normalized data. This is because you're practically implementing a mini-database within the functions of a real database. This is sometimes called the Inner-Platform Effect:
The inner-platform effect is the tendency of software architects to create a system so customizable as to become a replica, and often a poor replica, of the software development platform they are using. This is generally inefficient and such systems are often considered to be examples of an anti-pattern.
If you want simple queries, you should store data in normal rows and columns, not in JSON. Then you could get your result using quite ordinary SQL:
SELECT id, name, value FROM MyTable WHERE name = 'Size' AND value LIKE '%sma%';
I am trying to fetch the count the number of records from a Snowflake without certain keys in the json column of that particular record.
Here’s how the snowflake table looks like :
EMP_ID|DEPARTMENT_NAME|EID|DETAILS
EMP10001 | Finance |10008918 |{
"name": "Alec George",
"Year_Joined": "2013",
"Ready_to_transfer": "no",
"Ready_to_permanently_WFH": "yes",
}
Now I want to count records that doesn’t have have the keys that start with Ready_ in the details column of the snowflake table and group counts by the Department_Name.
Note : There can be multiple keys that start with Ready_ in the details.
Currently what’s happening is my count query is returning records where keys start with Ready_ is also listed.
You can flatten to get all the keys, then for each record you can count the number of keys that start with your desired string:
with data as (
select $1 emp_id, $2 dep, $3 eid, parse_json($4) details
from (values
('EMP10001','Finance', 10008918, '{
"name": "Alec George",
"Year_Joined": "2013", "Ready_to_transfer": "no", "Ready_to_permanently_WFH": "yes", }')
,('EMP10002','Finance', 10008918, '{
"name": "Alex George",
"Year_Joined": "2013", }')
)
)
select seq, count_if(detail.key like 'Ready_%') how_many_ready
from data, table(flatten(details)) detail
group by 1
;
Then you only need to count the # of elements that have a count > 0.
I have recently started working with a company who sends me data via JSON, the JSON looks like this:
[{
"name": "company1",
"dataset": null,
"data": [{
"x": "2015-01-01T00:00",
"y": 182
},
{
"x": "2015-01-02T00:00",
"y": 141
}
]
},
{
"name": "company2",
"dataset": null,
"data": [{
"x": "2015-01-01T00:00",
"y": 182
},
{
"x": "2015-01-02T00:00",
"y": 141
}
]
},
{
"name": "company3",
"dataset": null,
"data": [{
"x": "2015-01-01T00:00",
"y": 182
},
{
"x": "2015-01-02T00:00",
"y": 141
}
]
}
]
I get 57 of these daily (Almost identical with the only difference being that the Y value changes accoridng to which metric it is) one for each metric tracked by the company. As you can see the way in which they've written the JSON (X & Y Key value pairs) make it rather hard to store nicely.
I've made 57 tables in MySQL, one for each JSON that inserts the values for that specific metric however querying to get all activity for a day takes a LONG time due to the amount of joins.
I'm hoping one of you might be able to tell me the best way in whihc to insert this into a mysql db table for where I end up either 1 table containing all 57 values or the best way to query across 57 tables without waiting hours for mysql to load it.
this is a personal project for my own business so funds are tight and I am doing what I can at the moment - sorry if this sounds ridiculous!
If I were to be required to store this data, I would personally be inclined to use a table for all the results, with a company table holding the 'master' information about each company.
The company table would be structured like this:
id INT NOT NULL AUTO_INCREMENT,
name VARCHAR(50) -- Arbitrary size - change as needed
The company_update table would be structured like this:
company_update_id INT NOT NULL AUTO_INCREMENT,
company_id INT NOT NULL,
update_timestamp DATETIME,
update_value INT -- This may need to be another type
There would be a foreign key between company_update.company_id to company.company_id.
When receiving the JSON update:
Check that the name exists in the company table. If not, create it.
Get the company's unique ID (for use in the next step) from the company table.
For each item in the data array, add a record to company_update using the appropriate company ID.
Now in order to get results for all companies, I would just use a query like:
SELECT c.name,
cu.update_timestamp,
cu.update_value
FROM company c
INNER JOIN company_update cu ON cu.company_id = c.company_id
ORDER BY c.name, cu.update_timestamp DESC
Note that I have made the following assumptions which you may need to address:
The company name size is at most 50 characters
The y value in the data array is an integer
I have the following PostgreSQL table:
CREATE TABLE orders
(
id uuid NOT NULL,
order_date timestamp without time zone,
data jsonb
);
Where data contains json documents like this:
{
"screws": [
{
"qty": 1000,
"value": "Wood screw"
},
{
"qty": 500,
"value": "Drywall screw"
},
{
"qty": 500,
"value": Concrete screw"
}
],
"nails": [
{
"qty": 1000,
"value": "Round Nails"
}
]
}
How do I can get an overall quantity for all types of screws across all orders? Something like this :)
select value, sum(qty) from orders where section = 'screws' group by value;
I am not quite sure why you are trying to sum up the qty values, because the GROUP BY value makes only sense if there would be several times the same value which can be summed, e.g. if you would have twice the value Wood screw.
Nevertheless, this would be the query:
step-by-step demo:db<>fiddle
SELECT
elems ->> 'value' AS value,
SUM((elems ->> 'qty')::int) AS qty
FROM
orders,
jsonb_array_elements(data -> 'screws') elems
GROUP BY 1
Expand the screw array into one row per array element by jsonb_array_elements()
Get the qty value by the ->> operator (which gives out a text type) and cast this into type int
If really necessary, aggregate these key/value pairs.