Hey I've been searching and looking all day. Have gotten further into understanding my problem and the options. By I am at a lack of understanding.
My general issue is that I am trying to SELECT rows based on values in the JSON and order the selection by there selected values.
I have a table (elements), with two columns: Person and Tags. Tags contains a JSON array that can have multiple JSON objects inside it. Objects always have "name" and "sort".
+-------------+------------------------------------------------------------------+
| Person | tags
+-------------+------------------------------------------------------------------+
| William | [{"name": "apple", "sort": "1"}, {"name": "orange", "sort": "2"}]
| Anna | [{"name": "apple", "sort": "3"}, {"name": "orange", "sort": "1"}]
| Michael | [{"name": "apple", "sort": "2"}]
+-------------+------------------------------------------------------------------+
Ideally what I would like to do is tell the database using to SELECT * FROM elements WHERE tags (has an object where name is "apple") ORDER by (The sort value from the object where it matched);
So if I gave it "apple" it would list: William, Michael, Anna.
If I gave it "orange" it would list: Anna, William.
I have been looking into having SELECTs within SELECTs (Subqueries) but I can find the right combination with JSON. Below is the closest I have gotten, but I can tell it needs something more advanced?
SELECT *
FROM elements
WHERE JSON_SEARCH( tags, 'one', 'apple', NULL, '$[*].name' ) IS NOT NULL
This will return all the people, with apple tag, but it will not order them based on the sort.
Any suggestions are welcome, thanks in advance.
Here a "nice" Query that works for you. You only must change res.* to res.name for your query.
SELECT res.* FROM (
SELECT
SUBSTRING_INDEX( JSON_UNQUOTE (JSON_SEARCH(e.tags, 'one', 'apple')),'.',1) as idx
,e.* FROM `elements` AS e ) AS res
WHERE res.idx IS NOT NULL
ORDER BY JSON_UNQUOTE(JSON_EXTRACT(res.tags,CONCAT(res.idx,'.sort')));
SAMPLE
mysql> select * from elements;
+----+---------+-------------------------------------------------------------------+
| id | Person | tags |
+----+---------+-------------------------------------------------------------------+
| 1 | William | [{"name": "apple", "sort": "1"}, {"name": "orange", "sort": "2"}] |
| 2 | Anna | [{"name": "apple", "sort": "3"}, {"name": "orange", "sort": "1"}] |
| 3 | Michael | [{"name": "apple", "sort": "2"}] |
+----+---------+-------------------------------------------------------------------+
3 rows in set (0.00 sec)
Find apple
mysql> SELECT res.* FROM (
-> SELECT
-> SUBSTRING_INDEX( JSON_UNQUOTE (JSON_SEARCH(e.tags, 'one', 'orange')),'.',1) as idx
-> ,e.* FROM `elements` AS e ) AS res
-> WHERE res.idx IS NOT NULL
-> ORDER BY JSON_UNQUOTE(JSON_EXTRACT(res.tags,CONCAT(res.idx,'.sort')));
+----+---------+-------------------------------------------------------------------+
| id | Person | tags |
+----+---------+-------------------------------------------------------------------+
| 1 | William | [{"name": "apple", "sort": "1"}, {"name": "orange", "sort": "2"}] |
| 2 | Anna | [{"name": "apple", "sort": "3"}, {"name": "orange", "sort": "1"}] |
| 3 | Michael | [{"name": "apple", "sort": "2"}] |
+----+---------+-------------------------------------------------------------------+
3 rows in set (0.00 sec)
Find orange
mysql> SELECT res.* FROM (
-> SELECT
-> SUBSTRING_INDEX( JSON_UNQUOTE (JSON_SEARCH(e.tags, 'one', 'orange')),'.',1) as idx
-> ,e.* FROM `elements` AS e ) AS res
-> WHERE res.idx IS NOT NULL
-> ORDER BY JSON_UNQUOTE(JSON_EXTRACT(res.tags,CONCAT(res.idx,'.sort')));
+------+----+---------+-------------------------------------------------------------------+
| idx | id | Person | tags |
+------+----+---------+-------------------------------------------------------------------+
| $[1] | 2 | Anna | [{"name": "apple", "sort": "3"}, {"name": "orange", "sort": "1"}] |
| $[1] | 1 | William | [{"name": "apple", "sort": "1"}, {"name": "orange", "sort": "2"}] |
+------+----+---------+-------------------------------------------------------------------+
2 rows in set (0.01 sec)
There's a similar question here which might assist you.
You may be able to use JSON_EXTRACT however it gets complicated because you're storing an array of JSON data so you might also require a sub query.
I'll propose an alternate solution though. Have you considered restructuring your database across multiple tables instead of using a JSON blob?
You could have a table dedicated to storing the tags, something like this:
+-------------+------------------------------------------------------------------+
| tagid | name
+-------------+------------------------------------------------------------------+
| 1 | apple
| 2 | orange
| 3 | banana
+-------------+------------------------------------------------------------------+
Another table for storing the people:
+-------------+------------------------------------------------------------------+
| id | name
+-------------+------------------------------------------------------------------+
| 1 | William
| 2 | Anna
| 3 | Michael
+-------------+------------------------------------------------------------------+
And finally a table for storing the many-many relationship between people and tags. This is also where you can store the sort order:
+-------------+--------------+-----------+--------------------------------------+
| id | personid | tagid | sort
+-------------+--------------+-----------+--------------------------------------+
| 1 | 1 | 1 | 1
| 2 | 1 | 2 | 2
| 3 | 2 | 1 | 3
| 4 | 2 | 2 | 1
| 5 | 3 | 1 | 1
+-------------+--------------+-----------+---------------------------------------+
A database model that looks more like this will make complicated queries much simpler and won't require complex subqueries, just joins. It's likely to improve your ability to report on data relating to your tags, if that's something that's important to you.
Related
I am trying to fetch nested JSON objects and JSON List from Database using QueryDSL. I have used a native query with LISTAGG and JSON_OBJECT.
Native Query :
SELECT b.id,b.bankName,b.account,b.branch,(select CONCAT(CONCAT('[',LISTAGG(JSON_OBJECT('accountId' value c.accountId, 'name' value customer_name,'amount' value c.amount),',')),']') from CUSTOMER_DETAILS c where c.bankId = b.id) as customers from BANK_DETAILS b
BANK_DETAILS
+----+---------+---------+----------+
| id | BankName| account | branch |
+----+---------+---------+----------+
| 1 | bank1 | savings | branch1 |
| 2 | bank2 | current | branch2 |
+----+---------+---------+----------+
CUSTOMER_DETAILS
+----+-----------+---------------+----------+-----------+
| id | accountId | customer_name | amount | BankId |
+----+-----------+---------------+----------+-----------+
| 1 | 50123 | Abc1 | 150000 | 1 |
| 2 | 50124 | Abc2 | 25000 | 1 |
| 3 | 50125 | Abc3 | 50000 | 2 |
| 4 | 50126 | Abc4 | 250000 | 2 |
+----+-----------+---------------+----------+-----------+
Expected Output for the above tables
[{
"id": "1",
"bankName": "bank1",
"account": "savings",
"branch": "branch1",
"customers": [
{
"accountId": "50123",
"Name": "Abc1",
"amount": 150000
},
{
"accountId": "50124",
"Name": "Abc2",
"amount": 25000
},
]
},{
"id": "2",
"bankName": "bank3",
"account": "current",
"branch": "branch2",
"customers": [
{
"accountId": "50125",
"name": "Abc3",
"amount": 50000
},
{
"accountId": "50126",
"Name": "Abc4",
"amount": 250000
},
]
}]
i have tried with writing this native query in QueryDSL with the below multiple queries for make the same expected output with the forEach loop.
class Repository {
private SQLQueryFactory queryFactory;
public Repository (SQLQueryFactory queryFactory){
this.queryFactory = queryFactory;
}
public void fetchBankDetails(){
List<BankDetails> bankList = queryFactory.select(QBankDetails.bankDetails)
.from(QBankDetails.bankDetails);
bankList.forEach(bankData ->{
List<CustomerDetails> customerList = queryFactory.select(QCustomerDetails.customerDetails)
.from(QCustomerDetails.customerDetails)
.where(QCustomerDetails.customerDetails.bankId.eq(bankData.bankId));
bankData.setCustomerList(customerList)
});
System.out.println(bankList);
}
}
I need to improve my code and convert it into a single query using QueryDSL to return the expected output
Is there any other way or any suggestions?
This is a sample database 'test' with a JSON column 'arr' containing an array of JSON objects
+----+----------------------------------------------------------+
| id | arr |
+----+----------------------------------------------------------+
| 1 | [{"name": "aman"}, {"name": "jay"}] |
| 2 | [{"name": "yash"}, {"name": "aman"}, {"name": "jay"}] |
+----+----------------------------------------------------------+
I want to use JSON_CONTAINS to know if a value exists in a specific key of an object in the array.
Here's my query :
SELECT JSON_CONTAINS(arr, '"jay"', '$[*].name') from test WHERE id=1;
I get the following error:
ERROR 3149 (42000): In this situation, path expressions may not contain the * and ** tokens or an array range.
I know that I can try using JSON_EXTRACT() for this, but what am I doing wrong here ?
Is there any way to use JSON_CONTAINS with an array of JSON objects in MySQL.
Yes, it is possible using the following syntax:
SELECT JSON_CONTAINS(arr, '{"name": "jay"}') from test WHERE id=1;
db<>fiddle demo
Example:
+-----+--------------------------------------------------------+---+
| id | arr | r |
+-----+--------------------------------------------------------+---+
| 1 | [{"name": "aman"}, {"name": "jay"}] | 1 |
| 2 | [{"name": "yash"}, {"name": "aman"}, {"name": "jay"}] | 1 |
| 3 | [{"name": "yash"}, {"name": "aman"}] | 0 |
+-----+--------------------------------------------------------+---+
You must use JSON_SEARCH:
SELECT JSON_SEARCH(arr, 'one', 'jay', NULL, '$[*].name') IS NOT NULL
FROM test
WHERE id=1;
I have a MySQL table authors with columns id, name and published_books. In this, published_books is a JSON column. With sample data,
id | name | published_books
-----------------------------------------------------------------------
1 | Tina | {
| | "17e9bf8f": {
| | "name": "Book 1",
| | "tags": [
| | "self Help",
| | "Social"
| | ],
| | "language": "English",
| | "release_date": "2017-05-01"
| | },
| | "8e8b2470": {
| | "name": "Book 2",
| | "tags": [
| | "Inspirational"
| | ],
| | "language": "English",
| | "release_date": "2017-05-01"
| | }
| | }
-----------------------------------------------------------------------
2 | John | {
| | "8e8b2470": {
| | "name": "Book 4",
| | "tags": [
| | "Social"
| | ],
| | "language": "Tamil",
| | "release_date": "2017-05-01"
| | }
| | }
-----------------------------------------------------------------------
3 | Keith | {
| | "17e9bf8f": {
| | "name": "Book 5",
| | "tags": [
| | "Comedy"
| | ],
| | "language": "French",
| | "release_date": "2017-05-01"
| | },
| | "8e8b2470": {
| | "name": "Book 6",
| | "tags": [
| | "Social",
| | "Life"
| | ],
| | "language": "English",
| | "release_date": "2017-05-01"
| | }
| | }
-----------------------------------------------------------------------
As you see, the published_books column has nested JSON data (one level). JSON will have dynamic UUIDs as the keys and its values will be book details as a JSON.
I want to search for books with certain conditions and extract those books JSON data alone to return as the result.
The query that I've written,
select JSON_EXTRACT(published_books, '$.*') from authors
where JSON_CONTAINS(published_books->'$.*.language', '"English"')
and JSON_CONTAINS(published_books->'$.*.tags', '["Social"]');
This query performs the search and returns the entire published_books JSON. But I wanted just those books JSON alone.
The expected result,
result
--------
"17e9bf8f": {
"name": "Book 1",
"tags": [
"self Help",
"Social"
],
"language": "English",
"release_date": "2017-05-01"
}
-----------
"8e8b2470": {
"name": "Book 6",
"tags": [
"Social",
"Life"
],
"language": "English",
"release_date": "2017-05-01"
}
There is no JSON function yet that filters elements of a document or array with "WHERE"-like logic.
But this is a task that some people using JSON data may want to do, so the solution MySQL has provided is to use the JSON_TABLE() function to transform the JSON document into a format as if you had stored your data in a normal table. Then you can use a standard SQL WHERE clause to the fields returned.
You can't use this function in MySQL 5.7, but if you upgrade to MySQL 8.0 you can do this.
select authors.id, authors.name, books.* from authors,
json_table(published_books, '$.*'
columns(
bookid for ordinality,
name text path '$.name',
tags json path '$.tags',
language text path '$.language',
release_date date path '$.release_date')
) as books
where books.language = 'English'
and json_search(tags, 'one', 'Social') is not null;
+----+-------+--------+--------+-------------------------+----------+--------------+
| id | name | bookid | name | tags | language | release_date |
+----+-------+--------+--------+-------------------------+----------+--------------+
| 1 | Tina | 1 | Book 1 | ["self Help", "Social"] | English | 2017-05-01 |
| 3 | Keith | 2 | Book 6 | ["Social", "Life"] | English | 2017-05-01 |
+----+-------+--------+--------+-------------------------+----------+--------------+
Note that nested JSON arrays are still difficult to work with, even with JSON_TABLE(). In this example, I exposed the tags as a JSON array, and then use JSON_SEARCH() to find the tag you wanted.
I agree with Rick James — you might as well store the data in normalized tables and columns. You think that using JSON will save you some work, but it's won't. It might make it more convenient to store the data as a single JSON document instead of multiple rows across several tables, but you just have to unravel the JSON again before you can query it the way you want.
Furthermore, if you store data in JSON, you will have to solve this sort of JSON_TABLE() expression every time you want to query the data. That's going to make a lot more work for you on an ongoing basis than if you had stored the data normally.
Frankly, I have yet to see a question on Stack Overflow about using JSON with MySQL that wouldn't lead to the conclusion that storing data in relational tables is a better idea than using JSON, if the structure of the data doesn't need to vary.
You are approaching the task backwards.
Do the extraction as you insert the data. Insert into a small number of tables (Authors, Books, Tags, and maybe a couple more) and build relations between them. No JSON is needed in this database.
The result is an easy-to-query and fast database. However, it requires learning about RDBMS and SQL.
JSON is useful when the data is a collection of random stuff. Your JSON is very regular, hence the data fits very nicely into RDBMS technology. In that case, JSON is merely a standard way to serialize the data. But it should not be used for querying.
I have table that holds sort json number values that i need to sort id using defined json sort number...so i have table like this:
+----+------------+-----------------+
| id | channel | sort |
+----+------------+-----------------+
| 1 | US_CH 1 | ["1", "2", "4"] |
| 4 | US_CH 4 | ["1", "2", "4"] |
| 2 | US_CH 2 | ["1", "2", "4"] |
+----+------------+-----------------+
And would like to get to get this:
+----+------------+-----------------+
| id | channel | sort |
+----+------------+-----------------+
| 1 | US_CH 1 | ["1", "2", "4"] |
| 2 | US_CH 2 | ["1", "2", "4"] |
| 4 | US_CH 4 | ["1", "2", "4"] |
+----+------------+-----------------+
So the point is to get ID sort by value of json sort values in array. I know that sort json values are not json structure but i need to get using this number because i im working channel editor that update and add channels that have 5000 records (enigma2 stb) so i need using this number because it will store small data in database and inserting and updating will be more faster.
I try using JSON_SEARCH to extract single value but i need all values so that i can use like ORDER BY JSON_EXTRACT(sort, '$[extract numbers]')
Try this:
ORDER BY LOCATE(sort, CONCAT('"', id, '"'))
I've got several Postgres 9.4 tables that contain data like this:
| id | data |
|----|-------------------------------------------|
| 1 | {"user": "joe", "updated-time": 123} |
| 2 | {"message": "hi", "updated-time": 321} |
I need to transform the JSON column into something like this
| id | data |
|----|--------------------------------------------------------------|
| 1 | {"user": "joe", "updated-time": {123, "unit":"millis"}} |
| 2 | {"message": "hi", "updated-time": {321, "unit":"millis"}} |
Ideally it would be easy to apply the transformation to multiple tables. Tables that contain the JSON key data->'updated-time' should be updated, and ones that do not should be skipped. Thanks!
You can use the || operator to merge two jsonb objects together.
select '{"foo":"bar"}'::jsonb || '{"baz":"bar"}'::jsonb;
= {"baz": "bar", "foo": "bar"}