I have a column in db table which is of a JSON type.
This column will store such values:
column 1: ["bmw", "opel", "audi"]
column 2: ["opel", "bwm", "audi"]
column 3: ["audi"]
the order of values is different as well as the total number but in general there is a list of valid values to be stored.
On UI, there is a table with columns which can be sorted. When user clicks on a column Car Brand, ASC, I will need to sort the output from db based on the column values I mentioned above.
I'd like to receive the following output:
1: ["audi"]
2: ["audi", "bmw", "opel"]
3: ["audi", "bmw", "opel"]
I can't seem to find the solution for this using JSON path.
Can anybody help me?
Please do not suggest to store the values in a different way, it's not me who decides this.
You can't do this in MySQL.
What Akina describes in the comment above is like this demo:
set #j = '["opel", "bwm", "audi"]';
select * from json_table(#j, '$[*]' columns (a varchar(10) path '$')) j order by a;
+------+
| a |
+------+
| audi |
| bwm |
| opel |
+------+
Then the idea is to recombine these rows into a JSON array using JSON_ARRAYAGG(). However, this doesn't work:
select json_arrayagg(a) as j from (
select * from json_table(#j, '$[*]' columns (a varchar(10) path '$')) j order by a
) as a;
+-------------------------+
| j |
+-------------------------+
| ["opel", "bwm", "audi"] |
+-------------------------+
The ORDER BY in the subquery has been factored out. MySQL thinks there's no reason to sort rows in a derived table, they should be sorted as the last step of the outer query.
https://dev.mysql.com/doc/refman/5.7/en/aggregate-functions.html#function_json-arrayagg says:
JSON_ARRAYAGG(col_or_expr)
Aggregates a result set as a single JSON array whose elements consist
of the rows. The order of elements in this array is undefined.
https://bugs.mysql.com/bug.php?id=94696 is a feature request from 2019, asking for a way to sort elements before aggregating them with JSON_ARRAYAGG(). That ticket is in a "Verified" state, meaning it has been accepted as a feature request, but it has not been implemented.
There is no solution yet in MySQL 8.0 to use JSON_ARRAYAGG() to produce an array in a specific order.
So the best you can do is fetch them as sorted rows, as I showed in the intermediate step earlier in this answer, and then combine them into an array in that order in your client application code.
Related
Suppose we have 2 numbers of 3 bits each attached together like '101100', which basically represents 5 and 4 combined. I want to be able to perform aggregation functions like SUM() or AVG() on this column separately for each individual 3-bit column.
For instance:
'101100'
'001001'
sum(first three column) = 6
sum(last three column) = 5
I have already tried the SUBSTRING() function, however, speed is the issue in that case as this query will run on millions of rows regularly. And string matching will slow the query.
I am also open for any new databases or technologies that may support this functionality.
You can use the function conv() to convert any part of the string to a decimal number:
select
sum(conv(left(number, 3), 2, 10)) firstpart,
sum(conv(right(number, 3), 2, 10)) secondpart
from tablename
See the demo.
Results:
| firstpart | secondpart |
| --------- | ---------- |
| 6 | 5 |
With the current understanding I have of your schema (which is next to none), the best solution would be to restructure your schema so that each data point is its own record instead of all the data points being in the same record. Doing this allows you to have a dynamic number of data points per entry. Your resulting table would look something like this:
id | data_type | value
ID is used to tie all of your data points together. If you look at your current table, this would be whatever you are using for the primary key. For this answer, I am assuming id INT NOT NULL but yours may have additional columns.
Data Type indicates what type of data is stored in that record. This would be the current tables column name. I will be using data_type_N as my values, but yours should be a more easily understood value (e.g. sensor_5).
Value is exactly what it says it is, the value of the data type for the given id. Your values appear to be all numbers under 8, so you could use a TINYINT type. If you have different storage types (VARCHAR, INT, FLOAT), I would create a separate column per type (val_varchar, val_int, val_float).
The primary key for this table now becomes a composite: PRIMARY KEY (id, data_type). Since your previously single record will become N records, the primary key will need to adjust to accommodate that.
You will also want to ensure that you have indexes that are usable by your queries.
Some sample values (using what you placed in your question) would look like:
1 | data_type_1 | 5
1 | data_type_2 | 4
2 | data_type_1 | 1
2 | data_type_2 | 1
Doing this, summing the values now becomes trivial. You would only need to ensure that data_type_N is summed with data_type_N. As an example, this would be used to sum your example values:
SELECT data_type,
SUM(value)
FROM my_table
WHERE id IN (1,2)
GROUP BY data_type
Here is an SQL Fiddle showing how it can be used.
I have a table with JSON data like this:
{"a": [{"color": "blue", "value": 15}, {"color": "red", "value": 30}]}
I need to get the "value" that is inside the same object of "blue".
I thought to use the code below:
SELECT JSON_EXTRACT(my_data, '$.a[0].value');
The problem is that the "blue" object can be in any index of the array.
So, is there a way to retrieve the index first and then i'll query using the right index?
UPDATE
The Barmar's answer works but it needs to wrap in JSON_UNQUOTE()
Use JSON_SEARCH() to find the path to blue.
SELECT JSON_EXTRACT(my_data, JSON_UNQUOTE(REPLACE(JSON_SEARCH(my_data, 'one', 'blue'), '.color', '.value')))
JSON_SEARCH will return a string like $.a[0].color. REPLACE changes that to $.a[0].value, then you extract that element.
DEMO
Here's an example of using JSON_TABLE():
select j.* from d, json_table(d.data, '$.a[*]' columns (
color varchar(20) path '$.color',
value int path '$.value')
) as j;
+-------+-------+
| color | value |
+-------+-------+
| blue | 15 |
| red | 30 |
+-------+-------+
You can then apply conditions in the WHERE clause, as if you had stored the data in a normal table.
select j.* from d, json_table(d.data, '$.a[*]' columns (
color varchar(20) path '$.color',
value int path '$.value')
) as j
where j.color = 'blue';
+-------+-------+
| color | value |
+-------+-------+
| blue | 15 |
+-------+-------+
This requires you to write a complex query like this EVERY TIME you query the JSON data.
One wonders if it would have been easier to store the JSON in a normal table from the start.
I often recommend to MySQL users that storing data as JSON makes more work for you, if you need to make SQL expressions to reference individual fields within the JSON. I wouldn't use JSON in these cases, I'd explode the JSON array into rows, and the JSON fields into columns of a set of normal tables. Then you can write simpler queries, you can optimize with indexes, and you can use constraints and data types properly.
JSON is the most easily misused feature of the recent MySQL releases.
I've stumbled on a previously asked and answered question here:
How to use comparison operator for numeric string in MySQL?
I absolutely agree with the answer being the best mentioned. But it left me with a question myself while I was trying to create my own answer. I was trying to select the first number and convert it to an integer. Next I wanted to compare that integer with a number (3 in case of the question).
This is the query I've created:
SELECT experience,
CONVERT(SUBSTRING_INDEX(experience,'-',1), UNSIGNED INTEGER) AS num
FROM employee
WHERE #num >= 3;
For the sake of simplicity, asume the data inside experience is: 4-8
The query doesn't return any errors. But it doesn't return the data either. I know it's possible to compare the data inside a column with a user defined variable. But is it possible to compare data (the integer in this case) with the variable like I'm trying to do?
This is purely out of curiousity and to learn something.
Yes, a derived table will do. The inner select block below is a derived table. And every derived table needs a name. In my case, xDerived.
The strategy is to let the derived table cleanse the use of the column name. Coming out of the derived chunk is a clean column named num which the outer select is free to use.
Schema
create table employee
( id int auto_increment primary key,
experience varchar(20) not null
);
-- truncate table employee;
insert employee(experience) values
('4-5'),('7-1'),('4-1'),('6-5'),('8-6'),('5-9'),('10-4');
Query
select id,experience,num
from
( SELECT id,experience,
CONVERT(SUBSTRING_INDEX(experience,'-',1),UNSIGNED INTEGER) AS num
FROM employee
) xDerived
where num>=7;
Results
+----+------------+------+
| id | experience | num |
+----+------------+------+
| 2 | 7-1 | 7 |
| 5 | 8-6 | 8 |
| 7 | 10-4 | 10 |
+----+------------+------+
Note, your #num concept was faulty but hopefully I interpreted what you meant to do above.
Also, I went with 7 not 3 because all your sample data would have returned, and I wanted to show you it would work.
The AS num instruction names the result of convert as num, not a variable named #num.
You could repeat the convert
SELECT experience,CONVERT(SUBSTRING_INDEX(experience,'-',1),UNSIGNED INTEGER)
FROM employee
WHERE CONVERT(SUBSTRING_INDEX(experience,'-',1),UNSIGNED INTEGER) >= 3;
Or use a partial (derived) table (only one convert)
SELECT experience,num
FROM (select experience,
CONVERT(SUBSTRING_INDEX(experience,'-',1),UNSIGNED INTEGER) as num
FROM employee) as partialtable WHERE num>=3;
Much simpler. (Or at least much shorter.) This will work for the data as described, namely "number, -, other stuff".
SELECT experience,
0+experience AS 'FirstPart'
FROM employee
WHERE 0+experience >= 3
Why? 0+string is parsed as "convert the string to a number, then add it to 0". Converting a string will extract the digits up to the first non-digit, then convert that as numeric.
I have a column called "Permissions" in my table. The permissions are strings which can be:
"r","w","x","rw","wx","rwx","xwr"
etc. Please note the order of characters in the string is not fixed. I want to GROUP_CONCAT() on the "Permissions" column of my table. However this causes very large strings.
Example: "r","wr","wx" group concatenated is "r,wr,wx" but should be "r,w,x" or "rwx". Using distinct() clause doesn't seem to help much. I am thinking that if I could check if a permission value is a substring of the other column then I should not concatenate it, but I don't seem to find a way to accomplish that.
Any column based approach using solely string functions would also be appreicated.
EDIT:
Here is some sample data:
+---------+
| perm |
+---------+
| r,x,x,r |
| x |
| w,rw |
| rw |
| rw |
| x |
| w |
| x,x,r |
| r,x |
+---------+
The concatenated result should be:
+---------+
| perm |
+---------+
| r,w,x |
+---------+
I don't have control over the source of data and would like not to create new tables ( because of restricted privileges and memory constraints). I am looking for a post-processing step that converts each column value to the desired format.
A good idea would be to first normalize your data.
You could, for example try this way (I assume your source table is named Files):
Create simple table called PermissionCodes with only column named Code (type of string).
Put r, w, and x as values into PermissionCodes (three rows total).
In a subquery join Files to PermissionCodes on a condition that Code exists as a substring in Permissions.
Perform your GROUP_CONCAT aggregation on the result of the subquery.
If it is a case here, that for the same logical entires in Files there exists multiple permission sets that overlaps (i.e. for some file there is a row with rw and another row with w) then you would limit your subquery to distinct combinations of Files' keys and Code.
Here's a fiddle to demonstrate the idea:
http://sqlfiddle.com/#!9/6685d6/4
You can try something like:
SELECT user_id, GROUP_CONCAT(DISTINCT perm)
FROM Permissions AS p
INNER JOIN (SELECT 'r' AS perm UNION ALL
SELECT 'w' UNION ALL
SELECT 'x') AS x
ON p.permission LIKE CONCAT('%', x.perm, '%')
GROUP BY user_id
You can include any additional permission code in the UNION ALL of the derived table used to JOIN with Permissions table.
Demo here
I have a simple MySQL table made up of words and an associated number. The numbers are unique for each word. I want to find the first word whose index is larger than a given number. As an example:
-----------------------
| WORD: | F_INDEX: |
|---------------------|
| a | 5 |
| cat | 12 |
| bat | 4002 |
-----------------------
If I was given the number "9" I would want "cat" returned, as it is the first word whose index is larger than 9.
I know that I can get a full list of sorted rows by querying:
SELECT * FROM table_name ORDER BY f_index;
But would, instead, like to make a MySQL query that does this. (The confusion lies in the fact that I'm unsure as to how to keep track of the current row in my query). I know can loop with something like this:
CREATE PROCEDURE looper(desired_index INT)
BEGIN
DECLARE current_index int DEFAULT 0
// Loop here, setting current_index to whatever the next rows index is,
// then do a comparison to check it to our desired_index, breaking out
// if it is greater.
END;
Any help would be greatly appreciated.
Try this:
SELECT t.word
, t.f_index
FROM table_name t
WHERE t.f_index > 9
ORDER
BY t.f_index
LIMIT 1
It is much more efficient to have the database return the row you need, than it is to pull a whole bunch of rows and figure out which one you need.
For best performance of this query, you will want an index ON table_name (f_index,word).
Why don't you just use MYSQL statement to retrieve the first item you found from f_index where the f_index is greater than the value your pass in.
For example :
select word from table_name
where f_index > desired_index
order by f_index
limit 1