I have a MaraiDB (10.2.14) database containing a table where in one column JSON data is stored.
I'm puzzled as how to extract data from this column.
Example Data
SELECT 1 AS ID
, '[{"name":"x","score":2},{"name":"y", "score":8},{"name":"z","score":3}]' AS REPLY
UNION ALL
SELECT 2 AS ID
, '[{"name":"x","score":5},{"name":"y", "score":4},{"name":"z","score":3}]' AS REPLY
UNION ALL
SELECT 3 AS ID
, '[{"name":"x","score":2},{"name":"y", "score":2},{"name":"z","score":6}]' AS REPLY
UNION ALL
SELECT 4 AS ID
, '[{"name":"x","score":5},{"name":"y", "score":8},{"name":"z","score":6}]' AS REPLY
So how would I find all entries having "name":"x" and a "score":5. Additionally I need to get the "score" value of the "name":"y" of that entry.
My current dirty approach is
WITH JT1 AS (
SELECT 1 AS ID
, '[{"name":"x","score":2},{"name":"y", "score":8},{"name":"z","score":3}]' AS REPLY
UNION ALL
SELECT 2 AS ID
, '[{"name":"x","score":5},{"name":"y", "score":4},{"name":"z","score":3}]' AS REPLY
UNION ALL
SELECT 3 AS ID
, '[{"name":"x","score":2},{"name":"y", "score":2},{"name":"z","score":6}]' AS REPLY
UNION ALL
SELECT 4 AS ID
, '[{"name":"x","score":5},{"name":"y", "score":8},{"name":"z","score":6}]' AS REPLY
)
SELECT ID
, REGEXP_REPLACE(
REGEXP_REPLACE( EXTRACTED, '^.*"y",\\s', '')
, '[,\\]].*$', '') AS Y
, EXTRACTED
FROM (
SELECT ID
, JSON_EXTRACT(REPLY, '$[*].name','$[*].score') EXTRACTED
FROM JT1
) JT2
WHERE EXTRACTED RLIKE '"x", 5\\b'
;
So I first extract "name" and "score" which gives me column data like ["x", 5, "y", 4, "z", 3]. With that I do some nasty REGEXP search & replaces.
I feel there must be a better way.
I tried using COLUMN_CREATE, but "COLUMN_CREATE" seems not to be able to accept the result from JSON_EXTRACT as input. Now that I think about it, this seems logical as "name" and "score" here are properly ordered, but can I be sure it's always that sequence?
Can anyone give me a hint how to do this better?
If I understand what you need, a query like the following may be useful:
WITH `JT1` AS (
SELECT 1 AS `ID`
, '[{"name":"x","score":2},{"name":"y", "score":8},{"name":"z","score":3}]' AS `REPLY`
UNION ALL
SELECT 2 AS `ID`
, '[{"name":"x","score":5},{"name":"y", "score":4},{"name":"z","score":3}]' AS `REPLY`
UNION ALL
SELECT 3 AS `ID`
, '[{"name":"x","score":2},{"name":"y", "score":2},{"name":"z","score":6}]' AS `REPLY`
UNION ALL
SELECT 4 AS `ID`
, '[{"name":"x","score":5},{"name":"y", "score":8},{"name":"z","score":6}]' AS `REPLY`
)
SELECT
`ID`,
`REPLY`,
JSON_VALUE(
`REPLY`,
JSON_UNQUOTE(
REPLACE(
JSON_SEARCH(`REPLY`, 'one', 'y', NULL, '$[*].name'),
'name',
'score'
)
)
) `"name":"y"`
FROM
`JT1`
WHERE
JSON_VALUE(
`REPLY`,
JSON_UNQUOTE(
REPLACE(
JSON_SEARCH(`REPLY`, 'one', 'x', NULL, '$[*].name'),
'name',
'score'
)
)
) = 5;
See dbfiddle.
Related
I have json datas in the format as shown below.
**the number of objects under 'images'(VIEWS, ROOMS, etc.) varied for each JSON object. Not all objects have VIEWS...POOL, it may only have ROOMS.
"propertyId":{"ea":"12345678","h":"","vo":""},
"thumbnail":{"width":1000,"height":500,"link":"xxx.jpg"},
"hero":{"title":"Featured Image","width":1000,"height":669,"link":"yyy.jpg"},
"images":{
"VIEWS":[
{"title":"View from Property","width":1000,"height":666,"link":"a1.jpg"},
{"title":"View from Property","width":1000,"height":666,"link":"a2.jpg"}
],
"ROOMS":[
{"title":"Room","width":1000,"height":669,"link":"b1.jpg"},
{"title":"Room","width":1000,"height":662,"link":"b2.jpg"}
],
"EXTERIOR":[
{"title":"Terrace","width":1000,"height":666,"link":"c1.jpg"},
{"title":"Terrace","width":1000,"height":666,"link":"c2.jpg"}
],
"AMENITIES":[
{"title":"Property Amenity","width":1000,"height":666,"link":"d1.jpg"}
],
"POOL":[
{"title":"Outdoor Pool","width":1000,"height":666,"link":"e1"},
{"title":"Outdoor Pool","width":1000,"height":666,"link":"e2.jpg"}
]}
}
I wanted to insert the following data into the table with the following format in mysql 8.0 database.
JSON -> Table Reference
HotelId = propertyId.ea
hero_url = hero.link
hero_title = hero.title
media_title = images.(VIEWS/ROOMS/EXTERIOR/...).title
media_url = images.(VIEWS/ROOMS/EXTERIOR/...).link
>> ...other object may have different category (VIEWS/ROOMS...)
created_at and updated_at = date
After searching through the old topics, I was able to use the select statement below to display the format above. However, I'm confused as to it how works when using INSERT.
SELECT t.HotelID, t.hero_url, t.hero_title,SUBSTRING_INDEX(SUBSTRING_INDEX(t.media_title, ',', n.n), ',', -1) media_title, SUBSTRING_INDEX(SUBSTRING_INDEX(t.images, ',', n.n), ',', -1) images
FROM hotel_sample.property_photos t CROSS JOIN
(
SELECT a.N + b.N * 10 + 1 n
FROM
(SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) a
,(SELECT 0 AS N UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) b
ORDER BY n
) n
WHERE n.n <= 1 + (LENGTH(t.media_title) - LENGTH(REPLACE(t.media_title, ',', '')))
ORDER BY HotelID
Would appreciate if anyone could show me how I could insert the above JSON data into the table. (MYSQL 8.0)
CREATE TABLE `property_photos` (
`id` int NOT NULL AUTO_INCREMENT,
`HotelID` varchar(10) DEFAULT NULL,
`hero_url` varchar(255) DEFAULT NULL,
`hero_title` text,
`category` varchar(30) DEFAULT NULL,
`media_url` varchar(255) DEFAULT NULL,
`media_title` text,
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL
PRIMARY KEY (`id`)
To make things easier, let's say that I have a table representing pretty simple user's feed.
There are two "key" columns in my feed table:
object_id it's a ID of different assets, e.g. comment, post, etc.
entity_type_id it's a basically reference to another table in my DB.
The "children" tables may have some attributes in common, e.g. is_hidden, is_deleted and is_locked (however, they are not replicated across all tables).
Now, I'd like to implement a filter that should filter out my feed items, based on the values of these three attributes.
What I did so far?
SELECT `f`.*
FROM `feed` `f`
WHERE 1
-- !!! Other filters goes here. ---
AND
(
--
-- !!! Filter by status
--
( -- "Locked" (not all children tables have this column)
(
`f`.`entity_type_id` = 1 AND `f`.`object_id` IN ( SELECT `fb_comment_id` FROM `comments` WHERE `is_locked` = 1 AND `fb_page_id` IN('0123456789') )
)
OR
(
`f`.`entity_type_id` = 4 AND `f`.`object_id` IN ( SELECT `fb_post_id` FROM `posts` WHERE `is_locked` = 1 AND `fb_page_id` IN('0123456789') )
)
)
( -- "Hidden" (not all children tables have this column)
(
`f`.`entity_type_id` = 1 AND `f`.`object_id` IN ( SELECT `fb_comment_id` FROM `comments` WHERE `is_hidden` = 1 AND `fb_page_id` IN('0123456789') )
)
OR
(
`f`.`entity_type_id` = 4 AND `f`.`object_id` IN ( SELECT `fb_post_id` FROM `posts` WHERE `is_hidden` = 1 AND `fb_page_id` IN('0123456789') )
)
)
OR
(
-- "Deleted"
(
`f`.`entity_type_id` = 1 AND `f`.`object_id` IN ( SELECT `fb_comment_id` FROM `comments` WHERE `is_deleted` = 1 AND `fb_page_id` IN ('0123456789') )
)
OR
(
`f`.`entity_type_id` = 3 AND `f`.`object_id` IN ( SELECT `insta_comment_id` FROM `instagram_comments` WHERE `is_deleted` = 1 AND `insta_profile_id` IN ('9876543210') )
)
OR
(
`f`.`entity_type_id` = 4 AND `f`.`object_id` IN ( SELECT `fb_post_id` FROM `posts` WHERE `is_deleted` = 1 AND `fb_page_id` IN ('0123456789') )
)
OR
(
`f`.`entity_type_id` = 5 AND `f`.`object_id` IN ( SELECT `insta_post_id` FROM `instagram_posts` WHERE `is_deleted` = 1 AND `insta_profile_id` IN ('9876543210') )
)
)
)
As you can see I was using sub queries, but I was wondering is there a better way to write such queries?
I don't know if it's better, but I'd create a subquery that unions the necessary flag fields from your child tables and then just do a regular join to get the flag fields. If a flag field is not present for one of the tables, it can just be false.
Something like:
SELECT `f`.*
FROM `feed` `f`
JOIN
(
SELECT
1 AS `entity_type_id`
, fb_comment_id AS `object_id`
, is_locked
, is_hidden
, is_deleted
FROM
comments
UNION ALL
SELECT
4 AS `entity_type_id`
, fb_post_id AS `object_id`
, is_locked
, is_hidden
, is_deleted
FROM
posts
UNION ALL
SELECT
3 AS `entity_type_id`
, insta_comment_id AS `object_id`
, 0 AS is_locked
, 0 AS is_hidden
, is_deleted
FROM
instagram_comments
UNION ALL
SELECT
5 AS `entity_type_id`
, insta_post_id AS `object_id`
, 0 AS is_locked
, 0 AS is_hidden
, is_deleted
FROM
instagram_posts
) AS flag_summary ON (
flag_summary.entity_type_id = f.entity_type_id
AND flag_summary.object_id = f.object_id
)
Some tips:
Try to use INNER JOIN instead of WHERE + correlated queries. Create for example a table with all the tables in the sub-queries, and apply your filters. Do not forget to use PROCEDURE ANALYSE and index.
Avoid SELECT *, type all the variables you need.
Apply an EXPLAIN to know where you can improve your script.
I have two tables and want displays rows from the two one in the same page ordered by date created.
Here my query:
SELECT R.*, R.id as id_return
FROM return R
UNION
ALL
SELECT A.*, A.id as id_buy
FROM buy A
WHERE
R.id_buyer = '$user' AND R.id_buyer = A.id_buyer AND (R.stats='1' OR R.stats='3') OR A.stats='4'
ORDER
BY R.date, A.date DESC LIMIT $from , 20
With this query i get this error message:
Warning: mysqli_fetch_array() expects parameter 1 to be mysqli_result, boolean given in ...
And here how i think i can differentiate between the results: (Knowing if the result is from the table RETURN or from the table BUY)
if(isset($hist_rows["id_return"])) {
// show RETURN rows
} else {
// show BUY rows
}
Please what is wrong with the query, and if the method to differentiate between tables are correct ?
EDIT
Here my tables sample:
CREATE TABLE IF NOT EXISTS `return` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`id_buyer` INT(12) NOT NULL,
`id_seller` INT(12) NOT NULL,
`message` TEXT NOT NULL,
`stats` INT(1) NOT NULL,
`date` varchar(30) NOT NULL,
`update` varchar(30)
PRIMARY KEY (`id`)
)
CREATE TABLE IF NOT EXISTS `buy` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`id_buyer` INT(12) NOT NULL,
`product` INT(12) NOT NULL,
`title` VARCHAR(250) NOT NULL,
`stats` INT(1) NOT NULL,
`date` varchar(30) NOT NULL
PRIMARY KEY (`id`)
)
Be sure the two table return and buy have the same number (and type sequence) of colummns .. if not the query fails
try select only the column you need from both the table and be sure that these are in correspondenting number and type
SELECT R.col1, R.col2, R.id as id_return
FROM return R
UNION ALL
SELECT A.col1, A.col2, A.id as id_buy
FROM buy A
WHERE
........
Looking to your code you should select the same number and type of column form boith the table eg de sample below:
(where i have added the different column and selecting null from the table where are not present)
I have aslore referred the proper where condition to each table ..
SELECT
R.'from return' as `source_table`
, R.`id`
, R.`id_buyer`
, null as product
, null as title
, R.`id_seller` as id_seller
, R-`message`
, R.`stats`
, R.`date`
, R.`update`
FROM return R
WHERE R.id_buyer = '$user'
AND (R.stats='1' OR R.stats='3')
UNION ALL
SELECT
A.'from buy'
, A.`id`
, A.`id_buyer`
, A.`product`
, A.`title`
, null
, null
, A.`stats`
, A.`date`
, null
FROM buy A
WHERE
A.id_buyer = '$user'
AND A.stats='4'
ORDER BY `source table`, date DESC LIMIT $from , 20
for retrive te value of the first column you should use in your case
echo $hist_rows["source_table"];
Otherwise i the two table are in some way related you should look at a join (left join) for link the two table and select the the repated column
(but this is another question)
But if you need left join you can try
SELECT
R.`id`
, R.`id_buyer`
, R.`id_seller` as id_seller
, R-`message`
, R.`stats`
, R.`date`
, R.`update`
, A.`id`
, A.`id_buyer`
, A.`product`
, A.`title`
, null
, null
, A.`stats`
, A.`date`
FROM return R
LEFT JOIN buy A ON R.id_buyer = A.id_buyer
AND R.id_buyer = '$user'
AND (R.stats='1' OR R.stats='3')
AND A.stats='4'
ORDER BY R.date DESC LIMIT $from , 20
When you use union all, the queries need to have exactly the same columns in the same order. If the types are not quite the same, then they are converted to the same type.
So, you don't want union all. I'm guessing you want a join. Something like this:
SELECT r.co1, r.col2, . . ., r.id as id_return,
b.col1, b.col2, . . ., b.id as id_buy
FROM return r JOIN
buy b
ON r.id_buyer = b.id_buyer
WHERE r.id_buyer = '$user' and
(r.stats in (1, 3) OR A.stats = 4)
ORDER BY R.date, A.date DESC
LIMIT $from, 20;
This query is only a guess as to what you might want.
Since you're using a union, select a string that you set identifying each query:
SELECT 'R', R.*, R.id as id_return
FROM return R
UNION
ALL
SELECT 'A', A.*, A.id as id_buy
This way your string 'R' or 'A' is the first column, showing you where it came from. We can't really know why it's failing without the full query, but I'd guess your $from might be empty?
As for your
Warning: mysqli_fetch_array() expects parameter 1 to be mysqli_result, boolean given in ...
Run the query directly first to get the sql sorted out before putting it into your PHP script. The boolean false indicates the query failed.
I've trying to concatenate the values of 2 GROUP_CONCAT( columns ) from a single table that's been joined twice, then get the unique items from the list.
I can do all this outside of my query but if possible it would be nice to just pull the data from the DB with a JOIN and some fancy string manipulation.
Simply put, I want to produce 1,2,3,4 from selecting 1,2,3 and 1,3,4. The 1,2,3 adn 1,3,4 are the results of the GROUP_CONCAT on the twice joined table. I can get this far:
SELECT CONCAT_WS(
",",
"1,2,3",
"1,3,4"
)
Which outputs 1,2,3,1,3,4
I'd like to be able to do something like:
-- NOTE TO SKIM READERS: THIS QUERY WILL NOT WORK
SELECT
SORT_LIST(
DISTINCT
CONCAT_WS(
",",
"1,2,3",
"1,3,4"
)
)
-- NOTE TO SKIM READERS: THIS QUERY WILL NOT WORK
But I can't find anything like that in MySQL.
The 1,2,3 and 1,3,4 have already been produced with GROUP_CONCAT( DISTINCTcol)
As stated in my comment I worked out a way to achieve distinct concatenated lists of strings using a sub query:
DROP TABLE IF EXISTS `test1234`;
CREATE TABLE `test1234` (
`val` int(1),
`type` varchar(1)
);
INSERT INTO `test1234` VALUES
( 1, 'a' ),
( 2, 'a' ),
( 3, 'a' ),
( 1, 'b' ),
( 3, 'b' ),
( 4, 'b' );
SELECT GROUP_CONCAT( `val` ) AS `vals`
FROM (
(
SELECT `val` FROM `test1234` WHERE `type` = 'a'
) UNION DISTINCT (
SELECT `val` FROM `test1234` WHERE `type` = 'b'
)
) AS `test`;
DROP TABLE IF EXISTS `test1234`;
This selected 1,2,3,4
I'm trying to create a SQL statement to find the matching record based on the provided post code and stored post codes in the database plus the weight aspect.
The post codes in the database are between 1 or 2 characters i.e. B, BA ...
Now - the value passed to the SQL statement will always have 2 first characters of the client's post code. How can I find the match for it? Say I have a post code B1, which would only match the single B in the database plus the weight aspect, which I'm ok with.
Here's my current SQL statement, which also takes the factor of the free shipping above certain weight:
SELECT `s`.*,
IF (
'{$weight}' > (
SELECT MAX(`weight_from`)
FROM `shipping`
WHERE UPPER(SUBSTRING(`post_code`, 1, 2)) = 'B1'
),
(
SELECT `cost`
FROM `shipping`
WHERE UPPER(SUBSTRING(`post_code`, 1, 2)) = 'B1'
ORDER BY `weight_from` DESC
LIMIT 0, 1
),
`s`.`cost`
) AS `cost`
FROM `shipping` `s`
WHERE UPPER(SUBSTRING(`s`.`post_code`, 1, 2)) = 'B1'
AND
(
(
'{$weight}' > (
SELECT MAX(`weight_from`)
FROM `shipping`
WHERE UPPER(SUBSTRING(`post_code`, 1, 2)) = 'B1'
)
)
OR
('{$weight}' BETWEEN `s`.`weight_from` AND `s`.`weight_to`)
)
LIMIT 0, 1
The above however uses the SUBSTRING() function with hard coded number of characters set to 2 - this is where I need some help really to make it match only number of characters that matches the provided post code - in this case B1.
Marcus - thanks for the help - outstanding example - here's what my code look like for those who also wonder:
First I've run the following statement to get the right post code:
(
SELECT `post_code`
FROM `shipping`
WHERE `post_code` = 'B1'
)
UNION
(
SELECT `post_code`
FROM `shipping`
WHERE `post_code` = SUBSTRING('B1', 1, 1)
)
ORDER BY `post_code` DESC
LIMIT 0, 1
Then, based on the returned value assigned to the 'post_code' index my second statement followed with:
$post_code = $result['post_code'];
SELECT `s`.*,
IF (
'1000' > (
SELECT MAX(`weight_from`)
FROM `shipping`
WHERE `post_code` = '{$post_code}'
),
(
SELECT `cost`
FROM `shipping`
WHERE `post_code` = '{$post_code}'
ORDER BY `weight_from` DESC
LIMIT 0, 1
),
`s`.`cost`
) AS `cost`
FROM `shipping` `s`
WHERE `s`.`post_code` = '{$post_code}'
AND
(
(
'1000' > (
SELECT MAX(`weight_from`)
FROM `shipping`
WHERE `post_code` = '{$post_code}'
ORDER BY LENGTH(`post_code`) DESC
)
)
OR
('1000' BETWEEN `s`.`weight_from` AND `s`.`weight_to`)
)
LIMIT 0, 1
The following query will get all results where the post_code in the shipping table matches the beginning of the passed in post_code, then it orders it most explicit to least explicit, returning the most explicit one:
SELECT *
FROM shipping
WHERE post_code = SUBSTRING('B1', 1, LENGTH(post_code))
ORDER BY LENGTH(post_code) DESC
LIMIT 1
Update
While this query is flexible, it's not very fast, since it can't utilize an index. If the shipping table is large, and you'll only pass in up to two characters, it might be faster to make two separate calls.
First, try the most explicit call.
SELECT *
FROM shipping
WHERE post_code = 'B1'
If it doesn't return a result then search on a single character:
SELECT *
FROM shipping
WHERE post_code = SUBSTRING('B1', 1, 1)
Of course, you can combine these with a UNION if you must do it in a single call:
SELECT * FROM
((SELECT *
FROM shipping
WHERE post_code = 'B1')
UNION
(SELECT *
FROM shipping
WHERE post_code = SUBSTRING('B1', 1, 1))) a
ORDER BY post_code DESC
LIMIT 1