MySQL/MariaDB/SQLite: DISTINCT CONCAT - mysql

This question isn't necessarily just Laravel related, but I'm trying to fetch records, which are distinct by concatenated fields. I need this to work with both MySQL/MariaDB as well as SQLite for testing purposes.
While doing my research, I've found out that SQLite does not have CONCAT function - instead you're using || operator to concatenate items. MySQL on the other hand will not interpret || the same way, but I can always use the conditional statement just to cover both cases.
However, I still cannot get records I'm after - my table consists of:
| id | tagable_id | tagable_type | name | title | description | url | image | hits |
| 1 | 1 | App\Models\Article | a.. | A.. | A.. descr.. | https://localhost | https://localhost... | 0 |
| 2 | 1 | App\Models\Article | b.. | B.. | B.. descr.. | https://localhost | https://localhost... | 2 |
| 3 | 1 | App\Models\Article | c.. | C.. | C.. descr.. | https://localhost | https://localhost... | 3 |
| 4 | 1 | App\Models\Page | a.. | A.. | C.. descr.. | https://localhost | https://localhost... | 0 |
I need get only 4 records that are sorted ASC by number of hits and which are unique using CONCAT(table_id, tagable_type).
What the statement should return in this case would be records with id 1 and 4 - because 2 and 3 have the same tagable_id and tagable_type as record with id 1, which has lowest number of hits - effectively only returning only 2 records:
| id | tagable_id | tagable_type | name | title | description | url | image | hits |
| 1 | 1 | App\Models\Article | a.. | A.. | A.. descr.. | https://localhost | https://localhost... | 0 |
| 4 | 1 | App\Models\Page | a.. | A.. | C.. descr.. | https://localhost | https://localhost... | 0 |
I tried already:
DB::table('tags')
->selectRaw("DISTINCT CONCAT(`tagable_id`, '-', `tagable_type`), `id`, `name`, `title`, `description`, `url`, `image`")
->whereIn('name', $tags->toArray())
->orderBy('hits');
This however does not return distinct records - it will return all records regardless of the distinct concatenation - that is in MySQL / MariaDB - in SQLite it will tell me no such function: CONCAT.
I also tried:
DB::table('tags')
->selectRaw("CONCAT(`tagable_id`, '-', `tagable_type`) as `identifier`, `id`, `name`, `title`, `description`, `url`, `image`")
->whereIn('name', $tags->toArray())
->groupBy('identifier')
->orderBy('hits');
This time MySQL/MariaDB tells me that I need to include other fields in the group by as well tags.id' isn't in GROUP BY, but when I use it with SQLite and replace CONCAT function with (tagable_id || '-' || tagable_type) as identifier - it seem to work.
So at this stage I'm: MySQL: 0 | SQLite: 1
Any help would be much appreciated.
UPDATE
After hours of trying to get it resolved I've decided to add a new column
identifier to the table to overcome issue of the non available concat function - my code now looks like this:
Tag::with('tagable')->whereIn('id', function($query) use ($tags) {
$query->selectRaw('min(`id`) from `tags`')
->whereIn('name', $tags->toArray())
->groupBy('identifier');
})
->orderBy('hits')
->take(4)
->get();
This is still not quite what I'm after as it relies on the lowest id min(id) of the given identifier and if the record with the lowest id for the same identifier has higher number of hits then its sibling then the sibling will not be returned.

You can use DISTINCT to get distinct combinations, but not whole rows:
DB::table('tags')->distinct()->get(['tagable_id', 'tagable_type']);
You have to use a more complex query for that:
$join = DB::table('tags')
->select('tagable_id', 'tagable_type')->selectRaw('MIN(hits) as hits')
->whereIn('name', $tags->toArray())
->groupBy('tagable_id', 'tagable_type');
$sql = '(' . $join->toSql() . ') as grouped';
$tags = DB::table('tags')
->join(DB::raw($sql), function($join) {
$join->on('tags.tagable_id', '=', 'grouped.tagable_id')
->on('tags.tagable_type', '=', 'grouped.tagable_type')
->on('tags.hits', '=', 'grouped.hits');
})->mergeBindings($join)
->whereIn('name', $tags->toArray())
->get();
A solution that guarantees one record per unique combination:
$join = DB::table('tags')
->select('tagable_id', 'tagable_type')->selectRaw('MIN(hits) as hits')
->whereIn('name', $tags->toArray())
->groupBy('tagable_id', 'tagable_type');
$sql = '(' . $join->toSql() . ') as grouped';
$ids = DB::table('tags')
->selectRaw('MIN(id) as id')
->join(DB::raw($sql), function($join) {
$join->on('tags.tagable_id', '=', 'grouped.tagable_id')
->on('tags.tagable_type', '=', 'grouped.tagable_type')
->on('tags.hits', '=', 'grouped.hits');
})->mergeBindings($join)
->whereIn('name', $tags->toArray())
->groupBy('tags.tagable_id', 'tags.tagable_type')
->pluck('id');
$tags = DB::table('tags')->whereIn('id', $ids)->get();

Related

Laravel sub-query

I have a database that looks like this:πŸ‘‡πŸ‘‡
images πŸŒ…
| id | name | src | status |
| ------------- |---------------| ------------| ----------|
| 1 | nice sun set | 1020288.jpg | published |
| 2 | poor sun set | 1120288.jpg | published |
| 3 | best sun set | 3120288.jpg | deleted |
| ------------- |---------------| ------------| --------- |
image_views πŸ‘€
| id | image_id | browser_id🌎 | created_at |
| ------------- |---------------| ------------ | ------------------ |
| 1 | 2 | 1020288e3221 |2020-02-23 13:55:11 |
| 2 | 1 | 1120288221ww |2020-02-27 13:50:51 |
| ------------- |---------------| ------------ | ------------------ |
Now in my laravel App,
I want to get the most viewed image in the PAST last 7 days.
( i want to have a column of image_views and those views πŸ‘€ should be grouped by browser id ).
so here is what i have tried:πŸ‘‡πŸ‘‡
$image_views = DB::table('image_views')
->selectRaw('count(*) as view_count')
->where(function($query){
$query->where('image_views.image_id', 'images.id');
$query->whereDate('image_views.created_at', '>=', Carbon::now()->subDays(7)->toDateTimeString() );
});
$image = Image::select(['images.*', DB::raw('(' . $image_views->toSql() . ') as views ')])
->limit(1)
->orderBy('views', 'desc')
->where('images.status','published')
->mergeBindings($image_views)
->get();
return $image;
So unfortunately the posted above☝☝ code does not work😩
It only return blank results.
By the way i have lot of views in image_views table starting from 2⃣0⃣1⃣9⃣ to now, just that i couldn't post all here..
THE FUNNY THING IS THAT IF I CONVERT IT TO SQL AND PASTE IT IN PHPMYADMIN IT WORKS LIKE A CHARM
return $image->toSql();
//->mergeBindings($image_views)
//->get();
PLEASE SOMEONE TELL ME WHAT I AM DOING WRONG IN LARAVEL!!πŸ™Œ
Given images & image_views tables
$mostViewdImage = DB::table('image_views')
->join('images', 'image_views.image_id', '=', 'images.id')
->select('browser_id', DB::raw('count(image_id) as occurrence'), 'images.*')
->where('image_views.created_at', '>=', Carbon::now()->subDays(7)->toDateTimeString())
->groupBy('image_id', 'browser_id')
->orderByRaw('occurrence DESC')->first();
dump($mostViewdImage);
//Output
"select `browser_id`, count(image_id) as occurrence, `images`.* from `image_views` inner join `images` on `image_views`.`image_id` = `images`.`id` where `image_views`.`created_at` >= ? group by `image_id`, `browser_id` order by occurrence DESC limit 1" (2.02 s)
{#261 β–Ό
+"browser_id": "1020288e3221"
+"occurrence": 2
+"id": 2
+"name": "poor sun set"
+"src": "1120288.jpg"
+"status": "published"
}

MySQL - add text prefix before column name in SELECT statement

Here is my table:
| ID | NUMBER |
| 1 | 523 |
| 2 | 293 |
| 3 | 948 |
And now, I want to get all NUMBER values but I want to add in result two numbers - 48 - (without upadting existing results). So finally I want print these results:
| NUMBER |
| 48523 |
| 48293 |
| 48948 |
So I need a query, something like this:
SELECT '48' + `number` FROM `table`
but this query doesn't work fine (this query only update column name from NUMBER to 48 + NUMBER).
Any ideas?
Thanks.
You need CONCAT
SELECT CONCAT('48' , `number`) AS number FROM table
Demo

Custom column to represent order

I am using MySQL to return columns from a data table. What I want to do is use Order By to order the results by a date fields in ascending order and then also return a custom column along with the other desired columns where the custom column represents the index in relation to the Order By results. Right, I have the following query which just returns the desired columns and orders the results:
SELECT
`alert_id`,
`message`,
`expiration`
FROM
`alert`
WHERE
`is_active` = TRUE
ORDER BY
`expiration`
But what I'm having difficulty with is how to also return the custom column that represents the order. So for example, I would like the following (sample data) rows returned:
Pior to Order and Custom Column After the Order and Custom Column
+----------+---------+------------+ +----------+---------+------------+----------+
| alert_id | message | expiration | | alert_id | message | expiration | order_by |
+----------+---------+------------+ +----------+---------+------------+----------+
| 1 | alert1 | 2017-11-20 | | 5 | alert5 | 2017-11-16 | 1 |
| 5 | alert5 | 2017-11-16 | | 6 | alert6 | 2017-11-17 | 2 |
| 6 | alert6 | 2017-11-17 | | 1 | alert1 | 2017-11-20 | 3 |
+----------+---------+------------+ +----------+---------+------------+----------+
With AdrianE's assistance in mentioning the ROW_NUMBER function, even though I am using MySQL and that function is not available, I was able to emulate the function by using the following:
SET #order = 0;
SELECT
`alert_id`,
`message`,
DATE_FORMAT(`expiration`, '%M %D, %Y') AS formatted_date,
(#order := #order + 1) AS order_by
FROM
`alert`
WHERE
`is_active` = TRUE
ORDER BY
`expiration`;

Extract data from json inside mysql field

I've got a a table with rows, and one of the rows has a field with data like this
{"name":"Richard","lastname":null,"city":"Olavarria","cityId":null}
And i want to select all the distinct "city" values i've got. Only using mysql server.
Is it possible? I'm trying with something like this
SELECT id FROM table_name WHERE field_name REGEXP '"key_name":"([^"]*)key_word([^"]*)"';
But i can't make the regexp work
Thanks in advance
MySQL has got support for JSON in version 5.7.7
http://mysqlserverteam.com/json-labs-release-native-json-data-type-and-binary-format/
You will be able to use the jsn_extract function to efficiently parse your JSON string.
If you have an older version and you want to solve it purely in mysql then I am afraid you have to treat it as a string and cut the value out of it (just normal string functions or use regular expressions)
This is not elegant but it will work
http://sqlfiddle.com/#!9/97cfd/14
SELECT
DISTINCT(substring(jsonfield, locate('"city":',jsonfield)+8,
locate('","', jsonfield, locate('"city":',jsonfield))-locate('"city":',jsonfield)-8)
)
FROM
ForgeRock
I have wrapped this into a stored function for those constrained to MySQL <5.7.7:
CREATE FUNCTION `json_extract_string`(
p_json text,
p_key text
) RETURNS varchar(40) CHARSET latin1
BEGIN
SET #pattern = CONCAT('"', p_key, '":"');
SET #start_i = LOCATE(#pattern, p_json) + CHAR_LENGTH(#pattern);
if #start_i = CHAR_LENGTH(#pattern) then
SET #end_i = 0;
else
SET #end_i = LOCATE('"', p_json, #start_i) - #start_i;
end if;
RETURN SUBSTR(p_json, #start_i, #end_i);
END
Note this only works with string values but is a bit more robust than #DmitryK's answer, in that it returns an empty string if the key is not found and the key can be anywhere in the JSON string.
Yes , you can definitely to it using JSON_EXTRACT() function in mysql.
lets take a table that contains JSON (table client_services here) :
+-----+-----------+--------------------------------------+
| id | client_id | service_values |
+-----+-----------+------------+-------------------------+
| 100 | 1000 | { "quota": 1,"data_transfer":160000} |
| 101 | 1000 | { "quota": 2,"data_transfer":800000} |
| 102 | 1000 | { "quota": 3,"data_transfer":70000} |
| 103 | 1001 | { "quota": 1,"data_transfer":97000} |
| 104 | 1001 | { "quota": 2,"data_transfer":1760} |
| 105 | 1002 | { "quota": 2,"data_transfer":1060} |
+-----+-----------+--------------------------------------+
To Select each JSON fields , run this query :
SELECT
id, client_id,
json_extract(service_values, '$.quota') AS quota,
json_extract(service_values, '$.data_transfer') AS data_transfer
FROM client_services;
So the output will be :
+-----+-----------+----------------------+
| id | client_id | quota | data_transfer|
+-----+-----------+----------------------+
| 100 | 1000 | 1 | 160000 |
| 101 | 1000 | 2 | 800000 |
| 102 | 1000 | 3 | 70000 |
| 103 | 1001 | 1 | 97000 |
| 104 | 1001 | 2 | 1760 |
| 105 | 1002 | 2 | 1060 |
+-----+-----------+----------------------+
NOW, if you want lets say DISTINCT quota , then run this query :
SELECT
distinct( JSON_EXTRACT(service_values, '$.quota')) AS quota
FROM client_services;
So this will result into your desired output :
+-------+
| quota |
+-------+
| 1 |
| 2 |
| 3 |
+-------+
hope this helps!
See MariaDB's Dynamic Columns.
Also, search this forum for [mysql] [json]; the topic has been discussed often.
This may be a little late, but the accepted answer didn't work for me. I used SUBSTRING_INDEX to achieve the desired result.
SELECT
ID, SUBSTRING_INDEX(SUBSTRING_INDEX(JSON, '"mykey" : "', -1), '",', 1) MYKEY
FROM MY_TABLE;
Hope this helps.

SQL select statement optimizing (id, parent_id, child_ids)

we have a very old custom db (oracle, mysql, derby) with the restrictions: no new table fileds, no views, no functions, no procedures.
My table MYTABLE:
| id | ... | parent_id |
------------------------
| 1 | ... | |
| 2 | ... | 1 |
| 3 | ... | 1 |
| 4 | ... | 2 |
| 5 | ... | 1 |
and I my first statement:
select * from MYTABLE where id in ('1','2','3','4','5');
give my 5 records.
Then I need the information about the first (no deeper) child ids.
My current solution:
for (record in records) {
// get child ids as comma separated string list
// e.g. "2,3,5" for id 1
String childIds = getChildIds(record.id);
}
with the second statement in getChildIds(record.Id):
select id from MYTABLE where parent_id='record.Id';
So I have 1 + 5 = 6 statements for the required information.
I'm looking for a solution to select the records from the following "imaginary" table with the "imaginary" field "child_ids":
| id | ... | parent_id | child_ids |
------------------------------------
| 1 | ... | | 2,3,5 |
| 2 | ... | 1 | 4 |
| 3 | ... | 1 | |
| 4 | ... | 2 | |
| 5 | ... | 1 | |
Does anyone have an idea how I can get this information with only one statement (or with 2 statements)?
Thanks for your help, Thomas
FOR MYSQL:
How about using the GROUP_CONCAT() function like the following:
SELECT id, parent_id,
GROUP_CONCAT(child_id ORDER BY child_id SEPARATOR ',') AS child_ids
FROM MYTABLE
WHERE id IN ('1','2','3','4','5')
FOR ORACLE:
If you have a later version of Oracle you could use the LISTAGG() function:
SELECT parent_id,
LISTAGG(child_id, ', ') WITHIN GROUP (ORDER BY child_id) "child_ids"
FROM MYTABLE
WHERE id IN ('1','2','3','4','5')
GROUP BY parent_id
FOR DERBY:
I don't know anything about derby, but doing a little research it uses IBM DB2 SQL syntax. So, maybe using a combination of XMLSERIALIZE(), XMLAGG(), and XMLTEXT() will work for you:
SELECT parent_id,
XMLSERIALIZE(XMLAGG(XMLTEXT(child_id) ORDER BY child_id) AS CLOB(30K))
FROM table GROUP BY parent_id