Sorting/ordering in MySQL - mysql

I'm having a little problem with trying to sort the contents of a table programs by the column prog_id which holds the id of each program in the following format:
prog_id
1.0.1, 1.0.2, 1.0.3, ..., 1.0.10, 1.0.11, ..., 1.1.0, 1.1.1 etc
When I sort by prog_id i get
1.0.1, 1.0.10, 1.0.11, 1.0.2, 1.0.3 ...
which is correct as far as MySQL goes but not correct for the order in which the data should display.
I tried using another column, orderby in which I could save an index and order by that but I would have to enter the values manually and there are a few thousand rows in my table which would take quite a long time to do.
Any tricks I could use to get my data to display in the "proper" order? BTW, I'm using PHP & MySQL.


You could split them into their constituent parts like:
SELECT REPLACE(SUBSTRING(SUBSTRING_INDEX(prog_id, '.', 1),
LENGTH(SUBSTRING_INDEX(prog_id, '.', 1 -1)) + 1),
'.', '') AS id1,
REPLACE(SUBSTRING(SUBSTRING_INDEX(prog_id, '.', 2),
LENGTH(SUBSTRING_INDEX(prog_id, '.', 2 -1)) + 1),
'.', '') AS id2,
REPLACE(SUBSTRING(SUBSTRING_INDEX(prog_id, '.', 3),
LENGTH(SUBSTRING_INDEX(prog_id, '.', 3 -1)) + 1),
'.', '') AS id3
FROM programs
ORDER BY CAST(id1 AS INT(4)), CAST(id2 AS INT(4)), CAST(id3 AS INT(4))
The best method would be to create the the extra fields like yoda2k says, but if you don't have that access then you could use the above.
You could encapsulate that into a function like:
CREATE FUNCTION SPLIT_STR(
x VARCHAR(255),
delim VARCHAR(12),
pos INT
)
RETURNS VARCHAR(255)
RETURN REPLACE(SUBSTRING(SUBSTRING_INDEX(x, delim, pos),
LENGTH(SUBSTRING_INDEX(x, delim, pos -1)) + 1),
delim, '');
Then do:
SELECT SPLIT_STR(prog_id, '.', 1) AS id1,
SPLIT_STR(prog_id, '.', 2) AS id2,
SPLIT_STR(prog_id, '.', 3) AS id3,
FROM programs
ORDER BY CAST(id1 AS INT(4)), CAST(id2 AS INT(4)), CAST(id3 AS INT(4))


Not optimal solution -
...ORDER BY substring_index(prog_id, '.', 1), substring_index(substring_index(prog_id, '.', 2), '.', -1), substring_index(prog_id, '.', -1)
Odd solution, but try it -
...ORDER BY INET_ATON(prog_id)


You could use 3 fields e.g. major_version, minor_version, build_number, make them integer fields and use mysqls buildin "ORDER BY major_version,minor_version,build_number" which will order the fields in the desired way.

Related

Replace the special characters

data = "Qwsdyz_qwrbc_bcD_qwEr"
What I need is:
remove all the _
all characters to be in lower
the starting letter should be caps for all the 4 like this (QwsdyzQwrbcBcdQwer)
whatever changes made in above statement should't change the result like if we changed like Qwsdqwqyz_qwrwqeqwebc_bcqwD_qqwwEr_dadakjas i need the result like QesdqwqyzQwrwqeqwebcBcqwdQqwwerDasakjas
Please help me with MySQL coding.
set #data="Qwsdyz_qwrbc_bcD_qwEr";
select lower(SUBSTRING_INDEX(#data,"_",1)) into #data1;
select ucase(left(SUBSTRING_INDEX(#data,"_",2),1)) into #data2;
select lower(SUBSTRING_INDEX(#data,"_",2)) into #data3;
select substring(reverse(SUBSTRING_INDEX(reverse(#data3),"_",1)),2) into #data4;
select reverse((lower(SUBSTRING_INDEX(#data,"_",3)))) into #data5;
select (reverse(SUBSTRING_INDEX(#data5,"_",1))) into #data6;
select ucase(left(#data6,1)) into #data7;
select substring(#data6,2) into #data8;
select reverse(#data) into #data9;
select reverse(lower(SUBSTRING_INDEX(#data9,"_",1))) into #data10;
select ucase(left(#data10,1)) into #data11;
select substring(#data10,2) into #data12;
select concat(#data1,#data2,#data4,#data7,#data8,#data11,#data12) data;

You can use split functions replied in this question. Do sub-string and find every splited strings first character with UPPER() function do it upper character and LOWER() function do other characters to lowercase. And finally join with CONCAT() function.

This is just a messy concat() and substring_index():
select concat(concat(upper(left(lower(substring_index(data, '_', 1)), 1)),
lower(substr(lower(substring_index(data, '_', 1)), 2))
),
concat(upper(left(lower(substring_index(substring_index(data, '_', 2), '_', -1)), 1)),
lower(substr(lower(substring_index(substring_index(data, '_', 2), '_', -1)), 2))
),
concat(upper(left(lower(substring_index(substring_index(data, '_', 3), '_', -1)), 1)),
lower(substr(lower(substring_index(substring_index(data, '_', 3), '_', -1)), 2))
),
concat(upper(left(lower(substring_index(substring_index(data, '_', 4), '_', -1)), 1)),
lower(substr(lower(substring_index(substring_index(data, '_', 4), '_', -1)), 2))
)
)
from (select 'Qwsdyz_qwrbc_bcD_qwEr' as data) x
SQL is not optimized for string manipulations. I would advise you to do this in another tool, such as Python, if that is possible.
Here is a db<>fiddle.

alternative to splitting MySQL query result in multiple rows [duplicate]

I have a column that has comma separated data:
1,2,3
3,2,1
4,5,6
5,5,5
I'm trying to run a search that would query each value of the CSV string individually.
0<first<5 and 1<second<3 and 2<third<4
I get that I could return all queries and split it myself and compare it myself. I'm curious if there is a way to do this so MySQL does that processing work.
Thanks!

Use
substring_index(`column`,',',1) ==> first value
substring_index(substring_index(`column`,',',-2),',',1)=> second value
substring_index(substring_index(`column`,',',-1),',',1)=> third value
in your where clause.
SELECT * FROM `table`
WHERE
substring_index(`column`,',',1)<0
AND
substring_index(`column`,',',1)>5

It seems to work:
substring_index ( substring_index ( context,',',1 ), ',', -1)
substring_index ( substring_index ( context,',',2 ), ',', -1)
substring_index ( substring_index ( context,',',3 ), ',', -1)
substring_index ( substring_index ( context,',',4 ), ',', -1)
it means 1st value, 2nd, 3rd, etc.
Explanation:
The inner substring_index returns the first n values that are comma separated. So if your original string is "34,7,23,89", substring_index( context,',', 3) returns "34,7,23".
The outer substring_index takes the value returned by the inner substring_index and the -1 allows you to take the last value. So you get "23" from the "34,7,23".
Instead of -1 if you specify -2, you'll get "7,23", because it took the last two values.
Example:
select * from MyTable where substring_index(substring_index(prices,',',1),',',-1)=3382;
Here, prices is the name of a column in MyTable.

Usually substring_index does what you want:
mysql> select substring_index("foo#gmail.com","#",-1);
+-----------------------------------------+
| substring_index("foo#gmail.com","#",-1) |
+-----------------------------------------+
| gmail.com |
+-----------------------------------------+
1 row in set (0.00 sec)

You may get what you want by using the MySQL REGEXP or LIKE.
See the MySQL Docs on Pattern Matching

As an addendum to this, I've strings of the form:
Some words 303
where I'd like to split off the numerical part from the tail of the string.
This seems to point to a possible solution:
http://lists.mysql.com/mysql/222421
The problem however, is that you only get the answer "yes, it matches", and not the start index of the regexp match.

Here is another variant I posted on related question. The REGEX check to see if you are out of bounds is useful, so for a table column you would put it in the where clause.
SET #Array = 'one,two,three,four';
SET #ArrayIndex = 2;
SELECT CASE
WHEN #Array REGEXP CONCAT('((,).*){',#ArrayIndex,'}')
THEN SUBSTRING_INDEX(SUBSTRING_INDEX(#Array,',',#ArrayIndex+1),',',-1)
ELSE NULL
END AS Result;
SUBSTRING_INDEX(string, delim, n) returns the first n
SUBSTRING_INDEX(string, delim, -1) returns the last only
REGEXP '((delim).*){n}' checks if there are n delimiters (i.e. you are in bounds)

Building on #Oleksiy's answer, here is one that can work with strings of variable segment lengths (within reasonable limits), for example comma-separated addresses:
SELECT substring_index ( substring_index ( address,',',1 ), ',', -1) AS address_line_1,
IF(address_parts > 1, substring_index ( substring_index ( address,',',2 ), ',', -1), '') AS address_line_2,
IF(address_parts > 2, substring_index ( substring_index ( address,',',3 ), ',', -1), '') AS address_line_3,
IF(address_parts > 3, substring_index ( substring_index ( address,',',4 ), ',', -1), '') AS address_line_4,
IF(address_parts > 4, substring_index ( substring_index ( address,',',5 ), ',', -1), '') AS address_line_5
FROM (
SELECT address, LENGTH(address) - LENGTH(REPLACE(address, ',', '')) AS address_parts
FROM mytable
) AS addresses

It's working..
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(
SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(col,'1', 1), '2', 1), '3', 1), '4', 1), '5', 1), '6', 1)
, '7', 1), '8', 1), '9', 1), '0', 1) as new_col
FROM table_name group by new_col;

Mysql query trivia

Updated: Please assume, I will have exactly 4 elements in my version number x.x.x.x
Table xyz (id int, os varchar, version varchar)
id os version
1 mac 1.0.0.4
2 android 1.0.1.2
3 mac 1.0.0.14
4 mac 1.0.0.07
I want to find the maximum version number of each operating system.
select max(version), os from xyz group by os
but in the above data sample, it returns 1.0.0.4 as the highest version of mac, rather than returning 1.0.0.14. Is there any way to fix the above query? I know version is varchar :( If there is NO solution possible, then I can change datatype to other, but that will still not solve the issue.

If the 4 parts in the version are all numbers and not bigger than 255, you can use the INET_ATON() function, and its reverse INET_NTOA():
SELECT INET_NTOA(MAX(INET_ATON(version))), os
FROM xyz
GROUP BY os ;
Tested in SQL-Fiddle

Mysql uses text sort for this field when you try to get max:
SELECT version FROM test ORDER BY VERSION;
1.0.0.14
1.0.0.4
1.0.07
1.0.1.2.4
But even if you try to cast this to integer, you will get
SELECT version FROM test ORDER BY CAST(VERSION AS SIGNED);
1
1
1
1
And this really has a sense. How do you want MySQL to guess, what you want?
MySQL should be used to store data, and you should format it yourself.
A fine solution would be to use several fields for version:
id int, os varchar, versionMajor int, versionMinor int, versionMoreMinor int, versionEvenMoreMinor int
You should be able to sort and to format them as you wish.

It is ugly, but it's doing the job:
select t1.os,t1.version from xyz t1 where (t1.os,
((cast(SUBSTRING_INDEX(t1.version, '.', 1) as unsigned)+1)*1000
+(cast(SUBSTRING_INDEX(SUBSTRING_INDEX(t1.version, '.', -3), '.', 1) as unsigned)+1)*100
+(cast(SUBSTRING_INDEX(SUBSTRING_INDEX(t1.version, '.', -2), '.', 1) as unsigned)+1)*10
+(cast(SUBSTRING_INDEX(t1.version, '.', -1) as unsigned))+1))
=(select t2.os,
max((cast(SUBSTRING_INDEX(t2.version, '.', 1) as unsigned)+1)*1000
+(cast(SUBSTRING_INDEX(SUBSTRING_INDEX(t2.version, '.', -3), '.', 1) as unsigned)+1)*100
+(cast(SUBSTRING_INDEX(SUBSTRING_INDEX(t2.version, '.', -2), '.', 1) as unsigned)+1)*10
+(cast(SUBSTRING_INDEX(t2.version, '.', -1) as unsigned))+1)
from xyz t2 where t2.os=t1.os);
sqlfiddle

What you'd really like to do here is use a SPLIT function, but that's not available in MySQL. However, you can write a user-defined function that will do the same thing, or refactoring this function code into a view.
CREATE FUNCTION SPLIT_STR(
x VARCHAR(255),
delim VARCHAR(12),
pos INT
)
RETURNS VARCHAR(255)
RETURN REPLACE(SUBSTRING(SUBSTRING_INDEX(x, delim, pos),
LENGTH(SUBSTRING_INDEX(x, delim, pos -1)) + 1),
delim, '');
Source.
Additionally, you may want to consider that the reason this is such a pain in the butt is because you're essentially violating first normal form. A version number of the form you specify is actually four distinct values: major version, minor version, revision (or maintenance), and build. Your table structure should use the same format:
Table xyz (id int not null, os varchar not null, version_major int not null, version_minor int not null, version_revision int null, version_build int null)
Then you just use an ORDER BY clause to sort the data. You can use a calculated field or a view to display the formatted version number.
Now, I know why you chose to use a single field. Version numbers are inconsistent, and many version numbers do not include revision or build values, other include letters, etc. You can change the int to a varchar and cover 90% of cases in my experience, however.

You may try to convert string to the integer and get MAX()
SELECT MAX( CONVERT( REPLACE( `version`, '.', '' ), UNSIGNED ) ) from xyz
upd
There are some another way with ending detection, but it doesn't work properly with minor version, maybe you will tune this heavy query to get it works well for you
SELECT `version`,
CONVERT(
CONCAT_WS( '.',
REPLACE( SUBSTRING( `version`, 1, ( CHAR_LENGTH( `version` ) - LOCATE( '.', REVERSE( `version` ) ) ) ), '.', '' ),
SUBSTRING( `version`, 1 + LOCATE( '.', REVERSE( `version` ) ) * -1 )
),
DECIMAL ( 10, 3 )
)
FROM `xyz`
sorry, code improved

Thanks to #ntvf for idea.
This will do what I am looking for.
select a.os, xyz.version from xyz,
(SELECT os, MAX( CONVERT(REPLACE(`version`,'.',''),UNSIGNED)) as version from xyz group by os) a
where
a.os=xyz.os and
CONVERT(REPLACE( xyz.version, '.', '' ), UNSIGNED) = a.version

mysql split string [duplicate]

I have a column that has comma separated data:
1,2,3
3,2,1
4,5,6
5,5,5
I'm trying to run a search that would query each value of the CSV string individually.
0<first<5 and 1<second<3 and 2<third<4
I get that I could return all queries and split it myself and compare it myself. I'm curious if there is a way to do this so MySQL does that processing work.
Thanks!

Use
substring_index(`column`,',',1) ==> first value
substring_index(substring_index(`column`,',',-2),',',1)=> second value
substring_index(substring_index(`column`,',',-1),',',1)=> third value
in your where clause.
SELECT * FROM `table`
WHERE
substring_index(`column`,',',1)<0
AND
substring_index(`column`,',',1)>5

It seems to work:
substring_index ( substring_index ( context,',',1 ), ',', -1)
substring_index ( substring_index ( context,',',2 ), ',', -1)
substring_index ( substring_index ( context,',',3 ), ',', -1)
substring_index ( substring_index ( context,',',4 ), ',', -1)
it means 1st value, 2nd, 3rd, etc.
Explanation:
The inner substring_index returns the first n values that are comma separated. So if your original string is "34,7,23,89", substring_index( context,',', 3) returns "34,7,23".
The outer substring_index takes the value returned by the inner substring_index and the -1 allows you to take the last value. So you get "23" from the "34,7,23".
Instead of -1 if you specify -2, you'll get "7,23", because it took the last two values.
Example:
select * from MyTable where substring_index(substring_index(prices,',',1),',',-1)=3382;
Here, prices is the name of a column in MyTable.

Usually substring_index does what you want:
mysql> select substring_index("foo#gmail.com","#",-1);
+-----------------------------------------+
| substring_index("foo#gmail.com","#",-1) |
+-----------------------------------------+
| gmail.com |
+-----------------------------------------+
1 row in set (0.00 sec)

You may get what you want by using the MySQL REGEXP or LIKE.
See the MySQL Docs on Pattern Matching

As an addendum to this, I've strings of the form:
Some words 303
where I'd like to split off the numerical part from the tail of the string.
This seems to point to a possible solution:
http://lists.mysql.com/mysql/222421
The problem however, is that you only get the answer "yes, it matches", and not the start index of the regexp match.

Here is another variant I posted on related question. The REGEX check to see if you are out of bounds is useful, so for a table column you would put it in the where clause.
SET #Array = 'one,two,three,four';
SET #ArrayIndex = 2;
SELECT CASE
WHEN #Array REGEXP CONCAT('((,).*){',#ArrayIndex,'}')
THEN SUBSTRING_INDEX(SUBSTRING_INDEX(#Array,',',#ArrayIndex+1),',',-1)
ELSE NULL
END AS Result;
SUBSTRING_INDEX(string, delim, n) returns the first n
SUBSTRING_INDEX(string, delim, -1) returns the last only
REGEXP '((delim).*){n}' checks if there are n delimiters (i.e. you are in bounds)

Building on #Oleksiy's answer, here is one that can work with strings of variable segment lengths (within reasonable limits), for example comma-separated addresses:
SELECT substring_index ( substring_index ( address,',',1 ), ',', -1) AS address_line_1,
IF(address_parts > 1, substring_index ( substring_index ( address,',',2 ), ',', -1), '') AS address_line_2,
IF(address_parts > 2, substring_index ( substring_index ( address,',',3 ), ',', -1), '') AS address_line_3,
IF(address_parts > 3, substring_index ( substring_index ( address,',',4 ), ',', -1), '') AS address_line_4,
IF(address_parts > 4, substring_index ( substring_index ( address,',',5 ), ',', -1), '') AS address_line_5
FROM (
SELECT address, LENGTH(address) - LENGTH(REPLACE(address, ',', '')) AS address_parts
FROM mytable
) AS addresses

It's working..
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(
SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(col,'1', 1), '2', 1), '3', 1), '4', 1), '5', 1), '6', 1)
, '7', 1), '8', 1), '9', 1), '0', 1) as new_col
FROM table_name group by new_col;

Can MySQL split a column?

I have a column that has comma separated data:
1,2,3
3,2,1
4,5,6
5,5,5
I'm trying to run a search that would query each value of the CSV string individually.
0<first<5 and 1<second<3 and 2<third<4
I get that I could return all queries and split it myself and compare it myself. I'm curious if there is a way to do this so MySQL does that processing work.
Thanks!

Use
substring_index(`column`,',',1) ==> first value
substring_index(substring_index(`column`,',',-2),',',1)=> second value
substring_index(substring_index(`column`,',',-1),',',1)=> third value
in your where clause.
SELECT * FROM `table`
WHERE
substring_index(`column`,',',1)<0
AND
substring_index(`column`,',',1)>5

It seems to work:
substring_index ( substring_index ( context,',',1 ), ',', -1)
substring_index ( substring_index ( context,',',2 ), ',', -1)
substring_index ( substring_index ( context,',',3 ), ',', -1)
substring_index ( substring_index ( context,',',4 ), ',', -1)
it means 1st value, 2nd, 3rd, etc.
Explanation:
The inner substring_index returns the first n values that are comma separated. So if your original string is "34,7,23,89", substring_index( context,',', 3) returns "34,7,23".
The outer substring_index takes the value returned by the inner substring_index and the -1 allows you to take the last value. So you get "23" from the "34,7,23".
Instead of -1 if you specify -2, you'll get "7,23", because it took the last two values.
Example:
select * from MyTable where substring_index(substring_index(prices,',',1),',',-1)=3382;
Here, prices is the name of a column in MyTable.

Usually substring_index does what you want:
mysql> select substring_index("foo#gmail.com","#",-1);
+-----------------------------------------+
| substring_index("foo#gmail.com","#",-1) |
+-----------------------------------------+
| gmail.com |
+-----------------------------------------+
1 row in set (0.00 sec)

You may get what you want by using the MySQL REGEXP or LIKE.
See the MySQL Docs on Pattern Matching

As an addendum to this, I've strings of the form:
Some words 303
where I'd like to split off the numerical part from the tail of the string.
This seems to point to a possible solution:
http://lists.mysql.com/mysql/222421
The problem however, is that you only get the answer "yes, it matches", and not the start index of the regexp match.

Here is another variant I posted on related question. The REGEX check to see if you are out of bounds is useful, so for a table column you would put it in the where clause.
SET #Array = 'one,two,three,four';
SET #ArrayIndex = 2;
SELECT CASE
WHEN #Array REGEXP CONCAT('((,).*){',#ArrayIndex,'}')
THEN SUBSTRING_INDEX(SUBSTRING_INDEX(#Array,',',#ArrayIndex+1),',',-1)
ELSE NULL
END AS Result;
SUBSTRING_INDEX(string, delim, n) returns the first n
SUBSTRING_INDEX(string, delim, -1) returns the last only
REGEXP '((delim).*){n}' checks if there are n delimiters (i.e. you are in bounds)

Building on #Oleksiy's answer, here is one that can work with strings of variable segment lengths (within reasonable limits), for example comma-separated addresses:
SELECT substring_index ( substring_index ( address,',',1 ), ',', -1) AS address_line_1,
IF(address_parts > 1, substring_index ( substring_index ( address,',',2 ), ',', -1), '') AS address_line_2,
IF(address_parts > 2, substring_index ( substring_index ( address,',',3 ), ',', -1), '') AS address_line_3,
IF(address_parts > 3, substring_index ( substring_index ( address,',',4 ), ',', -1), '') AS address_line_4,
IF(address_parts > 4, substring_index ( substring_index ( address,',',5 ), ',', -1), '') AS address_line_5
FROM (
SELECT address, LENGTH(address) - LENGTH(REPLACE(address, ',', '')) AS address_parts
FROM mytable
) AS addresses

It's working..
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(
SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(col,'1', 1), '2', 1), '3', 1), '4', 1), '5', 1), '6', 1)
, '7', 1), '8', 1), '9', 1), '0', 1) as new_col
FROM table_name group by new_col;