I have a "person" column in a mySQL database that represents the age and weight of a person as a string separated by a comma.
Example:
"24,175"
I want to be able to separate and extract those values and cast them as numbers.
Example: turn "24,175" to
24 as age
175 as weight
So that I can write a query similar to the following
SELECT person
FROM TABLE
WHERE age>140 OR weight>1000
I want to be able to check for values that are not possible. i.e age>140 OR weight >1000.
I cannot modify the table/environment I'm working with
I only have access to queries.
I'm thinking about solving it this way
find the index where the comma exists. CHARINDEX(',',person)
Split the string into substrings using LEFT , RIGHT, CAST and CHARINDEX(',',person)
Cast age substring and weight substring to numbers using CAST(age AS INT) CAST(weight AS INT)
SELECT person
FROM TABLE
WHERE CAST(LEFT(person,CHARINDEX(',',person) AS INT)>150 OR CAST(RIGHT(person,CHARINDEX(',',person) AS INT) >1000
If I did anything wrong please correct me.
Are all the functions usable/supported by mySQL? (RIGHT, LEFT, CHARINDEX) Will this work?
Exception: Another value for this column could be "unknown". Will this cause errors if we're trying to check for the index of , if it doesn't exist in the string? Is there a way to include "unknown" cases in the result and have it output a message of "error, person not recognized"
you can also split is with SUBSTR_INDEX like this:
MariaDB [yourschema]> SELECT * FROM spliit;
+----+--------+
| id | d |
+----+--------+
| 1 | 24,175 |
+----+--------+
1 row in set (0.03 sec)
MariaDB [yourschema]> SELECT
-> SUBSTRING_INDEX(d, ',', 1) AS age
-> , SUBSTRING_INDEX(d, ',', -1) AS weight
->
-> FROM spliit;
+------+--------+
| age | weight |
+------+--------+
| 24 | 175 |
+------+--------+
1 row in set (0.00 sec)
MariaDB [yourschema]>
sample
yes, you can direct calculate with it in MySQL
MariaDB [yourschema]> SELECT
-> SUBSTRING_INDEX(d, ',', 1) + 2 AS age
-> , SUBSTRING_INDEX(d, ',', 1) * 12 AS `month`
-> , SUBSTRING_INDEX(d, ',', -1) + 3 AS weight
-> FROM spliit;
+------+-------+--------+
| age | month | weight |
+------+-------+--------+
| 26 | 288 | 178 |
+------+-------+--------+
1 row in set, 1 warning (0.03 sec)
MariaDB [yourschema]>
SELECT person
FROM TABLE
WHERE CAST(LEFT(person,LOCATE(',',person) AS INTEGER)>150 OR CAST(RIGHT(person,(LOCATE(',',person)+1) AS INTEGER) >1000
Instead of Char index use LOCATE im MqSQL
Also note the CAST function
You also can use VIRTUAL PERSITENT COLUMNS that calculate the fields automatis and you can also use a INDEX on each substr / Integer.
sample
MariaDB [yourschema]> CREATE TABLE `splitit` (
-> `id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
-> `d` VARCHAR(32) DEFAULT NULL,
-> age INT(11) AS (SUBSTRING_INDEX(d, ',', 1)) PERSISTENT,
-> weight INT(5) AS (SUBSTRING_INDEX(d, ',', -1)) PERSISTENT,
-> PRIMARY KEY (`id`),
-> INDEX idx_age (age),
-> INDEX idx_weight (weight)
-> ) ENGINE=INNODB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.79 sec)
MariaDB [yourschema]> INSERT INTO splitit (d) VALUES ('11,234'),('2,66'),('5,2');
Query OK, 3 rows affected (0.06 sec)
Records: 3 Duplicates: 0 Warnings: 0
MariaDB [yourschema]> SELECT * FROM splitit;
+----+--------+------+--------+
| id | d | age | weight |
+----+--------+------+--------+
| 1 | 11,234 | 11 | 234 |
| 2 | 2,66 | 2 | 66 |
| 3 | 5,2 | 5 | 2 |
+----+--------+------+--------+
3 rows in set (0.00 sec)
MariaDB [yourschema]>
You can do this all in the where clause:
where substring_index(person, ',', 1) + 0 > 140 or
substring_index(person, ',' -1) + 0 > 1000
Note that the + 0 does an silent conversion to integers. And, substring_index()is much more convenient than the functions in SQL Server.
You can readily incorporate this logic into a view:
create view v_table as
select t.*,
substring_index(person, ',', 1) + 0 as age,
substring_index(person, ',' -1) + 0 as weight
from table t;
If you want to filter out bad values within the view, you can use a MySQL extension and add:
having age > 140 or weight > 1000
after the from clause.
Related
Case
In our MySql database the data is stored in combined json-strings like this:
| ID | DATA |
| 100 | {var1str: "sometxt", var2double: 0,01, var3integer: 1, var4str: "another text"} |
| 101 | {var3integer: 5, var2double: 2,05, var1str: "txt", var4str: "more text"} |
Problem
Most of the DATA-fields hold over 2500 variables. The order of variables in the DATA-string is random (as shown in above example). Right now we only know how to extract data with the following querie:
select
ID,
json_extract(DATA,'var1str'),
json_extract(DATA,'var2double'),
FROM table
With this querie, only the values of var1str and var2double will be returned as result. Values of variable 3 and 4 are ignored. There is no overview of what possible variables are hiding in the data fields.
With almost 60.000 entries and over 3.000 possible unique variable names, I would like to create a query that loops through all of the 60.000 DATA-fields and extracts every unique variable name that is found in there.
Solution?
The querie I am looking for would give the following result:
var1str
var2double
var3integer
var4str
My knowledge of MySql is very limited. Any direction given to get to this solution is much appreciated.
What version of MySQL are you using?.
From MySQL 8.0.4 and later JSON_TABLE function is supported and can be useful in this case.
mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 8.0.11 |
+-----------+
1 row in set (0.00 sec)
mysql> DROP TABLE IF EXISTS `table`;
Query OK, 0 rows affected (0.09 sec)
mysql> CREATE TABLE IF NOT EXISTS `table` (
-> `ID` BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
-> `DATA` JSON NOT NULL
-> ) AUTO_INCREMENT=100;
Query OK, 0 rows affected (0.00 sec)
mysql> INSERT INTO `table`
-> (`DATA`)
-> VALUES
-> ('{"var1str": "sometxt", "var2double": 0.01, "var3integer": 1, "var4str": "another text"}'),
-> ('{"var3integer": 5, "var2double": 2.05, "var1str": "txt", "var4str": "more text"}');
Query OK, 2 rows affected (0.00 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> SELECT
-> DISTINCT `der`.`key`
-> FROM
-> `table`,
-> JSON_TABLE(
-> JSON_KEYS(`DATA`), '$[*]'
-> COLUMNS(
-> `key` VARCHAR(64) PATH "$"
-> )
-> ) `der`;
+-------------+
| key |
+-------------+
| var1str |
| var4str |
| var2double |
| var3integer |
+-------------+
4 rows in set (0.01 sec)
Be aware of the Bug #90610 ERROR 1142 (42000) when using JSON_TABLE.
I have a SQL script that works on MySQL (up to current versions). I built a copy of MariaDB-10.2.8 from source on a Linux machine and got the database up and running. However, my SQL script fails, because MariaDB is returning NULL from an UNHEX() call, where it shouldn't.
The call is producing a 20-byte random binary string in a particular format (it's a BitTorrent node ID). I concatenate some required bytes with some random bytes, with certain bytes being limited to a particular range of values. These are constructed as a 40-character hex string, which I then run through UNHEX().
The SQL is:
unhex( concat( '414C2',
hex( 8 + round( rand() * 7 ) ),
substr( sha( uuid( ) ), 6, 33 ),
hex( 2 + round( rand( ) ) * 8 ) ) )
If you take off the UNHEX() function, you get a 40-character hex string:
MariaDB [bt]> select concat('414c2', hex(8+round(rand()*7)),substr(sha(uuid()),6,33),hex(2+round(rand())*8));
+-----------------------------------------------------------------------------------------+
| concat('414c2', hex(8+round(rand()*7)),substr(sha(uuid()),6,33),hex(2+round(rand())*8)) |
+-----------------------------------------------------------------------------------------+
| 414c29115056f1bd332d4e2e3eb5edd3fc90c0a2 |
+-----------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
but if you UNHEX() it:
MariaDB [bt]> select unhex(concat('414c2', hex(8+round(rand()*7)),substr(sha(uuid()),6,33),hex(2+round(rand())*8)));
+------------------------------------------------------------------------------------------------+
| unhex(concat('414c2', hex(8+round(rand()*7)),substr(sha(uuid()),6,33),hex(2+round(rand())*8))) |
+------------------------------------------------------------------------------------------------+
| NULL |
+------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
By contrast, the same command on a MySQL instance:
mysql> select unhex(upper(concat('414c2', hex(8+round(rand()*7)),substr(sha(uuid()),6,33),hex(2+round(rand())*8))));
+-------------------------------------------------------------------------------------------------------+
| unhex(upper(concat('414c2', hex(8+round(rand()*7)),substr(sha(uuid()),6,33),hex(2+round(rand())*8)))) |
+-------------------------------------------------------------------------------------------------------+
| AL*w??
???r?%?? |
+-------------------------------------------------------------------------------------------------------+
1 row in set (0.02 sec)
UNHEX() of a 40-character hex string where all the bytes are printable works OK on MariaDB:
MariaDB [bt]> select unhex('4142434445464748494a4b4c4d4e4f5051525354');
+---------------------------------------------------+
| unhex('4142434445464748494a4b4c4d4e4f5051525354') |
+---------------------------------------------------+
| ABCDEFGHIJKLMNOPQRST |
+---------------------------------------------------+
1 row in set (0.00 sec)
Any idea why a random hex string wouldn't work?
Looks like a bug. We've filed a report for this:
https://jira.mariadb.org/browse/MDEV-13793
OK this is not a question 100%.
I got a big headache today with this and just wanted to know if anyone else had this problem.
Let's say we have a table named "products":
ID | product_code
4 | 5201279007942_AKYRO
If we run this query:
SELECT *
FROM `products`
WHERE product_code
IN ( '5201279007942' )
Mysql will return 0 rows as expected.
If we run this query:
SELECT *
FROM `products`
WHERE product_code
IN ( 5201279007942 )
Mysql will return the above row.
So the "IN" query will act as a "LIKE" query.
Is this behavior normal? Am i missing something here?
see the output. There is a difference between string ant int
SELECT
'5201279007942_AKYRO' IN ('5201279007942') AS string_with_string
, '5201279007942_AKYRO' IN (5201279007942) AS string_with_num
, CAST( '5201279007942_AKYRO' AS UNSIGNED) IN (5201279007942) AS cast_to_int_with_num;
sample
mysql> SELECT
-> '5201279007942_AKYRO' IN ('5201279007942') AS string_with_string
-> , '5201279007942_AKYRO' IN (5201279007942) AS string_with_num
-> , CAST( '5201279007942_AKYRO' AS UNSIGNED) IN (5201279007942) AS cast_to_int_with_num;
+--------------------+-----------------+----------------------+
| string_with_string | string_with_num | cast_to_int_with_num |
+--------------------+-----------------+----------------------+
| 0 | 1 | 1 |
+--------------------+-----------------+----------------------+
1 row in set, 2 warnings (0,00 sec)
mysql>
I have mysql table with multiple columns, a column has string data type namely code and the values of the column might be digits only or mixed of string and digits e.g: one row value has 57677 and other rows column value like 2cskjf893 or 78732sdjfh.
mysql> select * from testuser;
+------+--------------+
| id | code |
+------+--------------+
| 1 | 232 |
| 2 | adfksa121dfk |
| 3 | 12sdf |
| 4 | dasd231 |
| 5 | 897 |
+------+--------------+
5 rows in set (0.00 sec)
mysql> show create table testuser;
+----------+------------------------------------------------------------------ ---------------------------------------------------------------+
| Table | Create Table |
+----------+------------------------------------------------------------------ ---------------------------------------------------------------+
| testuser | CREATE TABLE `testuser` (
`id` int(11) DEFAULT NULL,
`code` varchar(20) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
+----------+------------------------------------------------------------------ ---------------------------------------------------------------+
1 row in set (0.00 sec)
mysql>
I would like to filter out only rows which has numeric value in code column or filter rows which has string values in code column.
try this
Mysql Query to fetch only integer values from a particular column
SELECT column FROM TABLE where column NOT REGEXP '^[0-9]+$' ;
Mysql Query to fetch only string values from a particular column
SELECT column FROM TABLE where column REGEXP '^[0-9]+$' ;
To get digits
SELECT id,code
FROM table_name
WHERE code REGEXP '^[0-9]+$';
To get string values
SELECT *
FROM table_name
WHERE code REGEXP '^[a-zA-Z]+$'
Use regular expressions
SELECT *
FROM myTable
WHERE code REGEXP '^[0-9]+$';
This will filter out column values with all digits.
You can use regular expressions in mysql too.
SELECT * FROM `table` WHERE `code` REGEXP '[0-9]';
Results will be the rows, where any number in the code column.
Try this,
select id,code from table_name where code NOT REGEXP '^[0-9]+$'
Or,
select id,code from table_name where code like '%[^0-9]%'
I have a list of data from SQL field. I want to sort by the field ASC but when i do it comes up in the wrong order. I know why it does it, but wondered if there was a solution around this problem. Ive heard of natsort php function, but not investigated it. Is there an easy way?
Academy
Under 10′s Blue
Under 10′s Green
Under 11′s Red
Under 11′s White
Under 13′s Blue
Under 13′s Red
Under 13′s White
Under 14′s Blue
Under 15′s Blue
Under 15′s Red
Under 15′s White
Under 16′s Red
Under 18′s Blue
Under 18′s Red
Under 7′s
Under 8′s Red
Under 9′s Red
There is a very simple approach to sort this list. For all values of Academy starting with Under you perform the sort algorithm based on the following ORDER BY clause:
ORDER BY
REPLACE(Academy,'Under ','') + 0,Academy
The first sort column is based on removing the string 'Under ' and then adding 0. This will force an ordering of the resulting integer.
Here is an example of computing the numeric value by removing 'Under ' first:
mysql> select REPLACE('Under 15\'s Red','Under ','') + 0;
+--------------------------------------------+
| REPLACE('Under 15\'s Red','Under ','') + 0 |
+--------------------------------------------+
| 15 |
+--------------------------------------------+
1 row in set (0.00 sec)
The second sort column will order by the string value of Academy. All 'Under 15's' are grouped together and alphanumerically sorted.
Here is your sample data from the question loaded into a table and sorted:
mysql> use test
Database changed
mysql> drop table if exists under99color;
Query OK, 0 rows affected (0.01 sec)
mysql> create table under99color
-> (academy varchar(30),
-> id int not null auto_increment,
-> primary key (id),
-> index academy (academy)) engine=MyISAM;
Query OK, 0 rows affected (0.04 sec)
mysql> show create table under99color\G
*************************** 1. row ***************************
Table: under99color
Create Table: CREATE TABLE `under99color` (
`academy` varchar(30) DEFAULT NULL,
`id` int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`),
KEY `academy` (`academy`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
1 row in set (0.00 sec)
mysql> insert into under99color (academy) values
-> ('Under 10\'s Blue'),('Under 10\'s Green'),('Under 11\'s Red'),
-> ('Under 11\'s White'),('Under 13\'s Blue'),('Under 13\'s Red'),
-> ('Under 13\'s White'),('Under 14\'s Blue'),('Under 15\'s Blue'),
-> ('Under 15\'s Red'),('Under 15\'s White'),('Under 16\'s Red'),
-> ('Under 18\'s Blue'),('Under 18\'s Red'),('Under 7\'s'),
-> ('Under 8\'s Red'),('Under 9\`s Red');
Query OK, 17 rows affected (0.00 sec)
Records: 17 Duplicates: 0 Warnings: 0
mysql> select academy from under99color
-> ORDER BY REPLACE(Academy,'Under ','') + 0,Academy;
+------------------+
| academy |
+------------------+
| Under 7's |
| Under 8's Red |
| Under 9`s Red |
| Under 10's Blue |
| Under 10's Green |
| Under 11's Red |
| Under 11's White |
| Under 13's Blue |
| Under 13's Red |
| Under 13's White |
| Under 14's Blue |
| Under 15's Blue |
| Under 15's Red |
| Under 15's White |
| Under 16's Red |
| Under 18's Blue |
| Under 18's Red |
+------------------+
17 rows in set (0.00 sec)
mysql>
Give it a Try !!!
You can add a field to the selection query that uses a CAST to bring it into a numeric. First you'll have to come up with a substring method that will select the number from the string in the first place (perhaps use a Field function on the space and the '). Once you've got it isolated as an integer, sorting at that point should be trivial.
Possible example (pseudo-code - may not work "out of the box"):
SELECT TeamType, CAST(SUBSTRING(TeamType, FIELD(' ', TeamType), FIELD('\'', TeamType) - Field(' ', TeamType)), UNSIGNED) As TeamAge
FROM Teams
ORDER BY TeamAge, TeamType
Your field is string! Thus it's sorting the string values.
It has sorted it alphabetically; you need to parse the output field and sort them based on the number later.
Well, you could store the number and the colour separately (with the option of a blank colour) and then order by number followed by colour. E.g:
SELECT CONCAT('Under ',ageIndex,'\'s ',colour) AS Team FROM Academy
ORDER BY ageIndex, colour
The possible advantage of this, depending on your requirements, is that you can then also run queries on ages and colours separately.