I have a table in mysql with many columns and I want to see maximum length of values. My purpose is that I do know that some of data is truncated when insert and I want to increase varchar length. But do not know, what columns. (Explanation little messy, but probably sql will make sense)
I tried:
select COLUMN_NAME, CHARACTER_MAXIMUM_LENGTH, DATA_TYPE, (SELECT LENGTH(COLUMN_NAME) as maxlen FROM my_database.my_table ORDER BY maxlen DESC LIMIT 1)
from information_schema.columns
where table_schema = 'my_database' AND
table_name = 'my_table'AND DATA_TYPE = 'varchar'
It works, but return the length of the column, but not data inside it. (I.e. id column is called id, I got 2).
If I use JOIN (ON TRUE condition), I got error that COLUMN_NAME is undefined.
Stored procedures does not allow for data return, and function does not allow dynamic sql inside it.
How to tell MySQL (in case of my query) to consider COLUMN_NAME not as a string, but as column name? If this is not, possible in select, how to get columns with maximum data inside them?
Desired result looks like:
column_1 | 25 | varchar | 20
column_2 | 25 | varchar | 7
I am interested only in varchar, as int does not make sense to adjust (and no need to). Columns has different length (varchar(20),varchar(25), etc.).
Update 1: This cannot be done also via loop (statements cannot be executed inside cursor).
I use something like this type of code to generate my view automticaly using table schema. Use can modify according to your need.
$sql = "show tables from DBName where Tables_in_yourtbalename = 'yourtbalename' ";
$result = executeQuery($sql, $conn);
$num = $result->num_rows;
if ($num) {
$sql = "show columns from yourtbalename where Extra != 'auto_increment'";
$result = executeQuery($sql, $conn);
$num2 = $result->num_rows;
while ($r = $result->fetch_assoc()) {
if ($r['Key'] == 'MUL' && ( preg_match("/^int/", $r['Type']) || preg_match("/^smallint/", $r['Type']) || preg_match("/^tinyint/", $r['Type']) || preg_match("/^bigint/", $r['Type']))) {
} else if ($r['Field'] == 'status') {
}
}
Where $r['Field'] is field name and $r['Type'] provides its type. For determining maxlength use
$maxlength="' . substr(str_replace(")", "", $r['Type']), 8, (strlen(str_replace(")", "", $r['Type']))));
Related
I want my the id field in my table to be a bit more " random" then consecutive numbers.
Is there a way to insert something into the id field, like a +9, which will tell the db to take the current auto_increment value and add 9 to it?
Though this is generally used to solve replication issues, you can set an increment value for auto_increment:
auto_increment_increment
Since that is both a session and a global setting, you could simply set the session variable just prior to the insert.
Besides that, you can manually do it by getting the current value with MAX() then add any number you want and insert that value. MySQL will let you know if you try to insert a duplicate value.
You have a design flaw. Leave the auto increment alone and shuffle your query result (when you fetch your data)
As far as i know, it's not possible to 'shuffle' your current IDs. If you wanted though, you could pursue non-linear IDs in the future.
The following is written in PDO, there are mysqli equivalents.
This is just an arbitrary INSERT statement
$name = "Jack";
$conn = new PDO("mysql:host=$dbhost;dbname=$dbname",$dbuser,$dbpass);
$sql = "INSERT INTO tableName (name) VALUES(:name)";
$q = $conn->prepare($sql);
$q->execute(':name' => $name);
Next, we use lastInsertId() to return the ID of the last inserted row, then we concatenate the result to rand()
$lastID = $conn->lastInsertId();
$randomizer = $lastID.rand();
Finally, we use our 'shuffled' ID and UPDATE the previously inserted record.
$sql = "UPDATE tableName SET ID = :randomizer WHERE ID=:lastID ";
$q = $conn->prepare($sql);
$q->execute(array(':lastID' => $lastID , ':randomizer' => $randomizer));
An idea.. (Not tested)
CREATE TRIGGER 'updateMyAutoIncrement'
BEFORE INSERT
ON 'DatabaseName'.'TableName'
FOR EACH ROW
BEGIN
DECLARE aTmpValueHolder INT DEFAULT 0;
SELECT AUTO_INCREMENT INTO aTmpValueHolder
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = 'DatabaseName'
AND TABLE_NAME = 'TableName';
SET NEW.idColumnName =aTmpValueHolder + 9;
END;
Edit : If the above trigger doesn't work try to update AUTO_INCREMENT value directly into the system's schema. But as noted by Eric, your design seems to be flawed. I don't see the point of having an auto-increment here.
Edit 2 : For a more 'random' and less linear number.
SET NEW.idColumnName =aTmpValueHolder + RAND(10);
Edit 3 : As pointed out by Jack Williams, Rand() produces a float value between 0 and 1.
So instead, to produce an integer, we need to use a floor function to transform the 'random' float into an integer.
SET NEW.idColumnName =aTmpValueHolder + FLOOR(a + RAND() * (b - a));
where a and b are the range of the random number.
Currently I'm manually creating a string where I concatenate all the values in each row in my table. I'm hashing this string for each row to get a hash value for the current values (/status) of the row, which I'm later is using to determine if the row has changed.
Instead of doing this manually, is there an build-in way i mySQL to get a unique hash value for each row?
you could do something like
SELECT MD5(concat(field1, field2, field3, ...)) AS rowhash
but you can't get away from listing which fields you want, as concat(*) is not an option (syntax error).
It's better to use concat_ws(). e.g. two adjacent column: 12,3 => 1,23 .
Sorry, this still has some problems. Think about the null value, empty string, string can contain ',', etc...
A program is required to generate the hash statement, which should replace null to specific value (for null-able columns), and also use the seldom used char/byte as separator.
There are problems with CONCAT, e.g. CONCAT('ab', 'c') vs CONCAT('a', 'bc'). Two different rows, but result is the same. You could use CONCAT_WS(';', 'ab', 'c') to get ab;c but in case of CONCAT_WS(';', ';', '') vs CONCAT_WS(';', '', ';') you still get the same result.
Also CONCAT(NULL, 'c') returns NULL.
I think the best way is to use QUOTE:
SELECT MD5(CONCAT(QUOTE(c1), QUOTE(c2), QUOTE(c3))) AS row_hash FROM t1;
Result of: select (concat(quote('a'), quote('bc'), quote('NULL'), quote(NULL), quote('\''), quote('')));
is: 'a''bc''NULL'NULL'\''''
Also, don't use GROUP_CONCAT() to get hash of table, it has limit: https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_group_concat_max_len
Instead, CHECKSUM TABLE might be better, but you can't skip columns with CHECKSUM TABLE https://dev.mysql.com/doc/refman/5.7/en/checksum-table.html
Well I made a little script that could do excactly what you want, and maybe what others want... so here it goes...for PHP that is...
first you have to make a list of columns of the table, then you make a "case when" statement for each column based on their type and put that in the concat_ws statement and then you hash it with sha1...i've used this method on very large tables (600000+ records) and the speed is quite good when selecting all records. also I think that it is faster to concat the required data in a concat_ws and explode it in php or whatever you are using, but that is just a hunch...
<?
$query= mysql_query("SHOW COLUMNS FROM $table", $linklive);
while ($col = mysql_fetch_assoc($query)) {
$columns[] = mysql_real_escape_string($col['Field']);
if ($col['Key'] == 'PRI') {
$key = mysql_real_escape_string($col['Field']);
}
$columnsinfo[$col['Field']] = $col;
}
$dates = array("date","datetime","time");
$int = array("int","decimal");
$implcols = array();
foreach($columns as $col){
if(in_array($columnsinfo[$col]['Type'], $dates)){
$implcols[] = "(CASE WHEN (UNIX_TIMESTAMP(`$col`)=0 || `$col` IS NULL) THEN '[$col EMPTY]' ELSE `$col` END)";
}else{
list($type, $rest) = explode("(",$columnsinfo[$col]['Type']);
if(in_array($columnsinfo[$col]['Type'], $dates)){
$implcols[] = "(CASE WHEN ( `$col`=0 || `$col` IS NULL ) THEN '[$col EMPTY]' ELSE `$col` END)";
}else{
$implcols[] = "(CASE WHEN ( `$col`='' || `$col` IS NULL ) THEN '[$col EMPTY]' ELSE `$col` END)";
}
}
}
$keyslive = array();
//echo "SELECT $key SHA1(CONCAT_WS('',".implode(",", $columns).")) as compare FROM $table"; exit;
$q = "SELECT $key as `key`, SHA1(CONCAT_WS('',".implode(", ",$implcols).")) as compare FROM $table";
?>
I have a table with millions of rows and a single column of text that is exactly 11,159 characters long. It looks like this:
1202012101...(to 11,159 characters)
1202020120...
0121210212...
...
(to millions of rows)
I realize that I can use
SELECT SUBSTR(column,2,4) FROM table;
...if I wanted to pull out characters 2, 3, 4, and 5:
1202012101...
1202020120...
0121210212...
^^^^
But I need to extract noncontiguous characters, e.g. characters 1,5,7:
1202012101...
1202020120...
0121210212...
^ ^ ^
I realize this can be done with a query like:
SELECT CONCAT(SUBSTR(colm,1,1),SUBSTR(colm,5,1),SUBSTR(colm,7,1)) FROM table;
But this query gets very unwieldy to build for thousands of characters that I need to select. So for the first part of the question - how do I build a query that does something like this:
SELECT CHARACTERS(string,1,5,7) FROM table;
Furthermore, the indices of the characters I want to select are from a different table that looks something like this:
char_index keep_or_discard
1 keep
2 discard
3 discard
4 discard
5 keep
7 discard
8 keep
9 discard
10 discard
So for the second part of the question, how could I build a query to select specific characters from the first table based on whether keep_or_discard="keep" for that character's index in the second table?
this function does what you want:
CREATE DEFINER = `root`#`localhost` FUNCTION `test`.`getsubset`(selection mediumtext, longstring mediumtext)
RETURNS varchar(200)
LANGUAGE SQL
NOT DETERMINISTIC
CONTAINS SQL
SQL SECURITY DEFINER
COMMENT 'This function returns a subset of characters.'
BEGIN
SET #res:='';
SET #selection:=selection;
WHILE #selection<>'' DO
set #pos:=CONVERT(#selection, signed);
set #res := concat_ws('',#res,SUBSTRING(longstring,#pos,1));
IF LOCATE(',',#selection)=0 THEN
SET #selection:='';
END IF;
set #selection:=SUBSTRING(#selection,LOCATE(',',#selection)+1);
END WHILE;
RETURN #res;
END
Note: the CONVERT('1,2,3,4',signed) will yield 1, but it will give a warning.
I have it defined to be available in the database test.
The function takes two parameters; a string(!) with a list of positions, and a long string from where you want the characters taken.
An example of using this:
mysql> select * from keepdiscard;
+---------+------------+
| charind | keepordisc |
+---------+------------+
| 1 | keep |
| 2 | discard |
| 3 | keep |
| 4 | discard |
| 5 | keep |
| 6 | keep |
+---------+------------+
6 rows in set (0.00 sec)
mysql> select * from test;
+-------------------+
| longstring |
+-------------------+
| abcdefghijklmnopq |
| 123456789 |
+-------------------+
2 rows in set (0.00 sec)
mysql> select getsubset(group_concat(charind ORDER BY charind),longstring) as result from keepdiscard, test where keepordisc='keep' group by longstring;
+--------+
| result |
+--------+
| 1356 |
| acef |
+--------+
2 rows in set, 6 warnings (0.00 sec)
The warnings stem from the fast conversion to integer that is done in the function. (See comment above)
How about dynamic sql? (You will need to build the select part of the query)
CREATE PROCEDURE example_procedure()
BEGIN
--
--build the concat values here
--
SET #ids := '';
SET #S = 'SELECT #ids := built_concat_of_values FROM table';
PREPARE n_StrSQL FROM #S;
EXECUTE n_StrSQL;
DEALLOCATE PREPARE n_StrSQL;
END
You can write a php script to do this for you:
<?php
//mysql connect
$conn = mysql_connect('localhost', 'mysql_user', 'mysql_password');
if (!$conn) {
echo 'Unable to connect to DB: ' . mysql_error();
exit;
}
//database connect
$db = mysql_select_db('mydb');
if (!$db) {
echo 'Unable to select mydb: ' . mysql_error();
exit;
}
//get the keep numbers you’re going to use.
//and change the number into string so, for example, instead of 5 you get 'SUBSTR(colm,5,1)'
$result = mysql_query("SELECT number FROM number_table WHERE keep_or_discard='keep'");
$numbers = array();
while ($row = mysql_fetch_assoc($result)) {
$row = 'SUBSTR(colm,' . $row . ',1)';
$numbers = $row;
}
//implode the array so you get one long string with all the substrings
//eg. 'SUBSTR(colm,1,1),SUBSTR(colm,5,1),SUBSTR(colm,12,1)'
$numbers = implode(",", $numbers);
//pull the numbers you need and save them to an array.
$result = mysql_query("SELECT " . $numbers . " FROM table");
$concat = array();
while ($row = mysql_fetch_assoc($result)) {
$concat= $row;
}
And there you have an array with the correct numbers.
I'm sorry if you can't/don't want to use PHP for this, I just don't really know how to do this without PHP, Perl, Python or some other similar language. Hopefully this solution will help somehow...
The source of your difficulty is that your schema does not represent the true relationships between the data elements. If you wanted to achieve this with "pure" SQL, you would need a schema more like:
table
ID Index Char
1 0 1
1 1 2
1 2 0
charsToKeep
ID Index Keep
1 0 false
1 1 true
1 2 true
Then, you could perform a query like:
SELECT Char FROM table t JOIN charsToKeep c ON t.ID = c.ID WHERE c.Keep = true
However, you probably have good reasons for structuring your data the way you have (my schema requires much more storage space per character and the processing time is also probably much longer from what I am about to suggest).
Since SQL does not have the tools to understand the schema you have embedded into your table, you will need to add them with a user-defined function. Kevin's example of dynamic SQL may also work, but in my experience this is not as fast as a user-defined function.
I have done this in MS SQL many times, but never in MySql. You basically need a function, written in C or C++, that takes a comma-delimited list of the indexes you want to extract, and the string from which you want to extract them from. Then, the function will return a comma-delimited list of those extracted values. See these links for a good starting point:
http://dev.mysql.com/doc/refman/5.1/en/adding-functions.html
http://dev.mysql.com/doc/refman/5.1/en/adding-udf.html
To build the concatenated list of indexes you want to extract from the char_index table, try the group_concat function:
http://dev.mysql.com/doc/refman/5.0/en/group-by-functions.html#function_group-concat
Hope this helps!
I have a table with 27 varchar fields. I want to make all fields lowercase, but i want to do it in one short mysql call.
This does a single field:
UPDATE table
SET field = LOWER(field)
How do I do the equivalent of this (which doesn't work):
UPDATE table
SET * = LOWER(*)
You can't do it with your creative attempt SET * = LOWER(*) etc.
You can however do it like this:
UPDATE table SET
column1 = LOWER(column1),
column2 = LOWER(column2),
-- etc, listing all text type columns
columnN = LOWER(columnN);
The reason there's no "shortcut" is probably because this pattern is so infrequently needed.
The consensus is that this cannot be done in a single mysql query.
Here is a super quick PHP script that does this for N fields (thanks for the idea #alex):
$sql = "SHOW COLUMNS
FROM table";
$results = mysqli_query($dbcon,$sql);
while($column = mysqli_fetch_assoc($results))
{
$column = $column["Field"];
$sql = "UPDATE table
SET $column = LOWER($column)";
$success = mysqli_query($dbcon,$sql);
}
Simple question: How can I return the field type of a MySQL table. I know about describe or show column but I just want to return that single parameter. e.g.:
SELECT fieldtype(mycol) FROM mytable
# should return INT or integer for example
You can use
SHOW FIELDS
FROM tableName where Field ='nameOfField'
This will return you result in format of
Field Type Null Key Default Extra
You can get this from the information_schema database:
select data_type
from information_schema.columns
where table_schema = 'myschema'
and table_name = 'mytable'
and column_name = 'mycol'
I know I'm a little late to the party. You may want to use COLUMN_TYPE rather than DATA_TYPE. It gives more info (such as precision) about the type.
SELECT COLUMN_TYPE
FROM information_schema.COLUMNS
WHERE TABLE_NAME = 'parts'
AND COLUMN_NAME = 'price'
...yields...
decimal(11,2)
The following syntax also works, using describe, enter the columnName after the tableName:
describe tbleName columnName;
ResultSet rs = Sstatement.executeQuery("SELECT * FROM Table Name");
ResultSetMetaData rsMetaData = rs.getMetaData();
int numberOfColumns = rsMetaData.getColumnCount();
System.out.println("resultSet MetaData column Count=" + numberOfColumns);
for (int i = 1; i <= numberOfColumns; i++) {
System.out.println("column number " + i);
System.out.println(rsMetaData.getColumnTypeName(i));
}