MySQL and GROUP_CONCAT() maximum length - mysql

I'm using GROUP_CONCAT() in a MySQL query to convert multiple rows into a single string.
However, the maximum length of the result of this function is 1024 characters.
I'm very well aware that I can change the param group_concat_max_len to increase this limit:
SET SESSION group_concat_max_len = 1000000;
However, on the server I'm using, I can't change any param. Not by using the preceding query and not by editing any configuration file.
So my question is:
Is there any other way to get the output of a multiple row query into a single string?

SET SESSION group_concat_max_len = 1000000;
is a temporary, session-scope, setting. It only applies to the current session You should use it like this.
SET SESSION group_concat_max_len = 1000000;
select group_concat(column) from table group by column
You can do this even in sharing hosting, but when you use an other session, you need to repeat the SET SESSION command.

The correct parameter to set the maximum length is:
SET ##group_concat_max_len = value_numeric;
value_numeric must be > 1024; by default the group_concat_max_len value is 1024.

Include this setting in xampp my.ini configuration file:
[mysqld]
group_concat_max_len = 1000000
Then restart xampp mysql

You can try this
SET GLOBAL group_concat_max_len = 1000000;

The correct syntax is mysql> SET ##global.group_concat_max_len = integer;
If you do not have the privileges to do this on the server where your database resides then use a query like:
mySQL="SET ##session.group_concat_max_len = 10000;"or a different value. Next line:
SET objRS = objConn.Execute(mySQL) your variables may be different.
then
mySQL="SELECT GROUP_CONCAT(......);" etc
I use the last version since I do not have the privileges to change the default value of 1024 globally (using cPanel).
Hope this helps.

The short answer: the setting needs to be setup when the connection to the MySQL server is established. For example, if using MYSQLi / PHP, it will look something like this:
$ myConn = mysqli_init();
$ myConn->options(MYSQLI_INIT_COMMAND, 'SET SESSION group_concat_max_len = 1000000');
Therefore, if you are using a home-brewed framework, well, you need to look for the place in the code when the connection is establish and provide a sensible value.
I am still using Codeigniter 3 on 2020, so in this framework, the code to add is in the application/system/database/drivers/mysqli/mysqli_driver.php, the function is named db_connect();
public function db_connect($persistent = FALSE)
{
// Do we have a socket path?
if ($this->hostname[0] === '/')
{
$hostname = NULL;
$port = NULL;
$socket = $this->hostname;
}
else
{
$hostname = ($persistent === TRUE)
? 'p:'.$this->hostname : $this->hostname;
$port = empty($this->port) ? NULL : $this->port;
$socket = NULL;
}
$client_flags = ($this->compress === TRUE) ? MYSQLI_CLIENT_COMPRESS : 0;
$this->_mysqli = mysqli_init();
$this->_mysqli->options(MYSQLI_OPT_CONNECT_TIMEOUT, 10);
$this->_mysqli->options(MYSQLI_INIT_COMMAND, 'SET SESSION group_concat_max_len = 1000000');
...
}

CREATE TABLE some_table (
field1 int(11) NOT NULL AUTO_INCREMENT,
field2 varchar(10) NOT NULL,
field3 varchar(10) NOT NULL,
PRIMARY KEY (`field1`)
);
INSERT INTO `some_table` (field1, field2, field3) VALUES
(1, 'text one', 'foo'),
(2, 'text two', 'bar'),
(3, 'text three', 'data'),
(4, 'text four', 'magic');
This query is a bit strange but it does not need another query to initialize the variable; and it can be embedded in a more complex query.
It returns all the 'field2's separated by a semicolon.
SELECT result
FROM (SELECT #result := '',
(SELECT result
FROM (SELECT #result := CONCAT_WS(';', #result, field2) AS result,
LENGTH(#result) AS blength
FROM some_table
ORDER BY blength DESC
LIMIT 1) AS sub1) AS result) AS sub2;

Related

Mysql function not returning the expected result

As I have mentioned in my question title below Mysql function returns null always :
CREATE DEFINER=`root`#`localhost` FUNCTION `nextCode`(tbl_name VARCHAR(30), prv_code VARCHAR(30)) RETURNS varchar(30) CHARSET utf8
READS SQL DATA
BEGIN
DECLARE nxtCode VARCHAR(30);
SELECT ds.prefix, ds.suffix, ds.is_used, ds.next_number, CHAR_LENGTH(ds.pattern)
INTO #prefix, #suffix, #isUsed, #nxtNum, #pLength
FROM ths_inventory.doc_sequnce ds WHERE ds.`table_name` = tbl_name;
SET nxtCode = CONCAT(#prefix, LPAD((CASE WHEN #isUsed
THEN
(ExtractNumber(prv_code) + 1)
ELSE
(#nxtNum)
END
), #pLength,'0'), #suffix);
RETURN nxtCode;
END
But once I change the below line :
CONCAT(#prefix, LPAD((CASE WHEN #isUsed
THEN
(ExtractNumber(prv_code) + 1)
ELSE
(#nxtNum)
END
), #pLength,'0'), #suffix)
To some static values like below :
CONCAT('PR', LPAD((CASE WHEN true
THEN
(ExtractNumber(prv_code) + 1)
ELSE
(5)
END
), 6,'0'), '')
function start returning values accordingly.
Here is how I call my function :
nextCode('item','PR000002');
UPDATE:
I defined this function to get the next possible code for Item table :
According to my requirement the next possible code should be PR000000005.
But instead of getting it, I always get empty result .
SELECT nextCode('item',(SELECT `code` FROM item ORDER BY id DESC LIMIT 1)) AS next_code;
Any help would be appreciable.
Run a query that uses the function, and then...
SELECT #prefix, #suffix, #isUsed, #nxtNum, #pLength;
...to inspect the values. The # prefix means these are user-defined variables, so they have session scope, not program scope, and will still hold their values after the funcfion executes.
This should help pinpoint your problem.
But, you have two other problems you will need to solve after that.
SELECT ... INTO does not set the target variables when no row matches the query, so once you fix your issue, you will get very wrong results if you pass in arguments that don't match anything.
To resolve this, the function needs to set all these variables to null before the SELECT ... INTO query.
SET #prefix = NULL, #suffix = NULL, #isUsed = NULL, #nxtNum = NULL, #pLength = NULL;
See https://dba.stackexchange.com/a/35207/11651.
Also, your function does not handle concurrency, so two threads trying to find the "next" value for the same table, concurrently, will produce the same answer, so you will need to insure that your code handles this correctly with unique constraints and transactions or other appropriate locks.

"Row cannot be located for updating" error [another]

I'm working with Delphi (XE3) and need to connect to a MySQL database. I'm having a strange problem that seems quite common but I still haven't entirely solved this problem.
traditional solutions include:
setting "Update Criteria" to adCriteriaKey.
ensure your table has a primary key (and tell the ADO Table about it)
problem 1:
start the application, execute the code: if the new value happens to match what's already in the database, I get the error at position "B".
problem 2:
start the application, execute the code: if the new value happens to be different from what's already in the database, it will execute successfully once and thereafter, give an error at position "A".
there shouldn't be any problem locating the record at anytime since the record primary key has not been changed.
object conMain: TADOConnection
Connected = True
ConnectionString =
'Provider=MSDASQL.1;Password=p;Persist Security Info=True;U' +
'ser ID=M;Extended Properties="Driver={MySQL ODBC 5.3 ANSI Dri' +
'ver};SERVER=yukon;DATABASE=db;UID=M;Pwd=p;PORT=3306;' +
'"'
LoginPrompt = False
Mode = cmShareDenyNone
Left = 48
Top = 24
end
object ADOTable1: TADOTable
Connection = conMain
IndexFieldNames = 'FacilityID'
TableName = 'facility'
Left = 152
Top = 24
end
procedure TForm1.Button1Click(Sender: TObject);
begin
conMain.Open;
ADOTable1.Open;
ADOTable1.Properties.Item['Update Criteria'].Value:=adCriteriaKey;
// **A**
if ADOTable1.Locate('facilityid', '{C0FADCC8-15C9-48C8-8003-3BBD4AB74586}', []) then
begin
ADOTable1.Edit;
ADOTable1.FieldByName('facilityaddress1').AsString:='mickey street';
ADOTable1.Properties.Item['Update Criteria'].Value:=adCriteriaKey;
// **B**
ADOTable1.Post;
end
else
showmessage('not found!');
ADOTable1.Close;
conMain.Close;
end;
it's as though the Post method or the connection left the database in some intermediate state...
here's what the database log says when I demonstrate problem 1.
M#D3400 on db
SET NAMES latin1
SET character_set_results = NULL
SET SQL_AUTO_IS_NULL = 0
select database()
select database()
SHOW GLOBAL STATUS
SELECT ##tx_isolation
set ##sql_select_limit=DEFAULT
select * from facility
SHOW KEYS FROM `facility`
UPDATE `db`.`facility` SET `FacilityAddress1`=? WHERE `facilityid`=?
UPDATE `db`.`facility` SET `FacilityAddress1`='mickey street22' WHERE `facilityid`='{C0FADCC8-15C9-48C8-8003-3BBD4AB74586}'
here's what the database log says when I demonstrate problem 2.
SHOW GLOBAL STATUS
M#D3400 on db
SET NAMES latin1
SET character_set_results = NULL
SET SQL_AUTO_IS_NULL = 0
select database()
select database()
SELECT ##tx_isolation
set ##sql_select_limit=DEFAULT
select * from facility
SHOW KEYS FROM `facility`
UPDATE `db`.`facility` SET `FacilityAddress1`=? WHERE `facilityid`=?
// SUCCESSFUL
UPDATE `db`.`facility` SET `FacilityAddress1`='mickey street22' WHERE `facilityid`='{C0FADCC8-15C9-48C8-8003-3BBD4AB74586}'
SHOW GLOBAL STATUS
SHOW GLOBAL STATUS
SHOW GLOBAL STATUS
db
select * from facility
UPDATE `db`.`facility` SET `FacilityAddress1`=? WHERE `facilityid`=?
// ERROR!
UPDATE `db`.`facility` SET `FacilityAddress1`='mickey street22' WHERE `facilityid`='{C0FADCC8-15C9-48C8-8003-3BBD4AB74586}'
SHOW GLOBAL STATUS
setting the "Update Criteria" in various places was of no help.
reduced the table down to just two fields: facilityid varchar(38), facilityaddress1 varchar(50). same result...
form http://www.connectionstrings.com/mysql-connector-odbc-5-2/, I found:
Provider=MSDASQL;Driver={MySQL ODBC 5.3 UNICODE Driver};Persist Security Info=True;Server=yukon;Database=ocean;User=M;Password=p;Option=2;
"Option=2" is suggested for VB
http://dev.mysql.com/doc/connector-odbc/en/connector-odbc-configuration-connection-parameters.html#codbc-dsn-option-combos
works!
Thank you all for your contributions.

Perl DBI / MySQL - can't for specific float values

I've got a problem with my perl and mysql code.
I have a database with a table "STORAGEDATA". In this table I have the column "CLOSETIME" with the datatype "double", which stores an epoch timestamp as a float value (the floating part represents milliseconds).
I now have the problem, that I can't find some of those values with a SELECT-Statement.
MySQL-Example:
SELECT * FROM STORAGEDATA WHERE CLOSETIME = 1360021730.666;
-- This will find the data I am looking for
SELECT * FROM STORAGEDATA WHERE CLOSETIME = '1360021730.666';
-- This works, too
SELECT * FROM STORAGEDATA WHERE CLOSETIME = 1360209405.574;
-- This will find my data, too
SELECT * FROM STORAGEDATA WHERE CLOSETIME = '1360209405.574';
-- But this does not find anything.
The searched values definitly exists in the table and are exactly saved as the values shown above (no trailing zeros or something).
Now I've got the problem that I don't know how to fetch the data using perl dbi.
Perl-Example:
my $sql = 'SELECT * FROM STORAGEDATA WHERE CLOSETIME = ?';
my #args = (1360209405.574);
# $dbh is a database-handler, providing the DBI-functionality
my $sth = $dbh->prepare($sql);
my #data;
if ($sth) {
$sth->execute(#args);
while ( my $dataset = $sth->fetchrow_hashref() ) { push #data, $dataset; }
}
This won't find anything, it seems like DBI inserts the arguments as strings. When I put my quittime directly into the code, it works:
my $sql = 'SELECT * FROM STORAGEDATA WHERE CLOSETIME = 1360209405.574';
my #args = ();
# $dbh is a database-handler, providing the DBI-functionality
my $sth = $dbh->prepare($sql);
my #data;
if ($sth) {
$sth->execute(#args);
while ( my $dataset = $sth->fetchrow_hashref() ) { push #data, $dataset; }
}
This will work, but that's not what I want to do. When I use DBIx, the problem is still the same. I could use "LIKE" instead of "=" to compare, but this would extremely slow down the query.
I tried to change the datatype for CLOSETIME to "Decimal(13,3)", but this didn't change anything.
I run MySQL in version 5.1.73.
Do you have any suggestions how I can solve this problem?
Solution
Either upgrade MySQL, or explicitly specify your predicate's type in the SQL itself:
SELECT * FROM STORAGEDATA WHERE CLOSETIME = CAST(? AS DECIMAL(13,3))
^^^^^^^^^^^^^^^^^^^^^^^^
Rationale
To my thinking, you show us the heart of the matter right away:
SELECT * FROM STORAGEDATA WHERE CLOSETIME = 1360209405.574;
-- This will find my data, too
SELECT * FROM STORAGEDATA WHERE CLOSETIME = '1360209405.574';
-- But this does not find anything.
Huh? That's not what I'd expect. This appears to be a bug in MySQL 5.1, remedied in at least 5.5 (possibly earlier). Smallest demonstration I could think of:
SELECT VERSION(), '1360209405.574' = 1360209405.574 AS "str-num comparison";
-- Under 5.5, TRUE: http://sqlfiddle.com/#!2/e705e6/1
SELECT VERSION(), '1360209405.574' = 1360209405.574 AS "str-num comparison";
-- Under 5.1, FALSE: http://sqlfiddle.com/#!8/e705e/1
Under 5.1, explicitly CASTing to DECIMAL worked for me.
try turning on bind type guessing
my $dbh= DBI->connect('DBI:mysql:test', 'username', 'pass',
{ mysql_bind_type_guessing => 1})
- OR after handle creation
$dbh->{mysql_bind_type_guessing} = 1;
You can use bind_param: https://metacpan.org/pod/DBI#bind_param
$sth->bind_param(1, $args[0], SQL_DOUBLE);
$sth->execute;

MySql auto_increment plus a certain number

I want my the id field in my table to be a bit more " random" then consecutive numbers.
Is there a way to insert something into the id field, like a +9, which will tell the db to take the current auto_increment value and add 9 to it?
Though this is generally used to solve replication issues, you can set an increment value for auto_increment:
auto_increment_increment
Since that is both a session and a global setting, you could simply set the session variable just prior to the insert.
Besides that, you can manually do it by getting the current value with MAX() then add any number you want and insert that value. MySQL will let you know if you try to insert a duplicate value.
You have a design flaw. Leave the auto increment alone and shuffle your query result (when you fetch your data)
As far as i know, it's not possible to 'shuffle' your current IDs. If you wanted though, you could pursue non-linear IDs in the future.
The following is written in PDO, there are mysqli equivalents.
This is just an arbitrary INSERT statement
$name = "Jack";
$conn = new PDO("mysql:host=$dbhost;dbname=$dbname",$dbuser,$dbpass);
$sql = "INSERT INTO tableName (name) VALUES(:name)";
$q = $conn->prepare($sql);
$q->execute(':name' => $name);
Next, we use lastInsertId() to return the ID of the last inserted row, then we concatenate the result to rand()
$lastID = $conn->lastInsertId();
$randomizer = $lastID.rand();
Finally, we use our 'shuffled' ID and UPDATE the previously inserted record.
$sql = "UPDATE tableName SET ID = :randomizer WHERE ID=:lastID ";
$q = $conn->prepare($sql);
$q->execute(array(':lastID' => $lastID , ':randomizer' => $randomizer));
An idea.. (Not tested)
CREATE TRIGGER 'updateMyAutoIncrement'
BEFORE INSERT
ON 'DatabaseName'.'TableName'
FOR EACH ROW
BEGIN
DECLARE aTmpValueHolder INT DEFAULT 0;
SELECT AUTO_INCREMENT INTO aTmpValueHolder
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = 'DatabaseName'
AND TABLE_NAME = 'TableName';
SET NEW.idColumnName =aTmpValueHolder + 9;
END;
Edit : If the above trigger doesn't work try to update AUTO_INCREMENT value directly into the system's schema. But as noted by Eric, your design seems to be flawed. I don't see the point of having an auto-increment here.
Edit 2 : For a more 'random' and less linear number.
SET NEW.idColumnName =aTmpValueHolder + RAND(10);
Edit 3 : As pointed out by Jack Williams, Rand() produces a float value between 0 and 1.
So instead, to produce an integer, we need to use a floor function to transform the 'random' float into an integer.
SET NEW.idColumnName =aTmpValueHolder + FLOOR(a + RAND() * (b - a));
where a and b are the range of the random number.

SQL Update statement - syntax error near "SET"?

I am generating the sql statement below based on some coldfusion logic, but it is erroring and I can't seem to find the cause, I have tried making many different modifications to it and nothing seems to be helping.
UPDATE MAIN_RECORDS
SET JONUM = NULL,
SET CUSTNAME = 'Super Sweet Name',
SET CONTACTDT = 02/28/2011,
SET ENGRECDT = 03/07/2011,
SET HOW_WR_DT = 03/07/2011,
SET COMM_DT = 03/29/2011,
SET FACAVALDT = NULL,
SET FAX_SUPDT = 03/07/2011,
SET LINENUM = 'CLPRO L6',
SET POLENUM = 'CLPRO 125 T T3',
SET REASON = '03/07/11 NO VAC FAC THIS IS THE WRONG INFORMATION IT WAS ON HERE TWICE',
SET REC_TYPE = 'H',
SET ORDER_TYPE = 'P',
SET CANCEL_ORDER = 'Y',
SET State_abbr = 'IL',
SET dbfk_state = 17,
SET xx_streetnumber = '2626',
SET xx_street = 'Fake St',
SET xx_city = 'NEWTON',
SET xx_class_of_service_ind = 'R',
SET xx_cellphone_ind = '1',
SET xx_assigned_phone = '3045653897',
SET xx_exchange_name = 'NEWTON',
SET XX_new_ref_code = '60',
SET xx_new_service_type = '11',
SET ORD_COMDT = 03/11/2011,
SET delivery_date = NULL
WHERE ordernum = '08824112' AND exchnum = '304565'
Currently the error that management studio is giving me is:
Msg 156, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'SET'.
You only need 1 SET statement, instead of the multiple ones you have.
Also, your dates need to have single quotes around them.
e.g.:
UPDATE MAIN_RECORDS
SET JONUM = NULL,
CUSTNAME = 'Super Sweet Name',
CONTACTDT = '02/28/2011',
ENGRECDT = '03/07/2011',
HOW_WR_DT = '03/07/2011', .....
Look at the UPDATE statement. The syntax in the post is all wrong :)
The relevant portion:
SET
{ column_name = { expression | DEFAULT | NULL }
| #variable = expression
| #variable = column = expression } [ ,...n ]
Note that SET can only be specified once. The ,...n signifies the previous consuct (that in the {}) can be specified an additional n times, separated with a comma: the SET keyword itself, however, is outside that construct.
Happy coding.
Well generally the command syntax for this would follow this logic
UPDATE table_name
SET column1=value, column2=value2,...
WHERE some_column=some_value
so only one SET not the multiple that you have
Only one SET is needed in update Keyword to update n number of columns - example:
Update Employee
set City = Chennai,Country ='India',Employee name = 'Vignesh'
where Employee Id = 1X234