Mariadb columnstore autoincrement not working - mysql

I'm trying to use Mariadb columnstore, but I'm having some errors when inserting data in a table with a autoincrement column defined.
The issue happens when using the JDBC driver.
CREATE TABLE schema.mytable
(
deaf_id bigint NOT NULL COMMENT 'autoincrement=1',
name varchar(80) NOT NULL,
country varchar(14) NOT NULL
) ENGINE=ColumnStore;
Using DBeaver, I try to insert a new data on the table:
INSERT INTO schema.mytable
(name, country)
VALUES('ny', 'usa');
But I get the following error:
SQL Error [1364] [HY000]: (conn:4) Field 'deaf_id' doesn't have a
default value
But using the mcsmysql, I can add the data successfully:
MariaDB [schema]> INSERT INTO schema.mytable (name, country) VALUES('ny', 'usa');
Query OK, 1 row affected, 1 warning (0.28 sec)
MariaDB [schema]> select * from schema.mytable;
+---------+------+---------+
| deaf_id | name | country |
+---------+------+---------+
| 1 | ny | usa |
+---------+------+---------+
1 row in set (0.07 sec)
MariaDB [bovespa]>
I tried to use the both drivers:
https://downloads.mariadb.com/Connectors/java/connector-java-2.0.1/mariadb-java-client-2.0.1.jar
https://downloads.mariadb.com/Connectors/java/connector-java-1.5.9/mariadb-java-client-1.5.9.jar
Is there any known issue for this ? Or am I missing something ?
Kleyson Rios.

Short answer:
Is not a client or driver issue. It is a Columnstore user privileges issue.
Suppose that your user that you use to connect to Columnstore is called 'myuser'. Run the following as root:
-- if necessary replace % below with localhost or whatever IP you use for the connection
GRANT ALL PRIVILEGES ON `infinidb_vtable`.* TO 'myuser'#'%';
FLUSH PRIVILEGES;
Now try to connect again with 'myuser', no matter whether you use DBeaver or whatever client. You should be able to run the insert.
Long answer:
First of all, let's review the proper create table syntax because they are different between Columnstore and Innodb when it comes to defining the autoincrement fields:
-- Columnstore syntax
CREATE TABLE schema.mytable
(
deaf_id bigint NOT NULL COMMENT 'autoincrement=1',
name varchar(80) NOT NULL,
country varchar(14) NOT NULL
) ENGINE=ColumnStore;
-- InnoDB syntax
CREATE TABLE schema.mytable
(
deaf_id bigint NOT NULL AUTO_INCREMENT,
name varchar(80) NOT NULL,
country varchar(14) NOT NULL,
PRIMARY KEY (`deaf_id`)
) ENGINE=InnoDB;
Don't mix them up as you might get errors or unexpected results.
The issue you described gave me some headaches too in the past. The error below is pretty misleading:
SQL Error [1364] [HY000]: (conn:4) Field 'deaf_id' doesn't have a
default value
If you tried to run:
select * from mytable;
you would get a much more useful error:
ERROR 1044 (42000): Access denied for user 'myuser'#'%' to database 'infinidb_vtable'
Based on this error I found this page which gives detailed explanations:
https://mariadb.com/kb/en/mariadb/columnstore-database-user-management/
So, just run the GRANT that I wrote at the beginning of the answer and your problem is solved. Don't forget to run the same grant on all the users that you want to be able to access Columnstore tables.
As a short advice before finishing..
Think twice whether you want to use Columnstore or stick with MySQL/MariaDB InnoDB.
We are currently under a Columnstore evaluation. It showed us some really impressive performance when it came to the speed of selects on huge tables. I doubt I could have obtained half of that performance with MySQL/MariaDB even after ellaborate tuning.
But Columnstore has its own drawbacks. I would mention two of them which gave us some headaches:
the speed of the inserts is poor (up to 20 times slower than on a simillar InnoDB table according to some of our tests)
Columnstore did not yet reach maturity so it might give you headaches at times
So choose your database engine carefully.
[Edit:]
I forgot to mention that I'm speaking about MariaDB Columnstore 1.0.9. Future versions might show different syntax/behaviour, etc..

GRANT PRIVILEGES didn't solve my problem. I fixed this by declaring the id autoincrement field as NULL instead of NOT NULL.
CREATE TABLE schema.mytable
(
id bigint NULL COMMENT 'autoincrement=1',
name varchar(80) NOT NULL,
country varchar(14) NOT NULL
) ENGINE=ColumnStore;

Related

Allow MySQL users access to select on all views, but not tables

How do I allow users to have select access to Views only, and not any of the underlying tables?
I know how to grand permissions one at a time using a
grant select on database.viewName to 'User'
but I am trying to make either a user or a profile that have access to all views without running the grants 1 by 1 whenever a new view is created.
In Sequel Pro, under Users>Global Privileges and Schema Privileges I added "Show View" but this is not working.
This answer was verified using MySQL 8.0.23.
Make sure that your views are set to run using the credentials of the DEFINER, that way the invoking user doesn't require any TABLE privileges; not doing this will result in an error. For more details on the {DEFINER|INVOKER} parameter see here.
Create your user, allocating no privileges at all.
As root or another user with sufficient privileges, execute the following command:
GRANT SELECT ON <database>.<view_name> TO '<user>'#'<host>';
So if my database were cars and the view were called service_history, and I had a user john.doe that needed to access this from any host, I would execute the following:
GRANT SELECT ON cars.service_history TO 'john.doe'#'%';
Note how '%' was used to indicate any host is acceptable, you can replace this with an IP address, maybe a hostname (but I have not tested that).
I don't believe this is possible.
Mysql GRANT commands take the form GRANT permission ON object_type
The object_type clause, if present, should be specified as TABLE, FUNCTION, or PROCEDURE when the following object is a table, a stored function, or a stored procedure.
https://dev.mysql.com/doc/refman/5.7/en/grant.html
I don't believe there's any facility for granting permissions solely on a view.
Keep in mind that, in mysql, a view is just a query which is executed whenever you SELECT from it. It has no data storage or indexes of its own. As such, I expect you have to have access to the underlying tables in order for the view to be usable.
I will show it, might take a few edits to do it. Because I have to attach console output at the bottom.
Schema
create table users
( userId int auto_increment primary key,
userName varchar(100) not null
-- etc
);
insert users(userName) values ('Joe'),('Gertrude');
-- drop table secretdata;
create table SecretData
( id int auto_increment primary key,
userId int not null,
theKey varchar(100) not null,
theValue varchar(1000) not null,
key(userId),
CONSTRAINT fk_sd_users FOREIGN KEY (userId) REFERENCES users(userId)
);
insert secretdata(userId,theKey,theValue) values
(1,'FB Password','8*&Fjj_anchovieS'),
(1,'Proper response','I love the meal, just like I like it'),
(2,'thing7','goat');
The view
CREATE VIEW userSecrets AS
select u.userId,u.userName,s.theKey,s.theValue
from users u
join SecretData s
on s.userId=u.userId;
select * from userSecrets where userId=2; -- works here at the moment
Note my db name is so_gibberish.
The following is what you are asking about for grant rights to view:
GRANT SELECT ON so_gibberish.userSecrets TO 'john'#'host';
↑ ↑ users so provisioned can then rifle off queries against the view. ↑ ↑
however,
CREATE USER 'plebian2'#'localhost' IDENTIFIED BY 'mypass';
GRANT SELECT ON so_gibberish.userSecrets TO 'plebian2'#'localhost';
That user does have rights to use the view, but does not have access to the underlying tables directly.
Console output:
OS Prompt>mysql -u plebian2 -p
Enter password: ******
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 938
Server version: 5.6.24-log MySQL Community Server (GPL)
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| so_gibberish |
+--------------------+
2 rows in set (0.00 sec)
mysql> use so_gibberish;
Database changed
mysql> show tables;
+------------------------+
| Tables_in_so_gibberish |
+------------------------+
| usersecrets |
+------------------------+
1 row in set (0.00 sec)
mysql> select * from users;
ERROR 1142 (42000): SELECT command denied to user 'plebian2'#'localhost' for table 'users'
mysql> select * from usersecrets;
+--------+----------+-----------------+--------------------------------------+
| userId | userName | theKey | theValue |
+--------+----------+-----------------+--------------------------------------+
| 1 | Joe | FB Password | 8*&Fjj_anchovieS |
| 1 | Joe | Proper response | I love the meal, just like I like it |
| 2 | Gertrude | thing7 | goat |
+--------+----------+-----------------+--------------------------------------+
3 rows in set (0.00 sec)
Lastly, I hope, see Manual Page entitled Access Control for Stored Programs and Views to see how the Definer permissions factor into the end-state effective rights. In particular, when the definer has less privileges than the user account using the view (a rather weird condition, but there it is).
Roles
In response to your comment under the Answer.
I was basically dissatisfied with the lack of roles embedded in mysql coming from the Microsoft world where they existed. Meaning, there are made-up roles, users are attached to roles, and grants are performed at the role level (in the MSFT world). There, in MSSQL, at the bottom of each create whatever block (stored proc etc), would be my block that did grants at the role level.
So coming into mysql, I had to create that role to user mapping. Good news, I only had to do it once. And the end of the chunk (if exist drop / create) that I run, it makes calls that directly manipulate the INFORMATION_SCHEMA.user_privileges and the like. But at a very high level, just like with MSFT, with the end result being what I wanted. That is, permissions granted.
That is a whole can of worms. But without it, good luck.
The commands like grant and some other commands can be performed on your own under the hood. There is little sacred about them. Yet they are there to protect novice programmers. And those that foresake those Warning Signs saying be careful are often left thinking,
"How do I get my data back. I know it's in there somewhere. Maybe I
shouldn't have done that under the hood thing I just did"
But definitely something to tinker with, especially in a throwaway db (one you don't care about). Even better, do it on a mysql instance you don't care about. Because we are talking about system wide implications at a higher level than at the db level.
I got this working by using user() as the filter on the tables and setting SQL SECURITY DEFINER, using separate schemas for views and source tables.
Example code
CREATE
ALGORITHM = UNDEFINED
DEFINER = `xxxxxx`#`%`
SQL SECURITY DEFINER
VIEW `XX`.`assets` AS
SELECT
`a`.`ID` AS `ID`,
`a`.`serial_no` AS `serial_no`,
`a`.`sys_id` AS `sys_id`,
`a`.`status` AS `status`,
`a`.`unit_name` AS `unit_name`,
`a`.`ip` AS `ip`,
`a`.`tzOffset` AS `tzOffset`
FROM
((`YY`.`Cust_Link_Table` `l`
JOIN `YY`.`systems` `s` ON ((`l`.`cust_id` = `s`.`cust_id`)))
JOIN `YY`.`assets` `a` ON ((`s`.`sys_id` = `a`.`sys_id`)))
WHERE
(`l`.`db_user` = CONVERT( LEFT(USER(), (LOCATE('#', USER()) - 1)) USING UTF8MB4))

MySQL: unexpected SELECT result after INSERT into table with autoincrement column

I'm seeing a weird behavior when I INSERT some data into a table and then run a SELECT query on the same table. This table has an auto-increment primary key (uid), and this problem occurs when I try to then select results where 'uid IS NULL'.
I've golfed this down to the following SQL commands:
DROP TABLE IF EXISTS test_users;
CREATE TABLE test_users (uid INTEGER PRIMARY KEY AUTO_INCREMENT, name varchar(20) NOT NULL);
INSERT INTO test_users(name) values('foo');
SELECT uid FROM test_users WHERE uid IS NULL;
SELECT uid FROM test_users WHERE uid IS NULL; -- no output from this query
I'd expect SELECT uid FROM test_users WHERE uid IS NULL to never return anything, but it does, sometimes. Here's what I've found:
Version of MySQL/MariaDB seems to matter. The machine having this problem is running MySQL 5.1.73 (CentOS 6.5, both 32-bit and 64-bit). My other machine running 5.5.37-MariaDB (Fedora 19, 64-bit). Both running default configs, aside from being configured to use MyISAM tables.
Only the first SELECT query after the INSERT is affected.
If I specify a value for uid rather than let it auto-increment, then it's fine.
If I disconnect and reconnect between the INSERT and SELECT, then I get the expected no results. This is easiest to see in something like Perl where I manage the connection object. I have a test script demonstrating this at https://gist.github.com/avuserow/1c20cc03c007eda43c82
This behavior is by design.
It's evidently equivalent to SELECT * FROM t1 WHERE id = LAST_INSERT_ID(); which would also work only from the connection where you just did the insert, exactly as you described.
It's apparently a workaround that you can use in some environments that make it difficult to fetch the last inserted (by your connection) row's auto-increment value in a more conventional way.
To be precise, it's actually the auto_increment value assigned to the first row inserted by your connection's last insert statement. That's the same thing when you only inserted one row, but it's not the same thing when you insert multiple rows with a single insert statement.
http://dev.mysql.com/doc/connector-odbc/en/connector-odbc-usagenotes-functionality-last-insert-id.html

Why does this table name require back-ticks?

I am seeing a curious name dependency in the following MySQL table definition. When I code the table as shown, it seems to break MySQL. When I perform "select * from dictionary_pair_join" from MySQLQueryBrowser, the status bar says "No resultset returned" -- no column names and no errors. When I insert a row into the table, the status bar says "1 row affected by the last command, no resultset returned", and subsequent selects give the same "No resultset returned" response.
When I enclose the tablename in backticks in the "select" statement, all works fine. Surely there are no mysql entities named "dictionary_pair_join"!
Here is the table definition:
DROP TABLE IF EXISTS dictionary_pair_join;
CREATE TABLE dictionary_pair_join (
version_id int(11) UNSIGNED NOT NULL default '0',
pair_id int(11) UNSIGNED default NULL,
KEY (version_id),
KEY (pair_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Here is the broken select statement:
select * from dictionary_pair_join;
Here is its working counterpart:
select * from `dictionary_pair_join`;
Why are backticks required in the select statement?
Update: This also fails in the Python mysqldb interface, which is why I started looking at it. I can put the backticks into my Python "select" generators, but I was hoping this was some stupid and easily-changed nit. I suppose I can also find a different name.
I've uprated the comments from Quassnoi and Software Guy, together they've persuaded me that it's just a bug in mysql/mysqldb/mysqlquerybrowser.
I changed the table name (to "dictionary_pair_cons") and the problem went away, in both the query browser and mysqldb.

Can I force MySql table name case sensitivity on file systems that aren't case sensitive

So our target environment is linux, making mysql case-sensitive by default. I am aware that we can make our linux environment not case sensitive with the lower_case_table_names variable, but we would rather not. We have a few times been bitten with a case mismatch because our dev rigs are OSX, and mysql is not case sensitive there.
Is there a way we can force table names to be case sensitive on my OSX install of MySql (5.0.83 if that matters) so that we catch a table name case mismatch prior to deploying to the integration servers running on linux?
Set lower_case_table_names=0 in my.cnf.
If you installed via homebrew, the file is here:
/usr/local/Cellar/mysql/<version>/my.cnf
Queries with tables should now be case sensitive:
mysql> select count(*) from user;
ERROR 1146 (42S02): Table 'xxx.user' doesn't exist
mysql> select count(*) from User;
+----------+
| count(*) |
+----------+
| 1 |
+----------+
1 row in set (0.00 sec)
The best thing to do here is fix your table names so that there aren't any conflicts. Differentiating solely by case is a bad idea, and leads to confusion (as you probably know).
But try using single quote marks around the table names during creation. This works on SUSE/Linux/MySQL 5.0 with the query browser running on windows.
CREATE TABLE `MySchema`.`test` (
`COMMENT` text
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `MySchema`.`Test` (
`COMMENT` text
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
insert into MySchema.test values ('this is table test' );
insert into MySchema.Test values ('this is table Test' );
select * from MySchema.test;
select * from MySchema.Test;
Do you want it to fail if a non-case sensitive client requests a table using the wrong case? I believe it should fail if the MySQL database is running on Linux.
Check out this link
"One notable exception is Mac OS X, which is Unix-based but uses a default file system type (HFS+) that is not case sensitive. However, Mac OS X also supports UFS volumes, which are case sensitive just as on any Unix."

MySQL - A way to join all found rows in one column

CREATE TABLE dummy (
id INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY ,
name VARCHAR( 30 ) NOT NULL
) ENGINE = MYISAM ;
and running this query:
SELECT GROUP_CONCAT(`name` SEPARATOR "||") FROM `dummy`
This query joins name column in all rows with || in a single column. BUT, the result is truncated with mysql configuration as explained in mysql manual :
"... result is truncated to the maximum length that is given by the group_concat_max_len system variable, which has a default value of 1024 ..."
Also in manual it is said that this setting can be changed in run time by the following syntax:
SET [GLOBAL | SESSION] group_concat_max_len = val;
Does changing this configuration works in all mysql server environments? If not, how can I achieve the same result without GROUP_CONCAT without limits?
Also, I think that changing the configuration will not solve my problem because I don't know what to set the value of group_concat_max_len because number of rows in dummy table can be any number.
Thanks.
Have you tried using stored procedure to accomplish your task? You can create a temporary table with a single row/column and append to it while fetching rows from your table. In the end just SELECT the single value from the temporary table.
You can find information about stored routines in mysql manual and other places.