This documentation states:
http://snappydatainc.github.io/snappydata/rowAndColumnTables/
"Use the PARTITION_BY {COLUMN} clause to provide a set of column names that will determine the partitioning"
I want the following columns to be the partition keys: id_ and time_ .
CREATE TABLE EXAMPLE_COLUMN_TABLE ( id_ VARCHAR(64), name_ VARCHAR(128), time_ TIMESTAMP, number_ INTEGER ) USING column OPTIONS(PARTITION_BY 'time_, id_', buckets '113', PERSISTENT 'ASYNCHRONOUS');
When I run this, it says everything is fine, however, if I do a snappy>describe example_column_table, time_ and id_ still show up as IS_NULLABLE = YES.
If I do only PARTIION_BY 'time_', and then do a snappy> describe example_column_table, that column shows as IS_NULLABLE = NO.
So, I am concerned that multiple column name partitioning is not working?
The column table partitioning works fine even if you specify multiple partition columns. You can verify through query plan that both the columns will be picked as partitioning columns.
The IS_NULLABLE problem is specific to CLOB (VARCHAR) column type. Can you please try following DDL where the second partitioning column number_ is not a CLOB and it also becomes IS_NULLABLE --> NO
snappy> CREATE TABLE EXAMPLE_COLUMN_TABLE2 ( id_ VARCHAR(64), name_ VARCHAR(128), time_ TIMESTAMP, number_ INTEGER ) USING column OPTIONS(PARTITION_BY 'time_, number_', buckets '113', PERSISTENT 'ASYNCHRONOUS');
Related
I have two tables that have a huge list of columns. They are both the same structure, but different data. However, both tables have an index/auto-increment column that might be similar. Is there an easy way to run a command like this:
insert into table1 (select * from table2);
and have the insert ignore the auto-increment column from table 2? To avoid an error if there's a similar-value in the index column of tables 1 and 2? I want to copy everything over, and have new auto-increments for the table 2 data in table 1.
Alternatively, I don't care what the values are of the auto-increment index. If there was a way to merge the two tables and then re-generate unique AI columns that would also work.
I am aware I could get around this by specifying each field individually in both tables and leaving out the auto-increment column. I'm just wondering if there is an easier way to do this? If there isn't, is there an easy way of generating the field list/statement?
Here is the most efficient way I know of right now. Assuming the A.I. index is called "recno"
ALTER TABLE table1 DROP COLUMN recid;
ALTER TABLE table2 DROP COLUMN recid;
insert into table1 (select * from table2);
ALTER TABLE table1 ADD `recid` INT NOT NULL AUTO_INCREMENT [AFTER `column`], ADD PRIMARY KEY (`recid`);
ALTER TABLE table2 ADD `recid` INT NOT NULL AUTO_INCREMENT [AFTER `column`], ADD PRIMARY KEY (`recid`);
There's not really a way to do that. The * in the SELECT list means "all columns" in ordinal position. There's not exception for columns that meet specified criteria.
The same is true for an omitted list of columns that we're inserting into... it's all of the columns.
The most efficient way (in terms of database resources) to accomplish the specified goal is to list the columns, and omit the auto_increment column from the list.
INSERT INTO t (b,c,d,e) SELECT b,c,d,e FROM s ;
We can get a list of columns names for a table from information_schema.columns...
For example, to get a list of the column names in table2:
SELECT c.column_name
FROM information_schema.columns c
WHERE c.table_schema = 'mydatabase'
AND c.table_name = 'table2'
ORDER
BY c.ordinal_position
To exclude the auto_increment column from the list, we can add a WHERE clause, that excludes that column by it's name
WHERE c.column_name NOT IN ('my_autoincrement_column_name')
or we can check for 'auto_increment' occurring the EXTRA column
WHERE c.extra NOT LIKE '%auto_increment%'
To get a column list for each of the two tables, excluding the auto_increment columns, we could do something like this:
SET group_concat_max_len = 16777216 ;
SELECT GROUP_CONCAT( CONCAT('`',c.column_name,'`')
ORDER BY c.ordinal_position
SEPARATOR ','
) AS `-- column_list`
FROM information_schema.columns c
WHERE c.table_schema = 'mydatabase'
AND c.table_name IN ('table1','table2')
AND c.extra NOT LIKE '%auto_increment%'
GROUP
BY c.table_schema
, c.table_name
ORDER
BY c.table_schema
, c.table_name
EDIT
I'd take the column lists, and build a SQL query.
If we're going to make changes to table2 (dropping the auto_increment column as suggested in another answer to this question), and if we don't need to preserve the values in the column, the easiest change would be to just set all of the values in that column to NULL.
No need to modify table1.
We can remove the auto_increment attribute (and the NOT NULL constraint if that's specified) from the column in table2, and set the column to null. Assuming ai is the name of the auto_increment column, and assuming it's declared to be INT UNSIGNED datatype, we can do:
ALTER TABLE `table2` CHANGE `ai` `ai` INT UNSIGNED COMMENT '' ;
UPDATE `table2` SET `ai` = NULL ;
Then we can do an INSERT INTO table1 SELECT * FROM table2
And then add back the auto_increment attribute to the column in table2.
This approach is more expensive (in terms of the database resources) than a single INSERT ... SELECT.
Generating a column list is an extra step, but the resulting operation would be much more efficient. We could generate a list of columns from just the source table, and then replace the column name with literal NULL value...
INSERT INTO t SELECT NULL,b,c,d FROM s;
I've got a SQL 2008 R2 table defined like this:
CREATE TABLE [dbo].[Search_Name](
[Id] [bigint] IDENTITY(1,1) NOT NULL,
[Name] [nvarchar](300) NULL),
CONSTRAINT [PK_Search_Name] PRIMARY KEY CLUSTERED ([Id] ASC))
Performance querying the Name field using CONTAINS and FREETEXT works well.
However, I'm trying to keep the values of my Name column unique. Searching for an existing entry in the Name column is unbelievably slow for a large number of names (usually batches of 1,000), even with an index on the Name field. Query plans indicate I'm using the index as expected.
To search for an existing value, my query looks like this:
SELECT TOP 1 Id, Name from Search_Name where Name = 'My Name Value'
I've tried duplicating the Name column to another column and searching on the new column, but the net effect was the same.
At this point, I'm thinking I must be mis-using this feature.
Should I just stop trying to prevent duplication? I'm using a linking table to join these search name values to the underlying data. It seems somehow 'dirty' to just store a whole bunch of duplicate values...
...or is there faster way to take a list of 1,000 names and see which ones are already stored in the database?
The first change to make is to get the entire list to SQL Server at one time. Regardless of how you add the names to the existing table, doing it as a set operation will make a big difference in performance.
Passing the List as a table-valued parameter (TVP) is a clean way to handle it. Have a look here for an example. You can still use an OUTPUT clause to track which rows did or didn't make the cut, for example:
-- Some sample existing names.
declare #Search_Name as Table ( Id Int Identity, Name VarChar(32) );
insert into #Search_Name ( Name ) values ( 'Bob' ), ( 'Carol' ), ( 'Ted' ), ( 'Alice' );
select * from #Search_Name;
-- Some (prospective) new names.
declare #New_Names as Table ( Name VarChar(32) );
insert into #New_Names ( Name ) values ( 'Ralph' ), ( 'Alice' ), ( 'Ed' ), ( 'Trixie' );
select * from #New_Names;
-- Add the unique new names.
declare #Inserted as Table ( Id Int, Name VarChar(32) );
insert into #Search_Name
output inserted.Id, inserted.Name into #Inserted
select New.Name
from #New_Names as New left outer join
#Search_Name as Old on Old.Name = New.Name
where Old.Id is NULL;
-- Results.
select * from #Search_Name;
-- The names that were added and their id's.
select * from #Inserted;
-- The names that were not added.
select New.Name
from #New_Names as New left outer join
#Inserted as I on I.Name = New.Name
where I.Id is NULL;
Alternatively, you could use a MERGE statement and OUTPUT the names that were added, those that weren't, or both.
Currently we have an interessting problem regarding the sort order of MySQL in an enum-field. The fields enum entries have been sorted in the order we want it. Just to be save, we added a CONCAT around it, so it would be cast to char and ordered in alphabetical order, just as suggested by the MySQL-reference (MySQL Reference - Enum)
Make sure that the column is sorted lexically rather than by index number by coding ORDER BY CAST(col AS CHAR) or ORDER BY CONCAT(col).
But that didn't produce the expected results, so we started to investigate further. It seems that the order by statement doesn't work on a combination of enum and the concat function. I've wrote the following sample script, which should show my point:
CREATE TABLE test (
`col1` enum('a','b','c') COLLATE utf8_bin DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
INSERT INTO test
VALUES ('b'), ('c'), ('a');
SELECT * FROM test; -- b, c, a
SELECT * FROM test ORDER BY col1 ASC; -- a, b, c
SELECT * FROM test ORDER BY CAST(col1 AS CHAR) ASC; -- a, b, c
SELECT * FROM test ORDER BY CAST(col1 AS BINARY) ASC; -- a, b, c
SELECT * FROM test ORDER BY CONCAT(col1) ASC; -- b, c, a - This goes wrong
I am currently suspecting some kind of problem with the collation/encoding, but I'm not sure. My databases default encoding is also utf8. The MySQL version is 5.6.12 but it seems to be reproduceable with MySQL 5.1. The storage engine is MyIsam but it also occurs with the memory engine.
Any help would be appreciated.
Update:
As it seems the problem is produced only in MySQL 5.6 and by the collation of the column. With the first CREATE TABLE statement, the queries work fine.
CREATE TABLE test (
`col1` enum('a','b','c') COLLATE utf8_general_ci DEFAULT NULL
)
With the second they don't.
CREATE TABLE test (
`col1` enum('a','b','c') COLLATE utf8_bin DEFAULT NULL
)
The collation of the table and/or database don't seem to affect the queries. The queries can be tested in this SQL Fiddle
Strange,it works in this fiddle.Do you have a trigger or something?
http://sqlfiddle.com/#!2/0976a/2
BUT,in 5.6 goes haywire:
http://sqlfiddle.com/#!9/0976a/1
Mysql bug,probably.
More,if you input the values in the enum in the "proper" order it works:
http://sqlfiddle.com/#!9/a3784/1
IN the doc:
ENUM values are sorted based on their index numbers, which depend on
the order in which the enumeration members were listed in the column
specification. For example, 'b' sorts before 'a' for ENUM('b', 'a').
As per the document:
Under the Handling of Enumeration Literals section, it states that:
If you store a number into an ENUM column, the number is treated as
the index into the possible values, and the value stored is the
enumeration member with that index. (However, this does not work with
LOAD DATA, which treats all input as strings.) If the numeric value is
quoted, it is still interpreted as an index if there is no matching
string in the list of enumeration values. For these reasons, it is not
advisable to define an ENUM column with enumeration values that look
like numbers, because this can easily become confusing.
For example, the following column has enumeration members with string values of '0', '1', and '2', but numeric index values of 1, 2, and 3:
numbers ENUM('0','1','2')
If you store 2, it is interpreted as an
index value, and becomes '1' (the value with index 2). If you store
'2', it matches an enumeration value, so it is stored as '2'. If you
store '3', it does not match any enumeration value, so it is treated
as an index and becomes '2' (the value with index 3).
mysql> INSERT INTO t (numbers) VALUES(2),('2'),('3');
mysql> SELECT * FROM t;
+---------+
| numbers |
+---------+
| 1 |
| 2 |
| 2 |
+---------+
In your case:
INSERT INTO test
VALUES ('2'), ('3'), ('1');
Index value of '2' is 2, '3' is 3 and '1' is 1.
So the output is 2,3,1
I'm trying to create a trigger which will capture any event that will occur when I update any column in the table before and after updating, let's say I have 4 columns:
first_name address city country
Let's say I edited first_name lets say Jack to Henk.
It should insert in another table the command (i.e. update) ,time , description but inside the description I want it to write Jack was changed to John by current user(i.e using the current-user () function),if it is a city being updated from Mechinkova to Tostov, it should do the same do with other columns.
I know I want to have to add the concat function inside the trigger, I want it to be like this for example:
DROP TRIGGER IF EXISTS adminpanel.soft//
CREATE TRIGGER adminpanel.soft BEFORE UPDATE ON adminpanel.aggrement
FOR EACH ROW
BEGIN
INSERT INTO adminpanel.aggretrigger(cmd, time, cmd_user, last_name, city) VALUES("INSERT", NOW(), CURRENT_USER(), new.last_name, new.city);
END
//
What you are asking for is an audit trigger. It is very easy to implement.
Let us first slightly modify your main table. Let's add a field id of integer datatype as the primary key to the table, so your table would look like:
tablename
( id integer PK
, first_name varchar
, address varchar
, city varchar
, country varchar
)
Now, you will need a table, say UNIVERSAL_AUDIT_ENTRY table that will store the changes made to the data in your schema.
From what experience I have, I suggest you create this table as follows:
universal_audit_entry
( universal_audit_entryid integer PK
, table_name varchar -- captures the name of the table
, column_name varchar -- captures the name of the column
, entry_type varchar -- captures the event, e.g., 'INSERT' or 'UPDATE'
, primary_key_value integer -- captures, e.g., the value in tblename.id
, from_str varchar -- captures the value that was present before
, to_str varchar -- captures the value that was changed into
, timestamp datetime -- captures the timestamp of the event
, username varchar -- captures the name of user
)
Now with the universal_audit_entry table ready, your trigger should look somewhat like:
CREATE TRIGGER adminpanel.soft
BEFORE UPDATE ON adminpanel.aggrement
FOR EACH ROW
BEGIN
IF UPDATING(first_name) THEN
INSERT INTO universal_audit_entry VALUES
( 123 -- example for universal_audit_entryid
, 'TABLENAME'
, 'FIRST_NAME'
, 'UPDATE'
, new.id
, old.first_name
, new.first_name
, current_timestamp()
, current_user);
END IF;
END;
//
You can use similar logic to audit more columns in the same table and other tables also.
Note:
This code is not tested. I have added it here only for illustration purposes. This code for trigger is not supposed to be used directly.
new and old are the pseudo-records that are generated during an update statement. These records correspond to the rows that are being updated. :new means the row after the update statement runs and :old means the row before the update statement runs. This works in Oracle. Kindly make sure if it works in MySQL also.
EDIT
You can read more about MySQL triggers here. Read more about audit trail here and this SO question.
Does anyone know a script to alter the data type of a column in a table which is part of number of databases? e.g.
I have a user table in 20 different databases with same columns. I want to change the datatype of one of the columns from that table.
You can use the information_schema to generate DDL commands to modify the columns.
For example, if all of the tables are named "user" (and there are no "user" tables in other databases that you DO NOT want to change), and your column is named "change_me", and you want to make it an unsigned int that is never null, you could do something like this:
select concat('ALTER TABLE ',table_schema,
'.',table_name,
' MODIFY COLUMN ',column_name,
' INT UNSIGNED NOT NULL;') as sql_stmt
into outfile '/tmp/modify_columns.sql'
from information_schema.columns
where table_name = 'user'
and column_name = 'change_me';
\. /tmp/modify_columns.sql
Also remember that many times, the column must be empty to change data types and will depend upon DBMS. Please specify from and to what datatype. Alternatives:
desc foo;
Name Null Type
------------------------------ -------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BAR1 NUMBER
-- this will not work:
alter table foo modify ( bar1 varchar(255) );
--instead
alter table foo add ( bar2 varchar(255));
-- assume the DB will auto change type:
update foo set bar2=bar1;
alter table foo drop (bar1);
alter table foo rename column bar2 to bar1;