Is there any way to issue a mysql statement to create a table without having to assign the number of columns? I am working with the MySQL C API for grabbing some variables and then storing them in a table. The issue that I am encountering is that I have to create the table (obviously) before inserting the variables into the table. These variables sometimes are structures (two, three or four variables into a single table), so I am looking for a way of not having to say:
CREATE TABLE Structures(ID varchar(10) primary key, name varchar(25))
but creating a table on where any number of columns can be inserted?
Let me know if I am being a bit vague in here.
No, you can't. You can however add columns at runtime using ALTER TABLE.
However, personally, I wouldn't recommend that. You should know what your database looks like, before you start implementing it.
The other way to code this is to use two tables and a one-to-many between them.
For instance, you might have a tables like this - pcode,
table experiment
experiment_id: long
experiment_header: varchar(50)
table experiemnt_data
experiemnt_data_id: long
experiment_id: long
key: varchar(20)
value: long
#id = insert into experiment (experiment_header) value("test run")
insert into experiment_data (experiment_id, key, value) value(#id, 'x', 1)
insert into experiment_data (experiment_id, key, value) value(#id, 'y', 20)
AS #Mark and #attis said:
You can't. You can however add columns at runtime using ALTER
TABLE.
However, personally, I wouldn't recommend that. You should know what
your database looks like, before you start implementing it.
I think the best solution could be:
Create two tables :
column with (id, name)
values with (id, column_id, value)
then you just have to join them to easily get you results, and you can easily add others "columns"
You can also store everything in values table, but your data may be inconsistent, and, in my mind, it's faster to look for a number than to compare strings (table lock, index etc...)
I wanted to comment #Mark post, but can't (reputation too low)
Related
I need a solution to insert a lot of rows in concurrent time in my sql DB.
I have a rule, that everytime I insert to my transaction table, I need a unique ID that's composed by currentTime+transactionSource+sequenceNumber. my problem is, when I test my service using Jmeter, the service is down when the concurrent insert process is up to 3000 rows. the problem relies on, the duplication of the unique ID I generate. so, there are some duplications. in my assumption, the duplication happen because a previous insert process hasnt finished, but there's another insert process. So,it generates unique ID duplication.
Can anyone give me suggestion in what the best way for doing this? Thank you.
MySQL has three wonderful methods to ensure that an id is unique:
auto_increment columns
uuid()
uuid_short()
Use them! The most common way to implement a unique id is the first one:
create table t (
t_id int auto_increment primar key,
. . .
)
I strongly, strongly advise you not to maintain your own id. You get race conditions (as you have seen). Your code will be less efficient than the code in the database. If you need the separate components, you can implement them as columns in the table.
In other words, your fundamental problem is your "rule". And there are zillions of databases in the world that work perfectly well without such a rule.
Why don't you let the database handle the insert id and then update the row with a secondary field containing the format you want ? If you have dupplicates, you can always append the row id to this identifier so it will always be unique.
I have multiple tables.
They all have the following fields in them:
item_title | item_description | item_thumbnail | item_keywords
Would I be better off having a single items_table with an extra item_type field and then joining with the respective table, or just keep them all in separate tables?
Depends on the context. If your items have very little differentiation and you’re certain you’re not going to have a scenario in 6 months, 12 months, 2 years where you need items separated, then go the route of one generic “items” table. If a particular item type does have specific requirements, then you can create a separate table that contains this data and create a LEFT JOIN when querying to include the extra data.
I’d also suggest looking at other database types. Judging from your scenario (lots of item types with little variance in the data stored) I think you may benefit from a document-based database engine like MongoDB rather than a relational data-based database engine like MySQL.
OK, so the tables share fields. Do they also share constraints1?
If yes, then go ahead and merge them together.
If not, you may keep them separate, of may merge them together, depending on what kind of tradeoff you are willing to make.
For example, if tables have separate foreign keys, you may keep them separate, or you may merge them into a single table, but keep FKs separate:
item_title
item_description
item_thumbnail
item_keywords
table1_id REFERENCES table1 (table1_id)
table2_id REFERENCES table2 (table2_id)
...
CHECK (
(table1_id IS NOT NULL AND table2_id IS NULL ...)
OR (table1_id IS NULL AND table2_id IS NOT NULL ...)
...
)
(NOTE: MySQL Doesn't enforce CHECK, so you'll need to do the equivalent enforcement from a trigger or client code, or use a different DBMS if you can.)
I'd need to know more about your database to figure out which is better.
with an extra item_type field and then joining with the respective table,
Never enforce FKs in code, if you can help it. Even if you merge the tables together, don't merge FKs, instead do something like the above. Enforcing FKs in code in the context of the concurrent environment (where multiple clients can try to modify the same data at the same time) is difficult to do correctly and with good performance - it's much better to let the DBMS do it for you.
BTW, what is item_keywords? It it's a comma-separated list of keywords (or similar), you'll need to normalize further and extract the keywords into their own separate table.
1 Domain (data type and CHECK), key (PRIMARY KEY and UNIQUE) and referential (FOREIGN KEY) constraints.
I believe that it is good to have as less table as possible. It is easy to maintain. It is hard to imagine that if you have 3000 type of item_type. Then, there would be 3,000 different table. So single table is good idea to me in your case. In the future, when you run into situation when you need to separate the table, you can easily do so.
So the short answer, YES.
If i understand well, you only need to normalize your schema:
items:
item_id
item_name
item_description
items_types
item_id
type_id
types
type_id
item_file_name
So this way you can have any number of items with any number of types
Is this you want to do???
I would suggest you to use one table for item and one table for type for the following reasons (assume there are 10 types).
I am not sure which programming language you are using. As a Java developer, i will have to create each entity class for each type if I have multiple tables. So i would rather have only one class and have a type as an attribute.
When you have to display all of the types in the same page, you will have to execute the select query from all 10 tables for 10 types.
When you introduce a new type, you have to write the code to for the CRUD and Business specific operations. The developer will keep on adding the code for every new type.
Basically, if you have one table for item and one table for type, you won't have to change the database schema and code for each new type you introduce. But if you are sure that, the number of types is less and won't change, you can consider using muiltiple tables.
Create two separate tables and join them as per your required output.
i.e>
1.1'st TABLE (master table==>item_type)
item_type(item_type_id,item_type_name,status)
2.2'nd TABLE(child table==>item_details)
item_details(item_id,item_type_id,item_title,item_description,item_thumbnail,item_keywords)
See more examples..
I feel signle table would be more suitable. It will avoid more joins, complication in program(Code) and errors in compare of multiple tables. Even it will be better from the management point of view like db clustering etc.
If you have so many tables which needs to have the same repeated columns then yes it is a good way to create a separate table for the common fields. This is more efficient if these repeated columns are not fixed and can be changed like adding one more column to the list of common default columns.
So how could you do that?
The idea is to create a seperate table and put the common default columns there.
This table is like a dummy table i.e. the columns can be added/deleted as needed.
For example-
Table - DefaultFields
Columns - item_title | item_description | item_thumbnail | item_keywords
You can then also be able to insert the values in the DefaultFields table dynamically in a loop like:
"INSERT INTO DefaultFields (item_table, item_title , item_description,item_thumbnail ,item_keywords) VALUES('"+ field.item_table + "','" + field.item_title + "','" + field.item_description+ "','" + field.item_thumbnail + "','" + field.item_keywords)");
NOTE: field is the object that holds the values in a table wise loop.
Then further you can alter your tables to create these default fields from DefaultFields table like:
"ALTER TABLE " + item_table+ " ADD COLUMN [" + field.item_title + "] Text"
This can be repeated for each table to alter it as needed.
In this design pattern, even if you want to:
1) add one more column or
2) delete pre existing column or
3) change pre existing column name
Then you can do so in the dummy table and the rest is updated by the ALTER table command in corresponding tables.
In my opinion... I would say no, never.
There is two reason for that:
You really want to preserve a logical meaning in your database. For now it's pretty obvious for you how it's organised. But in two month (or 1 year), will it be so evident? If somebody join the project, isn't it easier for him to understand if the different logical block of your app are separated? I mean... It's true that a human and a cat are animals. Is it still logical to store both of them inside the same box?
Performance. The shorter the table, the faster your request will be. The data will still take as much space on your disk. And i don't talk about the comparison for knowing which type of item you are looking for. I mean, if you want to select all the pages of your application, just compare the two request:
Multiple tables:
Select * from pages_tbl;
Single table:
Select * from item_tbl where type = 'page';
What will you gain from this design? No performance, no disk space, no readability. I really don't see a good reason for it.
So, i need to get max number of field called chat_id and after that i need to increment it by one and insert some data in that field, so the query should look something like this:
SELECT MAX(`chat_id`) FROM `messages`;
Lets say it returns me 10 now i need to insert new data
INSERT INTO `messages` SET `chat id` = 11 -- other data here....
So it would work the way i want but my question is what if betwen that time while i'm incrementing and inserting new record other user gonna do the same? than there would already be record with 11 id and it could mess my data is there a way to make sure that the right id goes where i need, btw i can't user auto increment for this.
EDIT as i said i cannot use auto increment because that table already have id field with auto increment, this id is for different porpuse, also it's not unique and it can't be unique
EDIT 2 Solved it by redoing my whole tables structure since no one gave me better ideas
Don't try to do this on your own. You've already identified one of the pitfalls of that approach. I'm not sure why you're saying you can't use auto increment here. That's really the way to go.
CREATE TABLE messages (
chat_id INT NOT NULL AUTO_INCREMENT,
....
)
If you cannot use an auto-increment primary key then you will either have to exclusively lock the table (which is generally not a good idea), or be prepared to encounter failures.
Assuming that the chat_id column is UNIQUE (which it should be from what you 're saying), you can put these two queries inside a loop. If the INSERT succeeds then everything is fine, you can break out of the loop and continue. Otherwise it means that someone else managed to snatch this particular id out of your hands, so repeat the process until successful.
At this point I have to mention that you should not actually use a totally naive approach in production code (e.g. you might want to put an upper limit in how many iterations are possible before you give up) and that this solution will not work well if there is a lot of contention for the database (it will work just fine to ensure that the occasional race does not cause you problems). You should examine your access patterns and load before deciding on this.
AUTO_INCREMENT would solve this problem. But for other similar situations this would be a great use of transactions. If you're using InnoDb engine you can use transactions to ensure that operations happen in a specific order so that your data stays consistent.
You can solve this by using MySQL's built-in uuid() function to calculate the new primary key value, instead of leaving it to the auto increment feature.
Alter your table to make messages.chat_id a char(36) and remove the AUTO_INCREMENT clause.
Then do this:
# Generate a unique primary key value before inserting.
declare new_id char(36);
select uuid() into new_id;
# Insert the new record.
insert into messages
(chat_id, ...)
values
(new_id, ...);
# Select the new record.
select *
from messages
where chat_id = new_id;
The MySQL's documentation on uuid() says:
A UUID is designed as a number that is globally unique in space and time. Two calls to UUID() are expected to generate two different values, even if these calls are performed on two separate devices not connected to each other.
Meaning it's perfectly safe to use the value generated by uuid as a primary key value.
This way you can predict what the primary key value of the new record will be before you insert it and then query by it knowing for sure that no other process has "stolen" that id from you in between the insert and the select. Which in turn removes the need for a transaction.
I have a DB consisting of 4 fields.My application will retrieve data from that db. I have one primary key(the id).I also want depending on the id, provide other data that will be organized in a new table. What is better? Create a new table and search again into it, or given the fact that I have already found the row because of the id, create a new element that will be a table. For example can I create a new element named info, and make it be to something like an array,as I want 11 rows,and 2 columns for the info. My SQL code so far is this:
CREATE TABLE people (
id INT NOT NULL AUTO_INCREMENT PRIMARY KEY ,
name VARCHAR( 100 ) NOT NULL ,
sex BOOL NOT NULL DEFAULT '1',
birthyear INT NOT NULL
)
What changes do I need to make? This table is already created.
If each row in the existing table now also needs associating with an 11x2 set of data, you're best off creating another table.
Don't try to stuff 22 items of data into a single field, it's a really bad idea.
If, however, it's always the same (22 items), you could just add 22 fields. It depends on how that data is going to be used, searched, joined on, etc.
Exactly how to do that depends on your RDBMS and your interface to it. It may be easier to create a whole new table and copy the old data across. Or the environment you have may allow you to add the columns and it do the leg work for you.
I think it would be best to create a separate new table to contain the additional data. That is primarly because you have more than one record per ID in the original table.
The records in the new table would have a foreign key peopleID field linking them to the people table.
I believe you are hinting at embedding tables. Which isn't really what MySQL is meant to do. Instead, you should do the following; Create a table like that in your example. Then create a new table that will have a column for an ID (which will be the same as that in the people table) and the other various columns. You can then do an inner join to join the two together. Additionally, if you want to reference different tables for different rows, you may want to add in a column for what 'type' it is.
Alternatively, you could use a 'No-SQL' solution like Mongo. This lets you add things dynamically. But I wouldn't suggest doing this until you have a decent grasp of a relational database.
I'm sure this is either totally impossible or really easy:
If I'm creating a table and I want one of the columns to have limited options, it seems that I use either the ENUM or SET value type. But I have to define the possible values at that moment. What if I have another table which has two columns, a primary key column and a data column, and I want the ENUM for my new table to be set to the primary key of the already existing column?
I'm sure I can just write in the values long-hand, but ideally what I need is for new values to be entered into the list table and for the table with the enum column to just accept that the value choices will include anything new added to that list table.
Is this possible without needing to manipulate the structure of the new table each time something is added to the list?
i think this link help :
http://dev.mysql.com/doc/refman/5.0/en/enum.html
have a discussion of it
in the user comments
start :
"In MySQL 5.0, you can convert an enum's values into a dynamically-defined table of values, which then provides effectively a language-neutral method to handle this kind of conversion (rather than relying on PHP, Tcl, C, C++, Java, etc. specific code).
"
he do it with stored PROCEDURE
The easiest way is to use a regular column without contraints. If you're interested in all the current values, use DISTINCT to query them:
select distinct YourColumn from YourTable
That way, you don't have any maintenance and can store whatever you like in the table.
The foreign key table you mention is also a good option. The foreign key will limit the original column. Before you do the actual insert, you run a query to expand the "enum" table:
insert into EnumTable (name)
select 'NewEnumValue'
where not exists (select * from EnumTable where name = 'NewEnumValue')
Not sure what exactly you're trying to achieve btw; limit the column, but automatically expand the choices when someone breaks the limit?