I am wondering if I can load data from flat files into multiple tables (requiring multiple inserts) using a stored procedure.
I have a stored procedure already
CREATE PROCEDURE `Insert_Hockey` (cardyear YEAR, fname VARCHAR(45), lname VARCHAR(45), brand VARCHAR(45), card_id VARCHAR(8))
BEGIN
/* Create another generic 'Item' in Items table */
INSERT INTO Items(category_id) VALUES (2);
/* Need to use the AUTO_INCREMENTED item_id from Items below so use LAST_INSERT_ID */
INSERT INTO Hockey_Cards VALUES(LAST_INSERT_ID(), cardyear, fname, lname, brand, card_id);
END
Now say I have a bunch of data in a spreadsheet about hockey cards. I can export to a tab delimitted format. I want to LOAD DATA FROM FILE but use the tab delimitted data as arguments to the stored procedure. Is something like that possible? If not, how else would you go about importing data in a simple manner in situations like this, where inserts have dependencies on previous inserts.
I am trying to create a few stored procedures / transactions (I don't know if there's a difference in my case) for situations like this in my database. There are a few situations where I use the general table -> specific table type of pattern, where specific table has a foreign key pointing to the general table. So an insert into the specifics requires a prior insert into the general, grab the AUTO_INCREMENTED primary key and use that for an insert into the 'specific' table.
I'm not sure that you can do exactly what you're asking for, but a couple of alternatives that might help:
Why not use an insert trigger? So you'd have a table for your import data, and an insert trigger on that table. The trigger should get fired for every record, so calling your SP from the trigger should get you what you want.
The syntax for INSERT DATA INFILE allows for some very complex import code. Given a the relatively simple SP code you've shown us, I would expect that you would be able to get INSERT DATA INFILE to do this for you itself, without needing an SP. The code will be more complex than the SP, but it certainly should be possible.
Related
This is my mssql UDT
create type ConditionUDT as Table
(
Name varchar(150),
PackageId int
);
This is my mssql Stored Procedure
create Procedure [dbo].[Condition_insert]
#terms_conditions ConditionUDT readonly
as
begin
insert into dbo.condition (name, p_id)
select [Name],[PackageId]
from #terms_conditions;
end
There is a workaround solution if you do not have any other choice but definitely migrate from sql server to mysql.
The closest structural predefined object that takes on many rows in mysql is an actual table. So you need 1 table per UDDT of sql server. Make sure you use a specific schema or naming conversion so you know those tables are UDDT emulations.
The idea is fill in the info, use them into the sp and then delete them. You need however to gurantee who reads what and that info are deleted after usage, consumed. So:
For any of those tables you need 2 columns, i suggest put them always first. That will be the key and the variable name. The key can be char(38) and use UUID() to get a unique identifier. It can also be int and use the connectionid() instead. Unique identifier is better however as ensures that nobody will ever use information not indented for him no matter what. The variable name will be the one used into the sql server parameter, just a string. This way:
You know what UDDT you use out of the table name.
You know the identity of your process through the key.
You know the 'variable' out of the name.
So, in your application code you:
Begin transaction.
Insert the data into the proper (UDDT emulator) tables using a key and the variable name(s)
Supply to the stored procedure the key and the variable name(s). You can use the same key for many table type parameters within the same sp call.
The stored procedure can now use that information as before from the UDDT variable using key and variable name as filters to query the proper UDDT emulated table.
Delete the data you insert
Commit
On catch, rollback.
For simplicity your sp can read the data into temp table and you do not need to change a line of code from the original sql server sp for this aspect.
Transaction into your app code will help you make sure your temporary variable data will either be deleted or never committed no matter what goes wrong.
As Larnu thought might be the case, MySQL doesn't support user defined types at all, let alone user defined table types.
You will have to make them all separate scalar parameters.
I am trying to create a near real time dwh. My first attempt is every 15 minutes load a table into my application from my DWH.
I would like to avoid all the possible problems that a near real time DWH can face. One of those problems is query an empty table that shows the value for a multiselect html tag.
To solve this I have thought the following solution but I do not know if there exists a standard to solve this kind of problem.
I create a table like this to save the possible values of the multiselect:
CREATE TABLE providers (
provider_id INT PRIMARY KEY,
provider_name VARCHAR(20) NOT NULL,
delete_flag INT NOT NULL
)
Before the insert I update the table like this:
UPDATE providers set my_flag=1
I insert rows with an ETL process like this:
INSERT INTO providers (provider_name, delete_flag) VALUES ('Provider1',0)
From my app I query the table like this:
SELECT DISTINCT provider_name FROM providers
While the app still working and selecting all providers without duplicated (The source can delete, add or update one provider, so I always have to still updated respect the source) and without showing an error because table is empty I can run this statement just after the insert statement:
DELETE FROM providers WHERE delete_flag=1
I think that this is a good solution for small tables, or big tables with few changes, but what happens when a table is big? Exist some standard to solve this kind of problems?
We can not risk user usability because we are updating data.
There are two aproaches to publich a bulk change of a dimenstion without taking a maintainance window that would interupt the queries.
The first one is simple using a transactional concept, but performs bad for large data.
DELETE the replaced dimension records
INSERT the new or changed dimension records
COMMIT;
Note that you need no logical DELETE flag as the changes are visible only after the COMMIT - so the table is never empty.
As mentioned this approach is not suitable if you have a large dimension with lot of changes. In such case you may use the EXCHANGE PARTITION feature as of MySQL 5.6
You define a temporary table with he same structure as your dimension table, that is partitioned with only one partition containing all data.
CREATE TABLE dim_tmp (
id INT NOT NULL,
col1 VARCHAR(30),
col2 VARCHAR(30)
)
PARTITION BY RANGE (id) (
PARTITION pp VALUES LESS THAN (MAXVALUE)
);
Populate the table with the complete new dimension definition and switch this temporary table with your dimension table.
ALTER TABLE dim_tmp EXCHANGE PARTITION pp WITH TABLE dim;
After this statement the data from the temporary table will be stored (published) in your dimension table (new definition) and the old state of the dimension will be stored in the temporary table.
Please check the documentation link above for constraints of this feature.
Disclaimer: I use this feature in Oracle DB and I have no experience with it in MySQL.
I'm using a single table and I have generated ProfileID (Which is a primary key for the table) in one procedure and now i want to use that same id in another procedure for inserting rest of the table fields. How to call that generated profileID . Query may not be clear but any kind of help is indeed welcome
All you need is last_insert_id()
http://dev.mysql.com/doc/refman/5.5/en/information-functions.html#function_last-insert-id
Just take a look at the examples shown there.
I would like to store all changes to my tables (in MySQL). I created a table 'audit' to track all changes of all tables in one table (so I don't have to create audit-tables for each table).
I then created triggers for each table and a stored procedure that inserts a record into the audit-table. The parameters for the stored procedure are the tablename and the primary id. Now I'm able to track the insert/update/delete dates for each record in my database.
But I also would like to trace all changes to the DATA with this procedure. For this I'd have to find a way to use the OLD and NEW records from the triggers in the stored procedure.
Anybody know how to do this?
Maybe some kind of serializing the records OLD and NEW into a string??
The only solution that I could derive was checking the OLD & NEW values of every field in my table.
BEGIN
-- record changes to table log_new
IF OLD.fieldA != NEW.fieldA
OR OLD.fieldB != NEW.fieldB
THEN
INSERT INTO log_new (
Is there a way to keep a timestamped record of every change to every column of every row in a MySQL table? This way I would never lose any data and keep a history of the transitions. Row deletion could be just setting a "deleted" column to true, but would be recoverable.
I was looking at HyperTable, an open source implementation of Google's BigTable, and this feature really wet my mouth. It would be great if could have it in MySQL, because my apps don't handle the huge amount of data that would justify deploying HyperTable. More details about how this works can be seen here.
Is there any configuration, plugin, fork or whatever that would add just this one functionality to MySQL?
I've implemented this in the past in a php model similar to what chaos described.
If you're using mysql 5, you could also accomplish this with a stored procedure that hooks into the on update and on delete events of your table.
http://dev.mysql.com/doc/refman/5.0/en/stored-routines.html
I do this in a custom framework. Each table definition also generates a Log table related many-to-one with the main table, and when the framework does any update to a row in the main table, it inserts the current state of the row into the Log table. So I have a full audit trail on the state of the table. (I have time records because all my tables have LoggedAt columns.)
No plugin, I'm afraid, more a method of doing things that needs to be baked into your whole database interaction methodology.
Create a table that stores the following info...
CREATE TABLE MyData (
ID INT IDENTITY,
DataID INT )
CREATE TABLE Data (
ID INT IDENTITY,
MyID INT,
Name VARCHAR(50),
Timestamp DATETIME DEFAULT CURRENT_TIMESTAMP)
Now create a sproc that does this...
INSERT Data (MyID, Name)
VALUES(#MyID,#Name)
UPDATE MyData SET DataID = ##IDENTITY
WHERE ID = #MyID
In general, the MyData table is just a key table. You then point it to the record in the Data table that is the most current. Whenever you need to change data, you simply call the sproc which Inserts the new data into the Data table, then updates the MyData to point to the most recent record. All if the other tables in the system would key themselves off of the MyData.ID for foreign key purposes.
This arrangement sidesteps the need for a second log table(and keeping them in sync when the schema changes), but at the cost of an extra join and some overhead when creating new records.
Do you need it to remain queryable, or will this just be for recovering from bad edits? If the latter, you could just set up a cron job to back up the actual files where MySQL stores the data and send it to a version control server.