I'm a newbie to sqlite3 and I'm trying to play around with some data that I got my hands on.
I'm currently running into the problem in which I cannot even create my table in my shell. I'm trying to run
.mode csv
.import filename tab1
But I'm getting this error
CREATE TABLE tab1(...) failed: duplicate column name: B-
Since I'm importing a .csv file, I thought that perhaps I do have a duplicate in my column names but they all seem to be distinct to me. Can someone point me in the right direction? I would really appreciate it :-)
Here is a short excerpt of my .csv file:
SID,Name,Cohort,Email,Gender,Ethnicity,Major,Grades,More grades,Last grades,Not sure,,
24239361,name1,Cohort 1-Fall 2013,name1#email.com,Female,Chinese,CS,B,C+,B-,B-,,
24474707,name2,Cohort 1-Fall 2013,name2#email.com,Male,Chinese,CS,B,B+,B-,B-,,
24266062,name3,Cohort 1-Fall 2013,name3#email.com,Male,White,CS,B-,B-,C,B ,,
UPDATE:
Edited my csv file to now only be:
SID,Name,Cohort
24239361,Name1,Cohort 1-Fall 2013
24474707,Name2,Cohort 1-Fall 2013
24266062,Name3,Cohort 1-Fall 2013
22181134,Name4,Cohort 1-Fall 2013
And when I import it and do .schema, I get this:
CREATE TABLE foo(
"SID" TEXT,
"Name" TEXT,
24239361" TEXT,
"Name1" TEXT,
24474707" TEXT,l 2013
"Name2" TEXT,
24266062" TEXT,l 2013
"Name3" TEXT,
22181134" TEXT,l 2013
"Name4" TEXT,
24527147" TEXT,l 2013
This is really strange because I'm skipping over the header column "Cohort" and instead reading all of my following lines to columns
I can't reproduce your issue exactly, but when I try with your sample data I get the following not-quite-identical error:
CREATE TABLE tab1(...) failed: duplicate column name:
And the reason for that is those two commas at the end of your first line (representing the columns of the new table). SQLite tries to make two columns with blank names and fails on the second. The solution is to either remove those commas from every line of the file, or to give those fields valid column names.
I wasn't able to specifically reproduce your issue, but it looks like SQLite is not seeing that first line for some reason or another and is trying to set the columns up based on one of the next lines (which have multiple fields with value B-, causing the same issue as above). You'll need to track down why that's happening. Alternatively, you can create the table first, and remove the column heading line from the file before you import it:
CREATE TABLE tab1 (SID INTEGER, Name TEXT, ...);
.mode csv
.import filename tab1
The sqlite3 tool expects the new-line characters in CSV files to be CR+LF (as specified in RFC 4180).
Your file looks like this:
00000000: 53 49 44 2c 4e 61 6d 65 2c 43 6f 68 6f 72 74 0d SID,Name,Cohort.
00000010: 32 34 32 33 39 33 36 31 2c 41 6c 6c 69 73 6f 6e 24239361,Xxxxxxx
00000020: 20 43 6f 72 69 6e 6e 65 20 59 65 65 2c 43 6f 68 Xxxxxxx Xxx,Coh
00000030: 6f 72 74 20 31 2d 46 61 6c 6c 20 32 30 31 33 0d ort 1-Fall 2013.
00000040: 32 34 34 37 34 37 30 37 2c 41 6e 74 68 6f 6e 79 24474707,Xxxxxxx
...
This file has Mac line endings (only CR), which would be valid for a normal text file.
You can manually change the row separator after setting CSV mode:
.mode csv
.sep , \r
I'm using MySQL UPDATE CASE WHEN ELSE statement to update an ordered list on my web application. I have a html table with the sortable drag-and-drop plugin, and on the stop event I hit a servlet that persists the new order to the MySQL Table.
The thing is that from time to time MySQL hangs when the user reorders the table very quickly. Like in the example below, there were 4 statements issued in the same second, and then it hanged.
22/01/2014 14:03:25 UPDATE `edited_table_name` SET `order` = CASE NR_DMD WHEN 92059 THEN 22 WHEN 124376 THEN 23 WHEN 124163 THEN 24 WHEN 127019 THEN 25 WHEN 123816 THEN 26 WHEN 124348 THEN 27 WHEN 127017 THEN 28 WHEN 126764 THEN 29 WHEN 126637 THEN 30 WHEN 122170 THEN 31 WHEN 125630 THEN 32 WHEN 125515 THEN 33 WHEN 124487 THEN 34 WHEN 124374 THEN 35 WHEN 124343 THEN 36 WHEN 124327 THEN 37 WHEN 126838 THEN 38 WHEN 126720 THEN 39 WHEN 126718 THEN 40 WHEN 123510 THEN 41 WHEN 122966 THEN 42 WHEN 124385 THEN 43 WHEN 122754 THEN 44 WHEN 124312 THEN 45 WHEN 122311 THEN 46 WHEN 121577 THEN 47 WHEN 121028 THEN 48 WHEN 123354 THEN 49 WHEN 121682 THEN 50 WHEN 120993 THEN 51 WHEN 120860 THEN 52 WHEN 120750 THEN 53 WHEN 120460 THEN 54 WHEN 120459 THEN 55 WHEN 120397 THEN 56 WHEN 119178 THEN 57 WHEN 111723 THEN 58 WHEN 127644 THEN 59 WHEN 127610 THEN 60 WHEN 127609 THEN 61 WHEN 127542 THEN 62 WHEN 125799 THEN 63 ELSE order END WHERE NR_DMD IN(92059,124376,124163,127019,123816,124348,127017,126764,126637,122170,125630,125515,124487,124374,124343,124327,126838,126720,126718,123510,122966,124385,122754,124312,122311,121577,121028,123354,121682,120993,120860,120750,120460,120459,120397,119178,111723,127644,127610,127609,127542,125799)
22/01/2014 14:03:25 UPDATE `edited_table_name` SET `order` = CASE NR_DMD WHEN 121419 THEN 10 WHEN 110378 THEN 11 WHEN 124376 THEN 12 WHEN 124163 THEN 13 WHEN 127019 THEN 14 WHEN 123816 THEN 15 WHEN 124348 THEN 16 WHEN 127017 THEN 17 WHEN 126764 THEN 18 WHEN 126637 THEN 19 WHEN 122170 THEN 20 WHEN 125630 THEN 21 WHEN 125515 THEN 22 WHEN 124487 THEN 23 WHEN 124374 THEN 24 WHEN 124343 THEN 25 WHEN 124327 THEN 26 WHEN 126838 THEN 27 WHEN 126720 THEN 28 WHEN 126718 THEN 29 WHEN 123510 THEN 30 WHEN 122966 THEN 31 WHEN 124385 THEN 32 WHEN 122754 THEN 33 WHEN 124312 THEN 34 WHEN 122311 THEN 35 WHEN 121577 THEN 36 ELSE order END WHERE NR_DMD IN(121419,110378,124376,124163,127019,123816,124348,127017,126764,126637,122170,125630,125515,124487,124374,124343,124327,126838,126720,126718,123510,122966,124385,122754,124312,122311,121577)
22/01/2014 14:03:25 UPDATE `edited_table_name` SET `order` = CASE NR_DMD WHEN 127585 THEN 21 WHEN 124376 THEN 22 WHEN 124163 THEN 23 WHEN 127019 THEN 24 WHEN 123816 THEN 25 WHEN 124348 THEN 26 WHEN 127017 THEN 27 WHEN 126764 THEN 28 WHEN 126637 THEN 29 WHEN 122170 THEN 30 WHEN 125630 THEN 31 WHEN 125515 THEN 32 WHEN 124487 THEN 33 WHEN 124374 THEN 34 WHEN 124343 THEN 35 WHEN 124327 THEN 36 WHEN 126838 THEN 37 WHEN 126720 THEN 38 WHEN 126718 THEN 39 WHEN 123510 THEN 40 WHEN 122966 THEN 41 WHEN 124385 THEN 42 WHEN 122754 THEN 43 WHEN 124312 THEN 44 WHEN 122311 THEN 45 WHEN 121577 THEN 46 WHEN 121028 THEN 47 WHEN 123354 THEN 48 WHEN 121682 THEN 49 WHEN 120993 THEN 50 WHEN 120860 THEN 51 WHEN 120750 THEN 52 WHEN 120460 THEN 53 WHEN 120459 THEN 54 WHEN 120397 THEN 55 WHEN 119178 THEN 56 WHEN 111723 THEN 57 WHEN 127644 THEN 58 WHEN 127610 THEN 59 WHEN 127609 THEN 60 ELSE order END WHERE NR_DMD IN(127585,124376,124163,127019,123816,124348,127017,126764,126637,122170,125630,125515,124487,124374,124343,124327,126838,126720,126718,123510,122966,124385,122754,124312,122311,121577,121028,123354,121682,120993,120860,120750,120460,120459,120397,119178,111723,127644,127610,127609)
22/01/2014 14:03:25 UPDATE `edited_table_name` SET `order` = CASE NR_DMD WHEN 127638 THEN 19 WHEN 127592 THEN 20 WHEN 124376 THEN 21 WHEN 124163 THEN 22 WHEN 127019 THEN 23 WHEN 123816 THEN 24 WHEN 124348 THEN 25 WHEN 127017 THEN 26 WHEN 126764 THEN 27 WHEN 126637 THEN 28 WHEN 122170 THEN 29 WHEN 125630 THEN 30 WHEN 125515 THEN 31 WHEN 124487 THEN 32 WHEN 124374 THEN 33 WHEN 124343 THEN 34 WHEN 124327 THEN 35 WHEN 126838 THEN 36 WHEN 126720 THEN 37 WHEN 126718 THEN 38 WHEN 123510 THEN 39 WHEN 122966 THEN 40 WHEN 124385 THEN 41 WHEN 122754 THEN 42 WHEN 124312 THEN 43 WHEN 122311 THEN 44 WHEN 121577 THEN 45 WHEN 121028 THEN 46 WHEN 123354 THEN 47 WHEN 121682 THEN 48 WHEN 120993 THEN 49 WHEN 120860 THEN 50 WHEN 120750 THEN 51 WHEN 120460 THEN 52 WHEN 120459 THEN 53 WHEN 120397 THEN 54 WHEN 119178 THEN 55 WHEN 111723 THEN 56 WHEN 127644 THEN 57 ELSE order END WHERE NR_DMD IN(127638,127592,124376,124163,127019,123816,124348,127017,126764,126637,122170,125630,125515,124487,124374,124343,124327,126838,126720,126718,123510,122966,124385,122754,124312,122311,121577,121028,123354,121682,120993,120860,120750,120460,120459,120397,119178,111723,127644)
I thought MySQL would deal with the simultaneous hits on the same table. But it looks like not. Perhaps it's some issue with table locks.
If you can give me any advice in any configuration or technique or even another better way to persist the reordered list, I'd be very greateful.
Thanks,
You could try to use the following session parameters before your update statement in your servlet.
"SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED"
That would allow that if two statements occurs almost instantanously, it would take the latest one even if the other process had not commited it to the server.
For sure it is better to send changes only if the end-user did not did any drag & drop for more than X seconds (ie: 2) and send only 1 request. That would solve the problem also.
Would you consider using something in your client side code so that you don't submit multiple AJAX requests simultaneously such as setTimeout when the list is reordered.
Perhaps something like this:
http://benalman.com/code/projects/jquery-message-queuing/examples/ajax/
I would suggest that you maintain a check for a period of inactivity and when there is a good amount of inactivity, send latest update (that has the cumulative update) to the server.
During the execution of your update query InnoDB locks every row that it traverses to find the rows it is modifying. So to reduce the number of locks you can split your query into two separate queries:
Get PRIMARY_KEY_LIST, suppose that the primary key of edited_table_name is id
SELECT id FROM edited_table_name WHERE NR_DMD IN(...)
Update basing on primary key list
UPDATE edited_table_name SET order = CASE... WHERE id IN (PRIMARY_KEY_LIST)
This will reduce the number of locks because it reduces the number of rows that the engine has to traverse.
Another option is to put an index on NR_DMD columns
I might have a handler to throttle the hits that come to the servlet probably in this scenario :
Rather than instantly applying every new order request that hits the servlet , I would store the info/details in a table say new_order_requests_table .
I would have a flag which is used/updated by this handler accordingly with status of itself , say : Idle , Under Progress , etc .
Whenever the above flag shows as Idle , I would consider to apply the latest row from the new_order_requests_table and probably delete all earlier rows from it .
For multiple users , you would need to modify the above accordingly .
Further , when it is needed to update many rows in a MySQL table in a single statement , sometimes it would be better with MyISAM which does table level locking as opposed to InnoDB which does row level locking . This is because InnoDB might be consuming considerable duration just to lock and unlock those many rows .You could try switching between MyISAM and InnoDB to find the better suitable engine for this table in your application .
I think you can do the following:
CREATE TEMPORARY TABLE Table1 (
NR_DMD int NOT NULL);
INSERT INTO Table1 (NR_DMD)
VALUES
(92059),(124376),(124163),(127019),(123816),(124348),(127017),(126764),(126637),
(122170),(125630),(125515),(124487),(124374),(124343),(124327),(126838),(126720),
(126718),(123510),(122966),(124385),(122754),(124312),(122311),(121577),(121028),
(123354),(121682),(120993),(120860),(120750),(120460),(120459),(120397),(119178),
(111723),(127644),(127610),(127609),(127542),(125799);
select * from Table1
22/01/2014 14:03:25
UPDATE `edited_table_name
` SET `order` = CASE NR_DMD WHEN 92059 THEN 22 WHEN 124376 THEN 23 WHEN 124163 THEN 24
WHEN 127019 THEN 25 WHEN 123816 THEN 26 WHEN 124348 THEN 27 WHEN 127017 THEN 28
WHEN 126764 THEN 29 WHEN 126637 THEN 30 WHEN 122170 THEN 31
WHEN 125630 THEN 32 WHEN 125515 THEN 33 WHEN 124487 THEN 34
WHEN 124374 THEN 35 WHEN 124343 THEN 36 WHEN 124327 THEN 37
WHEN 126838 THEN 38 WHEN 126720 THEN 39 WHEN 126718 THEN 40
WHEN 123510 THEN 41 WHEN 122966 THEN 42 WHEN 124385 THEN 43
WHEN 122754 THEN 44 WHEN 124312 THEN 45 WHEN 122311 THEN 46
WHEN 121577 THEN 47 WHEN 121028 THEN 48 WHEN 123354 THEN 49
WHEN 121682 THEN 50 WHEN 120993 THEN 51 WHEN 120860 THEN 52
WHEN 120750 THEN 53 WHEN 120460 THEN 54 WHEN 120459 THEN 55
WHEN 120397 THEN 56 WHEN 119178 THEN 57 WHEN 111723 THEN 58
WHEN 127644 THEN 59 WHEN 127610 THEN 60 WHEN 127609 THEN 61
WHEN 127542 THEN 62 WHEN 125799 THEN 63 ELSE order END
WHERE NR_DMD IN(Select NR_DMD from Table1);
Summary:
1)Create temp table and insert the values for first update.
2)
UPDATE tablename SET ColName =CASE STATEMENT
Where NR_DMD in ( SELECT NR_DMD FROM temp table)
3) Do the same for 2 other update statement.
I think this will help you to better perform.