ExecuteNonQuery not deleting when it has to delete large number of rows - mysql

I would like to go over all the tables in the database and delete all rows with particular id. When the number of rows are smaller, it works fine.
in a particular table the number of rows is 900000 and this doesn't work anymore.
Here is my current code:
Using connection = New MySqlConnection(connectionString)
Try
connection.open()
If listOfTables.Count > 0 Then
For Each table As String In listOfTables
Dim sqlc As String = "DELETE FROM " & dbConnection.DbName & "." & table & " WHERE id=" & cNumber& ";"
Dim command3 As New MySqlCommand(sqlc , connection)
command3.ExecuteNonQuery()
command3.Dispose()
Next
connection.close()
End If
Catch ex As Exception
End Try
End Using

Deleting 900000 rows in once from table is not a good idea. Why don't you write a stored procedure in MYSQL which do your work. You can Try creating stored procedure which deletes records in chunks of 1000 rows in the loop and do not stop until all the rows deleted.
Try out following pseudo code:
#counter = 0
LOOP
DELETE FROM Table_name
WHERE id BETWEEN #counter AND #counter +1000
AND "Your conditions"
SET #counter = #counter + 1000
sleep 2 -- Take a sleep before next
UNTIL end of table
It uses the id column as Primary-key with auto increment

I agree with the above answer about the stored procedure loop.
However, you never mentioned if you are getting an error/exception or if it just times out.
Perhaps you should:
Catch ex As Exception
MessageBox.Show(ex.Message)
connection.close()
End Try

Related

How to use last_insert_id() with "ADODB.Connection" object correctly?

I'm having a problem in retrieving last record id from database. This code below, is the closer I can get. But still, it return record id, as 0; ,then when I execute again, it will return, record of previous execute, not the current one.
sql = "insert into program (prog_det,budget,prog_obj,outcome,target_group,awareness,engagement,issue,seq_no) value ('"&prog_title&"','"&prog_budget&"','"&prog_obj&"','"&prog_result&"','"&prog_target&"','"&prog_aware&"','"&prog_involment&"','"&prog_issues&"','99');"
sql2 = "select last_insert_id() as last_id"
set kpi_prog_conn=Server.CreateObject("ADODB.Connection")
set kpi_prog_rs=Server.CreateObject("ADODB.Recordset")
kpi_prog_conn.Open ObjConn
kpi_prog_conn.Execute(sql)
kpi_prog_conn.Open sql2,objConn,adLockPessimistic
response.write kpi_prog_rs("last_id")
Your penultimate line looks wrong
Try
kpi_prog_rs.Open sql2,kpi_prog_conn,adLockPessimistic
IS the ID you are trying to retrieve is the Primary key of the corresponding table? Try using Scope_Identity() instead of last_insert_id()
Query - SELECT SCOPE_IDENTITY() AS [LAST_IDENTITY]
It returns you the last inserted id into the table
I don't have mysql but try:
sql2 = "select last_insert_id() as last_id;"
sql = "insert into program (prog_det,budget,prog_obj,outcome,target_group,awareness,engagement,issue,seq_no) value ('"&prog_title&"','"&prog_budget&"','"&prog_obj&"','"&prog_result&"','"&prog_target&"','"&prog_aware&"','"&prog_involment&"','"&prog_issues&"','99');" & sql2
set kpi_prog_conn=Server.CreateObject("ADODB.Connection")
kpi_prog_conn.Open ObjConn
set kpi_prog_rs = kpi_prog_conn.Execute(sql)
anotherRecordset = kpi_prog_rs.NextRecordset
response.write anotherRecordset("last_id")

Returned Recordset is closed (mysql DB accessed via ODBC in VBA)

Long time viewer first time poster. I'm working on a database application with a front-end in Excel. Hence I am using VBA to access a MySQL database. The driver I'm using is the ODBC driver (Windows) and I have a valid connection to the database as lots of other queries function well.
What I'm trying to do is return the results from a series of SQL statements rather than trying to combine it all into one massive statement (which would probably work but be difficult to maintain/understand). I have added the flag FLAG_MULTI_STATEMENTS = 67108864 to the connection string which suppressed driver syntax errors.
But now when I run the following:
queryDB.Open SQLquery, conn
The recordset (queryDB) remains closed with no apparent error. The sql statement can be found here.
I may be generating errors that aren't being returned to VBA so any help here would be much appreciated.
NOTE: The sql statements work as I can paste that statement into phpMyAdmin and it returns the correct (non-empty) results. I don't know if the statements specifically are the problem but perhaps the use of CREATE TEMPORARY TABLE ... or the use of multiple statements in general.
Also I guess that the driver may be trying to return a result for each sql statement and VBA is only getting the first or something...
EDIT: the sql statement for future reference.
CREATE TEMPORARY TABLE tmpOrders AS
SELECT
o.customerName,
SUM(o.Sales) AS Sales,
SUM(TotalFobCost + TotalLandedCost + TotalLocalCost + TotalCmtCost) AS TotalCost,
YEAR(o.deliveryDate) AS YEAR,
MONTH(o.deliveryDate) AS MONTH
FROM devere_costing.orders_fixed_extras AS o
WHERE o.orderApproved = TRUE
AND o.orderCanceled = FALSE
AND o.deliveryDate BETWEEN '2014-01-01' AND '2014-03-31'
GROUP BY customerName, YEAR, MONTH
ORDER BY YEAR ASC, MONTH ASC, customerName ASC;
CREATE TEMPORARY TABLE tmpProj AS
SELECT p.customerName,
IF(p.MONTH > 9, p.YEAR, p.YEAR - 1) AS TrueYear,
1 + ((p.MONTH + 2) MOD 12) AS TrueMonth,
SUM(p.actualSalesInvoiced) AS salesInvoiced,
SUM(p.budget) AS budget
FROM devere_costing.sales_projection_data AS p
GROUP BY p.customerName, p.YEAR, p.MONTH
HAVING TrueYear BETWEEN YEAR('2014-01-01') AND YEAR('2014-03-31')
AND TrueMonth BETWEEN MONTH('2014-01-01') AND MONTH('2014-03-31');
CREATE TEMPORARY TABLE tmpLeft AS
SELECT
IF(o.customerName IS NULL, p.customerName, o.customerName) AS customerName,
p.budget AS TotalBudget,
o.Sales AS Sales,
p.salesInvoiced,
0 AS varianceToBudget,
o.TotalCost,
0 AS directMargin,
0 AS directMarginPercent,
IF(o.YEAR IS NULL, p.TrueYear, o.YEAR) AS YEAR,
IF(o.MONTH IS NULL, p.TrueMonth, o.MONTH) AS MONTH
FROM tmpOrders AS o
LEFT JOIN tmpProj AS p
ON (o.customerName = p.customerName
AND o.YEAR = p.TrueYear
AND o.MONTH = p.TrueMonth);
CREATE TEMPORARY TABLE tmpRight AS
SELECT
IF(o.customerName IS NULL, p.customerName, o.customerName) AS customerName,
p.budget AS TotalBudget,
o.Sales AS Sales,
p.salesInvoiced,
0 AS varianceToBudget,
o.TotalCost,
0 AS directMargin,
0 AS directMarginPercent,
IF(o.YEAR IS NULL, p.TrueYear, o.YEAR) AS YEAR,
IF(o.MONTH IS NULL, p.TrueMonth, o.MONTH) AS MONTH
FROM tmpOrders AS o
RIGHT JOIN tmpProj AS p
ON (o.customerName = p.customerName
AND o.YEAR = p.TrueYear
AND o.MONTH = p.TrueMonth);
(SELECT * FROM tmpLeft) UNION DISTINCT (SELECT * FROM tmpRight);
I have answered my own question!
The secret lies here:
So I was right in that there was more than one recordset returned. I just had to iterate through them to find the data that I want. The collection isn't indexed so you have to search through each one. In my case every sql statement does not return a recordset (that's why my recordset remained closed when I tried to open it). The only exception is the last sql statement which returns records. My loop looks like:
Dim rs As ADODB.Recordset
Set rs = queryDB(Sql)
' Loop through returned recordsets to find the data
Do
If Not rs Is Nothing Then
If rs.State = adStateOpen Then
' we have an open recordset. This means that the final select statement
' has returned this data.
Exit Do
Else
' Otherwise iterate through to the next recordset
Set rs = rs.NextRecordset
End If
Else
MsgBox "No recordset returned by sql statement"
GoTo ExitCode
End If
Loop
Answer copied from the question body:
I have answered my own question!
The secret lies here:
So I was right in that there was more than one recordset returned. I just had to iterate through them to find the data that I want. The collection isn't indexed so you have to search through each one. In my case every sql statement does not return a recordset (that's why my recordset remained closed when I tried to open it). The only exception is the last sql statement which returns records. My loop looks like:
Dim rs As ADODB.Recordset
Set rs = queryDB(Sql)
' Loop through returned recordsets to find the data
Do
If Not rs Is Nothing Then
If rs.State = adStateOpen Then
' we have an open recordset. This means that the final select statement
' has returned this data.
Exit Do
Else
' Otherwise iterate through to the next recordset
Set rs = rs.NextRecordset
End If
Else
MsgBox "No recordset returned by sql statement"
GoTo ExitCode
End If
Loop

vb.net how to create a variable to count entries going in as inserts and updates

I have a mysql database and a gridview. Now I want to count my inserts and updates. It has been suggested to set up a variables to represent each element e.g. nUpdates and nInserts, but I am unsure on how to do this? Here is the code I am using
For i = 0 To DataGridView1.RowCount - 1
Using dbSQL_cmd As New MySqlCommand()
student_id = DataGridView1.Item(i, 0).Value.ToString
name = DataGridView1.Item(i, 1).Value.ToString
age = DataGridView1.Item(i, 2).Value.ToString
adress = DataGridView1.Item(i, 3).Value.ToString.ToLower
dbSQL_query = "INSERT INTO student VALUES (?id, ?name, ?age, ?adress) " +
" on duplicate key update name=(name=?name, age=?age, adress=?adress)"
With dbSQL_cmd
.Parameters.AddWithValue("?id", student_id)
.Parameters.AddWithValue("?name", name)
.Parameters.AddWithValue("?age", age)
.Parameters.AddWithValue("?adress", adress)
' .Parameters.AddWithValue("?status", ?????)
.CommandText = dbSQL_query
.Connection = SQLConnection
.CommandType = CommandType.Text
End With
dbSQL_cmd.ExecuteNonQuery()
End Using
Next
First, check the number of rows in the table before and after your insert/update loop. The difference between them is the number of rows you inserted.
You know how many items there are in your gridview. From your code, each one is either an insert or an update, so subtract the number of inserts you did (from that first step) from the number of items, and that is the number of rows you updated.

VB.net adding parent and child records to MySQL db

I was asked to create a program that inserts records into one parent table and multiple child tables. My question is, how do I know what the PK is for the parent table, so that I may add it as a FK in the child? The PK for the parent is an auto number. As I stated in my title, I'm using VB.net, mySQL, through an ODBC connection. I have to do this through the code and cannot use stored procedures. Any suggestions?
thanks
my transaction looks like this:
Dim cmdText As String = "INSERT INTO candidate(first_name, last_name, phone1, phone2, email1, city, " _
& " state, country, zip,primary_contact_id ) VALUES (?,?, ?, ?,?,?, ?,?,?,?)"
If conn.State = ConnectionState.Closed Then
conn.Open()
End If
Dim SqlStatus As Integer
Dim trans As Odbc.OdbcTransaction = conn.BeginTransaction(IsolationLevel.ReadCommitted)
Dim cmd As OdbcCommand = New OdbcCommand(cmdText, conn, trans)
Try
cmd.Parameters.Clear()
cmd.CommandType = CommandType.Text 'The default is CommandType.Text
With cmd.Parameters
.Add("#first_name", OdbcType.VarChar).Value = fName
.Add("#last_name", OdbcType.VarChar).Value = lName
.Add("#phone1", OdbcType.VarChar).Value = phone
.Add("#phone2", OdbcType.VarChar).Value = mobilePhone
.Add("#email1", OdbcType.VarChar).Value = email
.Add("#city", OdbcType.VarChar).Value = city
.Add("#state", OdbcType.VarChar).Value = state
.Add("#country", OdbcType.VarChar).Value = country
.Add("#zip", OdbcType.VarChar).Value = zip
.Add("#primary_contact_id", OdbcType.Int).Value = getContactFK
End With
SqlStatus = cmd.ExecuteNonQuery
If Not SqlStatus = 0 Then
trans.Commit()
Me.Close()
Else
MsgBox("Not Updated")
End If
Catch ex As Exception
MsgBox(ex.Message)
Finally
cmd.Dispose()
trans.Dispose()
End Try
I'm still working on the code, so not sure if it works just yet
jason
Take a look at How to Get the Unique ID for the Last Inserted Row
Since you're going through ODBC and cannot use a stored proc you will have to execute two SQL statements together (as a batch). First your insert and then SELECT LAST_INSERT_ID()
It should look something like:
INSERT INTO ... ;
SELECT LAST_INSERT_ID();
Since you're expecting a result you need to execute from your client code as a SELECT statement. And since this is a batch operation with an insert you should also consider using a transaction.
You can use
"; select last_insert_id()"
At the end of your insert for the parent table. And then use
Dim id as Integer = cint(command.ExecuteScalar())
To get the resulting key to use in the child inserts

Insert multiple rows using one insert statement in Access 2010

I want to insert multiple values into an Access 2010 table, but I can't seem to find a way.
MySQL had a nice way:
INSERT INTO Production.UnitMeasure
VALUES
(N'FT2', N'Square Feet ', '20080923'),
(N'Y', N'Yards', '20080923'),
(N'Y3', N'Cubic Yards', '20080923');
Can something like this be done in SQL Server too?
As marc_s has pointed out, for SQL Server 2008 and later, you can just use table value constructors. For previous versions, you can use insert and select...union all, e.g.:
INSERT INTO Production.UnitMeasure
SELECT N'FT2',N'Square Feet ','20080923' union all
SELECT N'Y', N'Yards', '20080923' union all
SELECT N'Y3', N'Cubic Yards', '20080923'
(Specific documentation on Table Value Constructors in SQL Server. I can't find specific separate documentation on row value constructors, but that's what they are)
Use this confirm working query:
INSERT INTO Product (Code,Name,IsActive,CreatedById,CreatedDate )
SELECT * FROM
(
SELECT '10001000' AS Code,
'Blackburn sunglasses' AS Name,
1 AS IsActive,
1 AS CreatedById,
'2/20/2015 12:23:00 AM' AS CreatedDate
FROM Product
UNION
SELECT '10005200' AS Code,
'30 panel football' AS Name,
1 AS IsActive,
1 AS CreatedById,
'2/20/2015 12:23:09 AM' AS CreatedDate
FROM Product
) ;
For SQL-Server: Yes, and it can exactly like you write. Just be certain that the column values are in the same order as they appear in the table. Also: you must supply a value for each existing column.
For Access 2010: No. At least not by hardcoded values in the sql, but only by selecting multiple records from a table (in the same or in another database). See also the link in the answer of Khepri.
SQL Server definitely allows this: EDIT: [As of SQL Server 2008, thank you Marc_s]
INSERT INTO [Table]
([COL1], [COL2])
VALUES
('1#1.com', 1),
('2#2.com', 2)
As for the Access requirement, I'm no access guru but I found this MSDN documentation that shows how to do multiple inserts at once.
INSERT INTO target [(field1[, field2[, …]])] [IN externaldatabase]
SELECT [source.]field1[, field2[, …] FROM tableexpression
Doing some cursory reading beyond this, you can use a "dummy" from table if all of your values are known ahead of time as in your example.
Create a table called OneRow with a single integer column. Insert one row.
Then:
INSERT INTO Production.UnitMeasure
SELECT 'FT2', 'Square Feet ', '20080923' FROM OneRow
UNION ALL SELECT 'Y', 'Yards', '20080923' FROM OneRow
UNION ALL SELECT 'Y3', 'Cubic Yards', '20080923' FROM OneRow
Your exact syntax works on SQL Server 2008. For earlier use my above query without the FROM clauses and no helper table.
I know its late to answer, but there are a couple of methods which are still useful today, (not mentioned here).
There are two general methods.
Loop through using a VBA script with the 'Docmd.RunSQL' statement. - This is generally quite slow especially as the number of rows increases but easy to understand.
Copy your 'data array' into an excel worksheet and then use a database query link to the excel file (my preferred method) - Benefits of this is it is almost as fast as if the table was already in the database and not necessarily slowed down by the number of records as the previous method - slightly slower than the above when you have a small number of records however
The Docmd Method:
Option Compare Database
Option Base 1
'------- method created by Syed Noshahi --------
'https://www.linkedin.com/in/syed-n-928b2490/
Sub DoCmdRoutine()
Dim arr() As Variant ' create an unsized array ready for data
'--------------For the purposes of the Example, put some data in the array----------
ReDim arr(5, 5)
For i = LBound(arr) To UBound(arr)
For j = LBound(arr) To UBound(arr)
arr(i, j) = i * j
If i = LBound(arr) Then arr(i, j) = "col_" & arr(i, j) 'Append "col_" before the column names
Next
Next
'-------------Importing the data to a table in MS ACCESS----------------------------
sSQL = "INSERT INTO [#TableTemp] " ' change table to actual table name
DoCmd.SetWarnings False 'turn warnings off - no popups!
For i = 2 To UBound(arr) 'start at 2 assuming that the first columns are headers
vals = "" 'go through each column and copy the data to SQL string format
'replace any single quote with double quotes so it does not error importing into SQL
For j = 1 To UBound(arr, 2)
If IsDate(arr(i, j)) Then 'if a date, convert to a number and let access re-covert to date (best chance at success)
vals = vals & " cdate('" & CDbl(arr(i, j)) & "'),"
ElseIf IsNumeric(arr(i, j)) Then 'if a number put through as a number
vals = vals & arr(i, j) & ","
Else 'otherwise treat as a text value
vals = vals & Replace(arr(i, j), "'", "''", , , 1) & "',"
End If
Next
vals = " VALUES(" & Left(vals, Len(vals) - 1) & ")" 'put in correct sql format
DoCmd.RunSQL sSQL & vals 'Run the SQL statement and import into the database
Next
DoCmd.SetWarnings True 'turn warnings on
End Sub
The Excel Link Method:
Option Compare Database
Option Base 1
'------- method created by Syed Noshahi --------
'https://www.linkedin.com/in/syed-n-928b2490/
Sub ExcelLinkRoutine()
Dim arr() As Variant ' create an unsized array ready for data
Dim oExcel As Object ' Excel instance - late binding
' works with access 2007+, access 2003 has a different SQL syntax
'--------------For the purposes of the Example, put some data in the array----------
ReDim arr(5, 5)
For i = LBound(arr) To UBound(arr)
For j = LBound(arr) To UBound(arr)
arr(i, j) = i * j
If i = LBound(arr) Then arr(i, j) = "col_" & arr(i, j) 'Append "col_" before the column names
Next
Next
'----------------------------output the array to an excel file ---------------------
Set oExcel = CreateObject("Excel.Application")
oExcel.Workbooks.Add 1
Set wb = oExcel.ActiveWorkbook
'network file path & normal path examples below
'fileNameWithExtention = "\\networkpath\location\example999.xlsb" ' note that xlsb file format must be used
' other formats can be used by changing 'xlExcel12'
' ONLY change the path not the FILE NAME
fileNameWithExtention = "C:\Users\public\documents\example999.xlsb" ' same as above
checkFileExists = Dir(fileNameWithExtention)
If Len(checkFileExists) > 0 Then
'only delete the file if its called example999!
If checkFileExists = "example999.xlsb" Then
Kill fileNameWithExtention
End If
End If
With wb
.Sheets(1).Cells(1, 1).Resize(UBound(arr), UBound(arr, 2)).Value2 = arr()
.SaveAs fileNameWithExtention, 50 ' 50 means xlExcel12
.Close False
End With
Set wb = Nothing
Set oExcel = Nothing
'------------ Importing the data to a table in MS ACCESS-----------------------------
'NOTE1: The saved down excelfile MUST be named Sheet1
'NOTE2: if the file path contains special characters such as ,-'
' you may need find the correct way to input (or remove the special chars)
sSQL = "SELECT T1.* INTO [#TableTemp] " ' change table to actual table name
sSQL = sSQL & " FROM [Excel 12.0;HDR=YES;IMEX=2;ACCDB=YES;DATABASE=" & fileNameWithExtention & "].[Sheet1$] as T1;" ' linked table format
DoCmd.SetWarnings False 'turn warnings off - no popups!
DoCmd.RunSQL sSQL 'Run the SQL statement and import into the database
DoCmd.SetWarnings True 'turn warnings on
End Sub
OUTPUT:
Col_1
Col_2
Col_3
Col_4
Col_5
2
4
6
8
10
3
6
9
12
15
4
8
12
16
20
5
10
15
20
25
MS Access does not allow multiple insert from same sql window. If you want to insert, say 10 rows in table, say movie (mid, mname, mdirector,....), you would need to
open the sql windows,
type the 1st stmt, execute 1st stmt, delete 1st stmt
type the 2nd stmt, execute 2nd stmt, delete 2nd stmt
type the 3rd stmt, execute 3rd stmt, delete 3rd stmt ......
Very boring.
Instead you could import the lines from excel by doing:
Right-click on the table name that you have already created
Import from Excel (Import dialog box is opened)
Browse to the excel file containing the records to be imported in the table
Click on "Append a copy of the records to the table:"
Select the required table (in this example movie)
Click on "OK"
Select the worksheet that contains the data in the spreadsheet
Click on Finish
The whole dataset in the excel has been loaded in the table "MOVIE"
I know I'm a bit late to the game, but I was wanting to do the exact same thing you guys mentioned in your example. I was trying to insert a new list of default rows into a table/list using Access because I've had a lot of SQL experience, I was trying to do it the same way, however as you posters have noted, it's not possible to do the Unions and such.
However I just wanted to post a reply up here because in the case where you're manually typing in the values (string default values in this case) you can simply open Access in datasheet view, copy your data from Excel and just paste it into your Access table (or in my case, SharePoint list). You'll need to make sure you're columns are lined up exactly, but if you were going to manually type in your "insert" sql statements, just putting that info into an Excel spreadsheet shouldn't be a big deal.
In my case, my table/list only had a single column as a lookup, so I just copied the column from notepad++ and pasted it into the datasheet view.
Good luck everyone!
Check following,
INSERT INTO [Customer] ([Id],[FirstName],[LastName],[City],[Country],[Phone])VALUES(1,'Maria','Anders','Berlin','Germany','030-0074321')
INSERT INTO [Customer] ([Id],[FirstName],[LastName],[City],[Country],[Phone])VALUES(2,'Ana','Trujillo','México D.F.','Mexico','(5) 555-4729')