Uploading a CSV File to MySQL Server - mysql

Question

I am using a PowerShell script to upload a CSV file to my database. The end goal is to download the most recent CSV file uploaded to our FTP (which I have completed), and then upload this CSV file to our database. These CSV files are always in the same format, and I have created the database to match this format.
I am using a script I found online as a sort of outline to help me, however it still doesn't seem to be working. Below is the script and I am hoping someone can help me figure out either a better way to complete this objective, or what I am doing wrong.
# Database variables
$sqlserver = "****"
$database = "****"
$table = "****"
$user = "****"
$pass = "****!"
# CSV variables
$csvfile = "C:\Users\Lucy\Documents\FTPFiles\vc_report_20171211.csv"
$csvdelimiter = ","
$FirstRowColumnNames = $true
################### No need to modify anything below ###################
Write-Host "Script started..."
$elapsed = [System.Diagnostics.Stopwatch]::StartNew()
[void][Reflection.Assembly]::LoadWithPartialName("System.Data")
[void][Reflection.Assembly]::LoadWithPartialName("System.Data.SqlClient")
# 50k worked fastest and kept memory usage to a minimum
$batchsize = 50000
# Build the sqlbulkcopy connection, and set the timeout to infinite
$connectionstring = "Data Source=$sqlserver;User Id=$user;Password=$pass;Initial Catalog=$database;"
$bulkcopy = New-Object Data.SqlClient.SqlBulkCopy($connectionstring, [System.Data.SqlClient.SqlBulkCopyOptions]::TableLock)
$bulkcopy.DestinationTableName = $table
$bulkcopy.bulkcopyTimeout = 0
$bulkcopy.batchsize = $batchsize
# Create the datatable, and autogenerate the columns.
$datatable = New-Object System.Data.DataTable
# Open the text file from disk
$reader = New-Object System.IO.StreamReader($csvfile)
$columns = (Get-Content $csvfile -First 1).Split($csvdelimiter)
if ($FirstRowColumnNames -eq $true) { $null = $reader.readLine() }
foreach ($column in $columns) {
$null = $datatable.Columns.Add()
}
# Read in the data, line by line, not column by column
while (($line = $reader.ReadLine()) -ne $null) {
$null = $datatable.Rows.Add($line.Split($csvdelimiter))
# Import and empty the datatable before it starts taking up too much RAM, but
# after it has enough rows to make the import efficient.
$i++;
if (($i % $batchsize) -eq 0) {
$bulkcopy.WriteToServer($datatable)
Write-Host "$i rows have been inserted in $($elapsed.Elapsed.ToString())."
$datatable.Clear()
}
}
# Add in all the remaining rows since the last clear
if ($datatable.Rows.Count -gt 0) {
$bulkcopy.WriteToServer($datatable)
$datatable.Clear()
}
# Clean Up
$reader.Close(); $reader.Dispose()
$bulkcopy.Close(); $bulkcopy.Dispose()
$datatable.Dispose()
Write-Host "Script complete. $i rows have been inserted into the database."
Write-Host "Total Elapsed Time: $($elapsed.Elapsed.ToString())"
$i = 0;
# Sometimes the Garbage Collector takes too long to clear the huge datatable.
[System.GC]::Collect()
Our database is hosted by SiteGround so I need to connect to the database. At first I was getting the following error:
Exception calling "WriteToServer" with "1" argument(s): "A network-related or
instance-specific error occurred while establishing a connection to SQL Server.
The server was not found or was not accessible. Verify that the instance name is
correct and that SQL Server is configured to allow remote connections. (provider:
Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)"
At C:\Users\Lucy\Documents\FTPFiles\upload.ps1:61 char:5
+ $bulkcopy.WriteToServer($datatable)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : SqlException
Then I added the port 3306, to the end of the $sqlserver = "servername,3306". When I did the I new get the error:
Exception calling "WriteToServer" with "1" argument(s): "A network-related or
instance-specific error occurred while establishing a connection to SQL Server.
The server was not found or was not accessible. Verify that the instance name is
correct and that SQL Server is configured to allow remote connections. (provider:
Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)"
At C:\Users\Lucy\Documents\FTPFiles\upload.ps1:63 char:5
+ $bulkcopy.WriteToServer($datatable)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : SqlException