Powershell - mySQL Query, error every other run - mysql

I am working on a powershell script that needs some input from a mySQL database. For the life of me I can't tell what I've done wrong here.
Every other time I run this script, I get an error Exception calling "Open" with "0" argument(s): "Out of sync with server"[0]. So, the first run, it will pull the expected data and dump it on my screen, then on the next run I get that error. And the cycle just repeats. Here is my full code (right now its just a test query to pull then dump the data. If it matters, the mySQL server is running MariaDB 10.3.14 on a Ubuntu 18.04 host.
$error.Clear()
$sqlQuery = get-content -path "C:\querytext.sql" -Raw
$sqlUser = "myuser"
$sqlPass = "mypass"
$sqlHost = "myserver"
$sqlDB = "dbname"
$connectionString = "server= $sqlHost;port=3306;uid=$sqlUser;pwd=$sqlPass;database=$sqlDB"
Try{
$connection = New-Object MySql.data.MySqlClient.MySqlConnection
$connection.ConnectionString = $connectionString
$connection.Open()
$command = New-Object MySql.data.MySqlClient.MySqlCommand($sqlQuery,$connection)
$dataAdapter = New-Object MySql.data.MySqlClient.MySqlDataAdapter($command)
$dataSet = New-Object System.Data.DataSet
$dataAdapter.fill($dataSet, "data") | Out-Null
$command.Dispose()
$sqlResults = $dataSet.tables["data"]
}
Catch {
Write-Host "ERROR : Unable to run query : $query `n$Error[0]"
}
$connection.close()
$sqlResults | Format-Table
$sqlResults | ForEach-Object {
write-host $_.fname
}

Might I suggest using the SQL PS module:
https://learn.microsoft.com/en-us/sql/powershell/download-sql-server-ps-module?view=sql-server-2017
That page has installation instructions and its from Microsoft. Personally, the dotnet class you are using, it works, but its relatively difficult to work with.
Connecting to a DB is much simpler with this module and you do not have to worry about micromanaging connections.
Invoke-Sqlcmd -ServerInstance $sqlHost -Query $sqlQuery -Database $sqlDB -Username $sqlUser -Password $sqlPass
This will return a PS object like every other PS cmdlet.

Related

Powershell Export-CSV from MySQL database reader fails mid-export

I'm a bit new to PowerShell, and I've got a new requirement to get Data out of a MySQL database and into an Oracle one. The strategy I chose was to output to a CSV and then import the CSV into Oracle.
I wanted to get a progress bar for the export from MySQL into CSV, so I used the data reader to achieve this. It works, and begins to export, but somewhere during the export (around record 5,000 of 4.5mil -- not consistent) it will throw an error:
Exception calling "Read" with "0" argument(s): "Fatal error encountered during data read." Exception calling "Close" with "0" argument(s): "Timeout in IO operation" Method invocation failed because [System.Management.Automation.PSObject] does not contain a method named 'op_Addition'. Exception calling "ExecuteReader" with "0" argument(s): "The CommandText property has not been properly initialized."
Applicable code block is below. I'm not sure what I'm doing wrong here, and would appreciate any feedback possible. I've been pulling my hair out on this for days.
Notes: $tableObj is a custom object with a few string fields to hold table name and SQL values. Not showing those SQL queries here, but they work.
Write-Host "[INFO]: Gathering data from MySQL select statement..."
$conn = New-Object MySql.Data.MySqlClient.MySqlConnection
$conn.ConnectionString = $MySQLConnectionString
$conn.Open()
#
# Get Count of records in table
#
$countCmd = New-Object MySql.Data.MySqlClient.MySqlCommand($tableObj.SqlCount, $conn)
$recordCount = 0
try{
$recordCount = $countCmd.ExecuteScalar()
} Catch {
Write-Host "[ERROR]: (" $tableObj.Table ") Error getting Count."
Write-Host "---" $_.Exception.Message
Exit
}
$recordCountString = $recordCount.ToString('N0')
Write-Host "[INFO]: Count for table '" $tableObj.Table "' is " $recordCountString
#
# Compose the command
#
$cmd = New-Object MySql.Data.MySqlClient.MySqlCommand($tableObj.SqlExportInit, $conn)
#
# Write to CSV using DataReader
#
Write-Host "[INFO]: Data gathered into memory. Writing data to CSV file '" $tableObj.OutFile "'"
$counter = 0 # Tracks items selected
$reader=$cmd.ExecuteReader()
$dataRows = #()
# Read all rows into a hash table
while ($reader.Read())
{
$counter++
$percent = ($counter/$recordCount)*100
$percentString = [math]::Round($percent,3)
$counterString = $counter.ToString('N0')
Write-Progress -Activity '[INFO]: CSV Export In Progress' -Status "$percentString% Complete" -CurrentOperation "($($counterString) of $($recordCountString))" -PercentComplete $percent
$row = #{}
for ($i = 0; $i -lt $reader.FieldCount; $i++)
{
$row[$reader.GetName($i)] = $reader.GetValue($i)
}
# Convert hashtable into an array of PSObjects
$dataRows += New-Object psobject -Property $row
}
$conn.Close()
$dataRows | Export-Csv $tableObj.OutFile -NoTypeInformation
EDIT: Didn't work, but I also added this line to my connection string: defaultcommandtimeout=600;connectiontimeout=25 per MySQL timeout in powershell
Using #Carl Ardiente's thinking, the query is timing out, and you have to set the timeout to something insane to fully execute. You simply have to set the timeout value for your session before you start getting data.
Write-Host "[INFO]: Gathering data from MySQL select statement..."
$conn = New-Object MySql.Data.MySqlClient.MySqlConnection
$conn.ConnectionString = $MySQLConnectionString
$conn.Open()
# Set timeout on MySql
$cmd = New-Object MySql.Data.MySqlClient.MySqlCommand("set net_write_timeout=99999; set net_read_timeout=99999", $conn)
$cmd.ExecuteNonQuery()
#
# Get Count of records in table
#
...Etc....
Not that I've found the solution, but none of the connection string changes worked. Manually setting the timeout didn't seem to help either. It seemed to be caused from too many rows returned, so I broke up the function to run in batches, and append to a CSV as it goes. This gets rid of the IO / timeout error.

Can not run Insert query against mySQL database using Powershell, but can via the PHPMyAdmin UI

Im trying to run the following query against my mySQL database :
"INSERT INTO present (name) VALUES ('fred');"
I can run this query in the UI in PHPMyAdmin and it created the expected data, however, I can not run it from my PowerShell script. I can get data from the database using my PowerShell script, but can't seem to be able to create any. Any ideas ?
EDIT
Code I use to connect to database :
function ConnectToDatabase([string]$user, [string]$pass, [string]$MySQLHost, [string]$database) {
Log "--------"
Log "Connecting to database"
# Load MySQL .NET Connector Objects
[void][system.reflection.Assembly]::LoadWithPartialName("MySql.Data")
# Open Connection
$connStr = "server=" + $MySQLHost + ";port=3306;uid=" + $user + ";pwd=" + $pass + ";database="+$database+";Pooling=FALSE"
try {
$conn = New-Object MySql.Data.MySqlClient.MySqlConnection($connStr)
$conn.Open()
} catch [System.Management.Automation.PSArgumentException] {
Log "Unable to connect to MySQL server, do you have the MySQL connector installed..?"
Log $_
#Exit
} catch {
Log "Unable to connect to MySQL server..."
Log $_.Exception.GetType().FullName
Log $_.Exception.Message
#exit
}
Log "Connected to MySQL database : $MySQLHost\$database"
Log "--------"
return $conn
}
Running the query
$conn = ConnectToDatabase $user $pass $MySQLHost $database
$query = "INSERT INTO present (name) VALUES ('fred');"
$Command = New-Object MySql.Data.MySqlClient.MySqlCommand $query, $conn
I also had this to GET data from the database but I thought it was redundant to the Insert of data
$dataAdapter = New-Object MySql.Data.MySqlClient.MySqlDataAdapter($Command)
$dataSet = New-Object System.Data.DataSet
$recordCount = $dataAdapter.Fill($dataSet, "data")
return $dataSet.Tables[0]
The code you posted creates a MySqlCommand object, but doesn't actually execute the SQL statement. Use the ExecuteNonQuery() method for that:
$conn = ConnectToDatabase $user $pass $MySQLHost $database
$query = "INSERT INTO present (name) VALUES ('fred');"
$Command = New-Object MySql.Data.MySqlClient.MySqlCommand $query, $conn
$Command.ExecuteNonQuery()

MySQL Query Exception calling "Fill" with "2" argument(s) from Powershell v4

Trying to execute a MySql query from Powershell ver 4, I get this error:
Could not run MySQL Query Exception calling "Fill" with "2" argument(s): "Fatal error encountered during command execution."
Here's the SQL I'm trying from Powershell. The SQL is in variable $updateReportQuery:
$updateReportQuery = "call count_changed_mark(#count_changed_mark);"
MySQLQuery -ConnectionString $connection -query $updateReportQuery
For reference, here is a snippet from the MySQLQuery function used with other queries no problem:
# Run MySQL Querys
Write-Verbose "Run MySQL Querys"
$command = New-Object MySql.Data.MySqlClient.MySqlCommand($query, $connection)
$dataAdapter = New-Object MySql.Data.MySqlClient.MySqlDataAdapter($command)
$dataSet = New-Object System.Data.DataSet
$recordCount = $dataAdapter.Fill($dataSet, "data")
$dataSet.Tables["data"] | Format-Table
return $dataSet.Tables[“data”] #returns response as an object
When I output the SQL from the function to write-host then copy/paste the SQL in a SQL console, it runs fine. I only get the error from Powershell.
Thanks in advance for your help.
Problem is with the user variable. Must add Allow User Variables=True; to connection string.
Example:
$connection = "Server=localhost;Uid=youruserid;Pwd=yourpassword;database=yourdatabase;Allow User Variables=True;"

Powershell: Using a SELECT from a MySQL DB as a variable in PowerCLI

I am having an issue where I need to run a query (Fetch an IP address) against my MySql database and put the result into a variable (for use when cloning a VM) However, this doesn't seem to work as the Variable turns up empty. The 'NewVMIP' variable should end up containing the IP from the MySQL database. There are no errors when accessing the database and selecting the information, the issue seems to lie when trying to use the variable when cloning the VM, it is just blank.
# SELECT IP FROM DB #
[void][system.reflection.Assembly]::LoadFrom("C:\Program Files (x86)\MySQL\Connector.NET 6.9\Assemblies\v2.0\MySQL.Data.dll")
$myconnection = New-Object MySql.Data.MySqlClient.MySqlConnection
$myconnection.ConnectionString = "server=172.30.30.90;port=3306;userid=user;password=pwd123;database=db1;pooling=false"
$myconnection.Open()
$mycommand = New-Object MySql.Data.MySqlClient.MySqlCommand
$mycommand.Connection = $myconnection
$mycommand.CommandText = "SELECT ip FROM vmip WHERE id > 1 LIMIT 1"
$myreader = $mycommand.ExecuteReader()
while($myreader.Read()){ $myreader.GetString(0) }
$myreader = $NewVMIP
# CLONE THE VM #
$TemplateVM = "Windows QA - 172.30.30.110 - TEMPLATE"
$NewVMFor = Read-Host "Please input Who This VM is For?"
$NewVMDate = Read-Host "Please input todays date"
$NewVMName = "Windows QA - $NewVMIP - $NewVMFor - $NewVMDate"
New-VM -Name $NewVMName -VM $TemplateVM -VMHost "esxfo01.jhcllp.local"
Move-VM -VM $NewVMName -Destination "Windows QA"
Start-VM -VM $NewVMName
# APPLY IP TO VM #
$username = "USER123"
$password = "PASS123"
$secstr = New-Object -TypeName System.Security.SecureString
$password.ToCharArray() | ForEach-Object {$secstr.AppendChar($_)}
$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $username, $secstr
Invoke-Command -ComputerName 172.30.30.110 -ScriptBlock {
param($NewVMIP)
$subnet = "255.255.255.0"
$gateway = "172.30.30.1"
netsh int ip set address "Local Area Connection" static "$NewVMIP" "255.255.255.0" "172.30.30.1"
} -credential $cred -ArgumentList $NewVMIP
As mentioned, it always acts as though the variable 'NewVMIP' is empty.
Thank you in advance for your help.
I think you just a have a typo here. According to your code $NewVMIP is null since you never populate it.
Should $myreader = $NewVMIP not be $NewVMIP = $myreader instead maybe? Even if I'm wrong about that fix the fact remains that I never see where you populate the $NewVMIP variable.

Run a SQL Script Against MySQL using Powershell

I have a Powershell script that backs up my MySQL DB's each night using mysqldump. This all works fine but I would like to extend the script to update a reporting db (db1) from the backup of the prod db (db2). I have written the following test script but it does not work. I have a feeling the problem is the reading of the sql file to the CommandText but I am not sure how to debug.
[system.reflection.assembly]::LoadWithPartialName("MySql.Data")
$mysql_server = "localhost"
$mysql_user = "root"
$mysql_password = "password"
write-host "Create coonection to db1"
# Connect to MySQL database 'db1'
$cn = New-Object -TypeName MySql.Data.MySqlClient.MySqlConnection
$cn.ConnectionString = "SERVER=$mysql_server;DATABASE=db1;UID=$mysql_user;PWD=$mysql_password"
$cn.Open()
write-host "Running backup script against db1"
# Run Update Script MySQL
$cm = New-Object -TypeName MySql.Data.MySqlClient.MySqlCommand
$sql = Get-Content C:\db2.sql
$cm.Connection = $cn
$cm.CommandText = $sql
$cm.ExecuteReader()
write-host "Closing Connection"
$cn.Close()
Any assistance would be appreciated. Thanks.
This line:
$sql = Get-Content C:\db2.sql
Returns an array of strings. When that gets assigned to something expecting a string then PowerShell will concatenate the array of strings into a single string using the contents of the $OFS (output field separator) variable. If this variable isn't set, the default separator is a single space. Try this instead and see if it works:
$sql = Get-Content C:\db2.sql
...
$OFS = "`r`n"
$cm.CommandText = "$sql"
Or if you're on PowerShell 2.0:
$sql = (Get-Content C:\db2.sql) -join "`r`n"