I have a Powershell script that backs up my MySQL DB's each night using mysqldump. This all works fine but I would like to extend the script to update a reporting db (db1) from the backup of the prod db (db2). I have written the following test script but it does not work. I have a feeling the problem is the reading of the sql file to the CommandText but I am not sure how to debug.
[system.reflection.assembly]::LoadWithPartialName("MySql.Data")
$mysql_server = "localhost"
$mysql_user = "root"
$mysql_password = "password"
write-host "Create coonection to db1"
# Connect to MySQL database 'db1'
$cn = New-Object -TypeName MySql.Data.MySqlClient.MySqlConnection
$cn.ConnectionString = "SERVER=$mysql_server;DATABASE=db1;UID=$mysql_user;PWD=$mysql_password"
$cn.Open()
write-host "Running backup script against db1"
# Run Update Script MySQL
$cm = New-Object -TypeName MySql.Data.MySqlClient.MySqlCommand
$sql = Get-Content C:\db2.sql
$cm.Connection = $cn
$cm.CommandText = $sql
$cm.ExecuteReader()
write-host "Closing Connection"
$cn.Close()
Any assistance would be appreciated. Thanks.
This line:
$sql = Get-Content C:\db2.sql
Returns an array of strings. When that gets assigned to something expecting a string then PowerShell will concatenate the array of strings into a single string using the contents of the $OFS (output field separator) variable. If this variable isn't set, the default separator is a single space. Try this instead and see if it works:
$sql = Get-Content C:\db2.sql
...
$OFS = "`r`n"
$cm.CommandText = "$sql"
Or if you're on PowerShell 2.0:
$sql = (Get-Content C:\db2.sql) -join "`r`n"
Related
I am working on a powershell script that needs some input from a mySQL database. For the life of me I can't tell what I've done wrong here.
Every other time I run this script, I get an error Exception calling "Open" with "0" argument(s): "Out of sync with server"[0]. So, the first run, it will pull the expected data and dump it on my screen, then on the next run I get that error. And the cycle just repeats. Here is my full code (right now its just a test query to pull then dump the data. If it matters, the mySQL server is running MariaDB 10.3.14 on a Ubuntu 18.04 host.
$error.Clear()
$sqlQuery = get-content -path "C:\querytext.sql" -Raw
$sqlUser = "myuser"
$sqlPass = "mypass"
$sqlHost = "myserver"
$sqlDB = "dbname"
$connectionString = "server= $sqlHost;port=3306;uid=$sqlUser;pwd=$sqlPass;database=$sqlDB"
Try{
$connection = New-Object MySql.data.MySqlClient.MySqlConnection
$connection.ConnectionString = $connectionString
$connection.Open()
$command = New-Object MySql.data.MySqlClient.MySqlCommand($sqlQuery,$connection)
$dataAdapter = New-Object MySql.data.MySqlClient.MySqlDataAdapter($command)
$dataSet = New-Object System.Data.DataSet
$dataAdapter.fill($dataSet, "data") | Out-Null
$command.Dispose()
$sqlResults = $dataSet.tables["data"]
}
Catch {
Write-Host "ERROR : Unable to run query : $query `n$Error[0]"
}
$connection.close()
$sqlResults | Format-Table
$sqlResults | ForEach-Object {
write-host $_.fname
}
Might I suggest using the SQL PS module:
https://learn.microsoft.com/en-us/sql/powershell/download-sql-server-ps-module?view=sql-server-2017
That page has installation instructions and its from Microsoft. Personally, the dotnet class you are using, it works, but its relatively difficult to work with.
Connecting to a DB is much simpler with this module and you do not have to worry about micromanaging connections.
Invoke-Sqlcmd -ServerInstance $sqlHost -Query $sqlQuery -Database $sqlDB -Username $sqlUser -Password $sqlPass
This will return a PS object like every other PS cmdlet.
I'm a noob with PowerShell. and have a question.
I have a working PS1 script that runs a SQL query to get a ID parameter, then creates a PDF from a Crystal Report, form another program based on that returned SQL query and then, loops through the returned parameters and repeats.
What I need is to add an SQL update statement to the PS1 script, that updates a table based on the same ID that is returned from the SQL query, after the PDF is created. Before moving on to the next ID, or after the script creates all the PDF's based on the Id's.
This is what I need to add to the script.
Update DB2.dbo.table2
SET DB2.dbo.table2.Field_01 = 1
FROM DB2.dbo.table2
WHERE (DB2.dbo.Table2.ID = {ID})
The SP1 script that works looks like this.
$serverName="MAIN"
$databaseName="DB1"
$sqlCommand="select ID from ID_Query"
$connectionString = "Data Source=$serverName; " +
"Integrated Security=SSPI; " +
"Initial Catalog=$databaseName"
$connection = new-object system.data.SqlClient.SQLConnection($connectionString)
$command = new-object system.data.sqlclient.sqlcommand($sqlCommand,$connection)
$connection.Open()
$adapter = New-Object System.Data.sqlclient.sqlDataAdapter $command
$dataset = New-Object System.Data.DataSet
$adapter.Fill($dataSet) | Out-Null
$connection.Close()
foreach ($Row in $dataSet.Tables[0].Rows)
{
$commandLine= -join(#'
"C:\Program Files (x86)\CR_Program\Program_name\CR_Printer.exe" -report="C:\Crystal_report.rpt" -exportformat=PDF -exportfile="c:\test\{Parameters.ID}.pdf" -parameters"
'#,$($Row[0]),#'
"
'#)
cmd /c $commandLine
}
I hoping to mark a column field_01 to 1, so the script does not create another Crystal repoort.pdf for the same ID, as the ID will be marked to 1 then the query that runs wont see it.
or maybe there a better way to do this.
Thanks.
You can add a timestamp to the filename along with the first field of your DataTable (query results) supposing it's an ID :
# After $connection.Close()
$crPrinter = "C:\Program Files (x86)\CR_Program\Program_name\CR_Printer.exe"
$report = "-report='C:\Crystal_report.rpt'"
foreach ($Row in $dataSet.Tables[0].Rows)
{
$now = [DateTime]::Now.ToString("yyMMddHHmmss")
$outputFile = "-exportfile='c:\test\Report_id_$Row[0]_$now.pdf' -parameters"
$commandLine= [string]::Format("{0} {1} {2}", $crPrinter, $report, $outputFile)
cmd /c $commandLine
Start-Sleep 1 # Sleeps the script to offset a second
}
I'm a bit new to PowerShell, and I've got a new requirement to get Data out of a MySQL database and into an Oracle one. The strategy I chose was to output to a CSV and then import the CSV into Oracle.
I wanted to get a progress bar for the export from MySQL into CSV, so I used the data reader to achieve this. It works, and begins to export, but somewhere during the export (around record 5,000 of 4.5mil -- not consistent) it will throw an error:
Exception calling "Read" with "0" argument(s): "Fatal error encountered during data read." Exception calling "Close" with "0" argument(s): "Timeout in IO operation" Method invocation failed because [System.Management.Automation.PSObject] does not contain a method named 'op_Addition'. Exception calling "ExecuteReader" with "0" argument(s): "The CommandText property has not been properly initialized."
Applicable code block is below. I'm not sure what I'm doing wrong here, and would appreciate any feedback possible. I've been pulling my hair out on this for days.
Notes: $tableObj is a custom object with a few string fields to hold table name and SQL values. Not showing those SQL queries here, but they work.
Write-Host "[INFO]: Gathering data from MySQL select statement..."
$conn = New-Object MySql.Data.MySqlClient.MySqlConnection
$conn.ConnectionString = $MySQLConnectionString
$conn.Open()
#
# Get Count of records in table
#
$countCmd = New-Object MySql.Data.MySqlClient.MySqlCommand($tableObj.SqlCount, $conn)
$recordCount = 0
try{
$recordCount = $countCmd.ExecuteScalar()
} Catch {
Write-Host "[ERROR]: (" $tableObj.Table ") Error getting Count."
Write-Host "---" $_.Exception.Message
Exit
}
$recordCountString = $recordCount.ToString('N0')
Write-Host "[INFO]: Count for table '" $tableObj.Table "' is " $recordCountString
#
# Compose the command
#
$cmd = New-Object MySql.Data.MySqlClient.MySqlCommand($tableObj.SqlExportInit, $conn)
#
# Write to CSV using DataReader
#
Write-Host "[INFO]: Data gathered into memory. Writing data to CSV file '" $tableObj.OutFile "'"
$counter = 0 # Tracks items selected
$reader=$cmd.ExecuteReader()
$dataRows = #()
# Read all rows into a hash table
while ($reader.Read())
{
$counter++
$percent = ($counter/$recordCount)*100
$percentString = [math]::Round($percent,3)
$counterString = $counter.ToString('N0')
Write-Progress -Activity '[INFO]: CSV Export In Progress' -Status "$percentString% Complete" -CurrentOperation "($($counterString) of $($recordCountString))" -PercentComplete $percent
$row = #{}
for ($i = 0; $i -lt $reader.FieldCount; $i++)
{
$row[$reader.GetName($i)] = $reader.GetValue($i)
}
# Convert hashtable into an array of PSObjects
$dataRows += New-Object psobject -Property $row
}
$conn.Close()
$dataRows | Export-Csv $tableObj.OutFile -NoTypeInformation
EDIT: Didn't work, but I also added this line to my connection string: defaultcommandtimeout=600;connectiontimeout=25 per MySQL timeout in powershell
Using #Carl Ardiente's thinking, the query is timing out, and you have to set the timeout to something insane to fully execute. You simply have to set the timeout value for your session before you start getting data.
Write-Host "[INFO]: Gathering data from MySQL select statement..."
$conn = New-Object MySql.Data.MySqlClient.MySqlConnection
$conn.ConnectionString = $MySQLConnectionString
$conn.Open()
# Set timeout on MySql
$cmd = New-Object MySql.Data.MySqlClient.MySqlCommand("set net_write_timeout=99999; set net_read_timeout=99999", $conn)
$cmd.ExecuteNonQuery()
#
# Get Count of records in table
#
...Etc....
Not that I've found the solution, but none of the connection string changes worked. Manually setting the timeout didn't seem to help either. It seemed to be caused from too many rows returned, so I broke up the function to run in batches, and append to a CSV as it goes. This gets rid of the IO / timeout error.
Trying to execute a MySql query from Powershell ver 4, I get this error:
Could not run MySQL Query Exception calling "Fill" with "2" argument(s): "Fatal error encountered during command execution."
Here's the SQL I'm trying from Powershell. The SQL is in variable $updateReportQuery:
$updateReportQuery = "call count_changed_mark(#count_changed_mark);"
MySQLQuery -ConnectionString $connection -query $updateReportQuery
For reference, here is a snippet from the MySQLQuery function used with other queries no problem:
# Run MySQL Querys
Write-Verbose "Run MySQL Querys"
$command = New-Object MySql.Data.MySqlClient.MySqlCommand($query, $connection)
$dataAdapter = New-Object MySql.Data.MySqlClient.MySqlDataAdapter($command)
$dataSet = New-Object System.Data.DataSet
$recordCount = $dataAdapter.Fill($dataSet, "data")
$dataSet.Tables["data"] | Format-Table
return $dataSet.Tables[“data”] #returns response as an object
When I output the SQL from the function to write-host then copy/paste the SQL in a SQL console, it runs fine. I only get the error from Powershell.
Thanks in advance for your help.
Problem is with the user variable. Must add Allow User Variables=True; to connection string.
Example:
$connection = "Server=localhost;Uid=youruserid;Pwd=yourpassword;database=yourdatabase;Allow User Variables=True;"
I was contemplating the creation of a tool to load varbinary(max) fields in a SQL Server 2008 database with files as selected from a open file dialog. Is there any such tool or equivalent that I could use? My SQL server is in a hosted environment where I don't have physical access to the server so loading it with TSQL is not an option.
How about powershell?
# from: http://sev17.com/2010/05/t-sql-tuesday-006-blobs-filestream-and-powershell/
#
$server = "superfly\sqlexpress"
$database = "Yak"
$query = "INSERT dbo.FileStore VALUES (#FileData, #FileName)"
$filepath = "d:\yak.txt"
$FileName = get-childitem $filepath | select -ExpandProperty Name
$connection=new-object System.Data.SqlClient.SQLConnection
$connection.ConnectionString="Server={0};Database={1};Integrated Security=True" -f $server,$database
$command=new-object system.Data.SqlClient.SqlCommand($query,$connection)
$command.CommandTimeout=120
$connection.Open()
$fs = new-object System.IO.FileStream($filePath,[System.IO.FileMode]'Open',[System.IO.FileAccess]'Read')
$buffer = new-object byte[] -ArgumentList $fs.Length
$fs.Read($buffer, 0, $buffer.Length)
$fs.Close()
$command.Parameters.Add("#FileData", [System.Data.SqlDbType]"VarBinary", $buffer.Length)
$command.Parameters["#FileData"].Value = $buffer
$command.Parameters.Add("#FileName", [System.Data.SqlDbType]"NChar", 50)
$command.Parameters["#FileName"].Value = $FileName
$command.ExecuteNonQuery()
$connection.Close()