Pass Powershell Variables to MySQL - mysql

I am using SimplySQL (https://github.com/mithrandyr/SimplySql) to access MySQL from Powershell. I have a script which gets me yesterday's sunrise and sunset times. What I am attempting to do next is pass those variables into the mysql script to pull a corresponding column of values from between those hours. However, I am missing something.
Sunrise/Sunset:
$yesDate = (Get-Date).AddDays(-1) | Get-Date -Format "yyyy-MM-dd"
$daylight = (Invoke-RestMethod "https://api.sunrise-sunset.org/json?lat=35.608237&lng=-78.647497&formatted=0&date=$esDate").results
$sunrise = ($daylight.Sunrise | Get-Date -Format "HH:mm:ss")
$sunset = ($daylight.Sunset | Get-Date -Format "HH:mm:ss")
MySQL query:
CD "C:\Program Files\MySQL\MySQL Server 5.7\bin"
Open-MySqlConnection -UserName xxxxxxx -Password -Database xxxxxxxx
Invoke-SqlQuery -Query "select LogDateTime, UVindex from `monthly_new` where LogDateTime between (curdate()) - interval 1 day and (curdate())"
Close-SqlConnection
Right now, running each separately sets the variables, and returns all values from 12am to 11:59pm yesterday. But if I try to integrate the $sunrise and $sunset variables into the mysql query, it has no idea what is going on. It could be just a simple syntax issue, but I am not sure.

The right way is to avoid inserting parameters into SQL query text.
Take MySQL .Net connector from Oracle (actually you need only MySql.Data.dll file from whole installation)
Then for SELECT's use DataAdapter. It accepts connection string and SQL command.
ConnectionString may contain parameters.
Then add parameters with values.
Then create DataSet and Fill it using adapter.
Work with dataset.tables[0] as with array of objects with properties.
Something like that, I have no MySQL now to test.
Add-Type -Path "C:\Program Files (x86)\MySQL\MySQL Connector Net 8.0.21\Assemblies\v4.5.2\MySql.Data.dll"
$sqlCommandSelect = 'select LogDateTime, UVindex from `monthly_new` where LogDateTime between #param1 and #param2'
$sqlConnString = 'server=server;user=user;database=db;password=*****;"'
$dataAdapter = [MySql.Data.MySqlClient.MySqlDataAdapter]::new($sqlCommandSelect, $sqlConnString)
$dataAdapter.SelectCommand.Parameters.AddWithValue('#param1', [DateTime]::Today.AddDays(-1))
$dataAdapter.SelectCommand.Parameters.AddWithValue('#param2', [DateTime]::Today)
$dataSet = [System.Data.DataSet]::new()
$numRecords = $dataAdapter.Fill($dataSet)
foreach ($row in $dataSet.Tables[0])
{
Write-Host "$($row.LogDateTime)`t$($row.UVindex)"
}
$dataSet.Dispose()
$dataAdapter.Dispose()
Alternatively, you can directly write variables to the SQL Query, but you need to write it in format that MySQL accepts. I suppose this is o format:
$query = "select ... '($($date1.ToString('o')))' and '($($date2.ToString('o')))'"
Something like that, I'm not sure about quotes around ($($....))

Related

How to use a powershell custom object variable in a MySQL query

Im trying to make a script which takes data from serval places in our network and centralize them in one database. At the moment I'm trying to take data from AD and but it in my database but i get some weird outcome.
function Set-ODBC-Data{
param(
[string]$query=$(throw 'query is required.')
)
$cmd = new-object System.Data.Odbc.OdbcCommand($query,$DBConnection)
$cmd.ExecuteNonQuery()
}
$DBConnection = $null
$DBConnected = $FALSE
try{
$DBConnection = New-Object System.Data.Odbc.OdbcConnection
$DBConnection.ConnectionString = "Driver={MySQL ODBC 8.0 Unicode Driver};Server=127.0.0.1;Database=pcinventory;User=uSR;Password=PpPwWwDdD;Port=3306"
$DBConnection.Open()
$DBConnected = $TRUE
Write-Host "Connected to the MySQL database."
}
catch{
Write-Host "Unable to connect to the database..."
}
$ADEMEA = "ADSERVER.SERVER.WORK"
$addata = Get-ADComputer -filter * -property Name,CanonicalName,LastLogonDate,IPv4Address,OperatingSystem,OperatingSystemVersion -Server $ADEMEA | Select-Object Name,CanonicalName,LastLogonDate,IPv4Address,OperatingSystem,OperatingSystemVersion
ForEach($aditem in $addata){
Set-ODBC-Data -query "INSERT INTO ad VALUES( '$aditem.Name', '','','','','' )"
}
The result in my database looks someting like this
This happens, as $aditem is a custom Powershell object, and the SQL insert query doesn't quite know how to handle it. The outcome is a hashtable (aka key-value store) containing objects' attributes and attribute values.
As for fix, the good one is to use parametrized queries.
As for quick and dirty work-around that makes SQL injection easy, build insert string in a few parts. Using string formatting {} and -f makes it quite simple. Like so,
$q = "INSERT INTO ad VALUES( '{0}', '{1}', '{2}' )" -f $aditem.name, "more", "stuff"
write-host "Query: $q" # For debugging purposes
Set-ODBC-Data -query $q
The problem in quick and dirty is, as mentioned SQL injection. Consider what happens if the input is
$aditem.name, "more", "'); drop database pcinventory; --"
If the syntax is about right and permissions are adequate, it will execute the insertion. Right after that, it will drop your pcinventory database. So don't be tempted to use the fast approach, unless you are sure about what you are doing.

Reuse parameterized (prepared) SQL Query

i've coded an ActiveDirectory logging system a couple of years ago...
it never become a status greater than beta but its still in use...
i got an issue reported and found out what happening...
they are serveral filds in such an ActiveDirectory Event witch are UserInputs, so i've to validate them! -- of course i didnt...
so after the first user got the brilliant idea to use singlequotes in a specific foldername it crashed my scripts - easy injection possible...
so id like to make an update using prepared statements like im using in PHP and others.
Now this is a Powershell Script.. id like to do something like this:
$MySQL-OBJ.CommandText = "INSERT INTO `table-name` (i1,i2,i3) VALUES (#k1,#k2,#k3)"
$MySQL-OBJ.Parameters.AddWithValue("#k1","value 1")
$MySQL-OBJ.Parameters.AddWithValue("#k2","value 2")
$MySQL-OBJ.Parameters.AddWithValue("#k3","value 3")
$MySQL-OBJ.ExecuteNonQuery()
This would work fine - 1 times.
My Script runs endless as a Service and loops all within a while($true) loop.
Powershell clams about the param is already set...
Exception calling "AddWithValue" with "2" argument(s): "Parameter
'#k1' has already been defined."
how i can reset this "bind" without closing the database connection?
id like the leave the connection open because the script is faster without closing and opening the connections when a event is fired (10+ / sec)
Example Code
(shortend and not tested)
##start
function db_prepare(){
$MySqlConnection = New-Object MySql.Data.MySqlClient.MySqlConnection
$MySqlConnection.ConnectionString = "server=$MySQLServerName;user id=$Username;password=$Password;database=$MySQLDatenbankName;pooling=false"
$MySqlConnection.Open()
$MySqlCommand = New-Object MySql.Data.MySqlClient.MySqlCommand
$MySqlCommand.Connection = $MySqlConnection
$MySqlCommand.CommandText = "INSERT INTO `whatever` (col1,col2...) VALUES (#va1,#va2...)"
}
while($true){
if($MySqlConnection.State -eq 'closed'){ db_prepare() }
## do the event reading and data formating stuff
## bild some variables to set as sql param values
$MySQLCommand.Parameters.AddWithValue("#va1",$variable_for_1)
$MySQLCommand.Parameters.AddWithValue("#va2",$variable_for_2)
.
.
.
Try{ $MySqlCommand.ExecuteNonQuery() | Out-Null }
Catch{ <# error handling #> }
}
Change your logic so that the db_prepare() method initializes a MySql connection and a MySql command with parameters. Set the parameter values for pre-declared parameter names in loop. Like so,
function db_prepare(){
# ...
# Add named parameters
$MySQLCommand.Parameters.Add("#val1", <datatype>)
$MySQLCommand.Parameters.Add("#val2", <datatype>)
}
while($true) {
# ...
# Set values for the named parameters
$MySQLCommand.Parameters.SetParameter("#val1", <value>)
$MySQLCommand.Parameters.SetParameter("#val2", <value>)
$MySqlCommand.ExecuteNonQuery()
# ...
}

Executing a sql procedure with parameters from Powershell

I'm completely new to Powershell so i'm a little confused about how to call a SQL procedure that takes parameters. I have opened a connection to my database successfully and i've managed to get a procedure that doesn't take parameters to work so I know that the connection is fine.
The code to add a parameter and run the query is below:
$dateToUse = Get-Date -f yyyy/MM/dd
$MysqlQuery.CommandText = "GetJourneyByDepartureDate"
$MysqlQuery.Parameters.AddWithValue("_departureDate", $dateToUse)
$queryOutput = $MysqlQuery.ExecuteReader()
Whenever I try and run my script I get an error saying
Incorrect number of arguments for PROCEDURE dbo.GetJourneyByDepartureDate; expected 1, got 0
I've had a look around trying to find a solution but I don't understand enough about Powershell to know what solutions might be correct.
Also I am unable to post the SQL query but I have managed to run the procedure many times by just running the query in HeidiSQL passing the arguement manually
EDIT:
I've now changed my code slightly, it now looks like this:
$MysqlQuery.CommandText = "GetJourneyByDepartureDate"
$MysqlQuery.Parameters.Add("#_departureDate", [System.Data.SqlDbType]::Date) | out-Null
$MysqlQuery.Parameters['#_departureDate'].Value = $dateToUse
$parameterValue = $MysqlQuery.Parameters['#_departureDate'].value
Write-Host -ForegroundColor Cyan -Object "$parameterValue";
$queryOutput = $MysqlQuery.ExecuteReader()
I'm getting the $dateToUse value output on the console in the Write-Host line but i'm still getting the same Incorrect number of arguments error as before. SP is declared as below:
CREATE PROCEDURE `GetJourneyByDepartureDate`(IN `_departureDate` DATE) READS SQL DATA
In the end I found that I needed to set the CommandType to be StoredProcedure and also I needed to add the parameter but I was missing the direction and I apparently had to add a space after the '#' but i'm not sure why. My solution is below:
$MysqlCommand = New-Object MySql.Data.MySqlClient.MySqlCommand.Connection = $connMySQL #Create SQL command
$MysqlCommand.CommandType = [System.Data.CommandType]::StoredProcedure; #Set the command to be a stored procedure
$MysqlCommand.CommandText = "GetJourneyByDepartureDate"; #Set the name of the Stored Procedure to use
$MysqlCommand.Parameters.Add("# _departureDate", [System.Data.SqlDbType]::Date) | out-Null; #Set the input and output parameters
$MysqlCommand.Parameters['# _departureDate'].Direction = [system.data.ParameterDirection]::Input; #Set the _departureDate parameter to be an input parameter
$MysqlCommand.Parameters['# _departureDate'].Value = $dateToUse; #Set the _departureDate parameter value to be dateToUse

How to automatically export data from SQL Server 2012 to CSV file?

Hope I dont upset anybody by asking too simple a question!
I have a requirement to export data from a SQL Server 2012 table, to a CSV file. This needs to be done either every hour, or ideally if it is possible, whenever a new record is created or an existing record is updated/deleted. The table contains a list of all Sites we maintain. I need to export this CSV file to a particular location, as there is an API from a third party database which monitors this location and imports CSV files from there.
The data to be extracted from SQL is:
Mxmservsite.siteid as Marker_ID, mxmservsite.name as Name, 'SITE' as Group, '3' as Status,
'' as Notes, mxmservsite.zipcode as Post_Code, 'GB' as Country, '' as Latitude,
'' as Longitude, '' as Delete
Where dataareaid='ansa'
Anyone have any clues how I can go about doing this? Sorry, I am a newbie with SQL and still learning the basics! I have searched for similar questions in the past, but havent found anything. I know there is a utility called BCP, but not sure whether that would be the best way, and if it would be, then how do I use it to run every hour, or whenever there is a record update/delete/insert?
Cheers
Here's some powershell that would do what you're after; just schedule it using the Windows Task Scheduler:
function Execute-SQLQuery {
[CmdletBinding()]
param (
[Parameter(Mandatory = $true)]
[string]$DbInstance
,
[Parameter(Mandatory = $true)]
[string]$DbCatalog
,
[Parameter(Mandatory = $true)]
[string]$Query
,
[Parameter(Mandatory = $false)]
[int]$CommandTimeoutSeconds = 30 #this is the SQL default
)
begin {
write-verbose "Call to 'Execute-SQLQuery': BEGIN"
$connectionString = ("Server={0};Database={1};Integrated Security=True;" -f $DbInstance,$DbCatalog)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$connection.Open()
}
process {
write-verbose "`n`n`n-----------------------------------------"
write-verbose "Call to 'Execute-SQLQuery': PROCESS"
write-verbose $query
write-verbose "-----------------------------------------`n`n`n"
$command = $connection.CreateCommand()
$command.CommandTimeout = $CommandTimeoutSeconds
$command.CommandText = $query
$result = $command.ExecuteReader()
$table = new-object “System.Data.DataTable”
$table.Load($result)
Write-Output $table
}
end {
write-verbose "Call to 'Execute-SQLQuery': END"
$connection.Close()
}
}
Execute-SQLQuery -DbInstance 'myServer\InstanceName' -DbCatalog 'myDatabase' -Query #"
select Mxmservsite.siteid as Marker_ID
, mxmservsite.name as Name
, 'SITE' as Group
, '3' as Status
, '' as Notes
, mxmservsite.zipcode as Post_Code
, 'GB' as Country
, '' as Latitude
, '' as Longitude
, '' as Delete
From mxmservsite --this wasn't in your original code
Where dataareaid='ansa'
"# | Export-CSV '.\MyOutputFile.csv' -NoType
To have something triggered on any change is possible; i.e. you could create a trigger on the table, then use xp_cmdshell to execute a script or similar; but that's going to lead to performance problems (triggers are often a bad option if used without being fully understood). Also xp_cmdshell opens you up to some security risks.
There are many other ways to achieve this; currently I have a thing for PowerShell as it gives you loads of flexibility with little overhead.
Another option may be to look into using linked servers to allow your source database to directly update the target without need for CSV.
Another option - create a sql agent job that runs bcp.exe command to do the export for you, at any interval you want (every hour). With bcp.exe, you can specify your file location, column/row terminators, and the filtering query.
If you want to export at every change, you can add an after trigger as mentioned above, and simply exec the sql agent job, which will execute asynchronously. If you are concerned about performance, then you should test it out to understand the impact.
If you like #John's powershell script, stick it in a sql agent job and schedule it, if anything to keep all your SQL tasks centralized.
You'll need to specify the Server Name that you are currently on. You're not able to use a drive letter using D$, but need to use a Shared drive name. The following works in 2012.
-- Declare report variables
DECLARE #REPORT_DIR VARCHAR(4000)
DECLARE #REPORT_FILE VARCHAR(100)
DECLARE #DATETIME_STAMP VARCHAR(14)
DECLARE #Statement VARCHAR(4000)
DECLARE #Command VARCHAR(4000)
--SET variables for the Report File
SET #DATETIME_STAMP = (SELECT CONVERT(VARCHAR(10), GETDATE(), 112) + REPLACE(CONVERT(VARCHAR(8), GETDATE(),108),':','')) -- Date Time Stamp with YYYYMMDDHHMMSS
SET #REPORT_DIR = '\\aServerName\SharedDirectory\' -- Setting where to send the report. The Server name and a Shared name, not a drive letter
SET #REPORT_FILE = #REPORT_DIR + 'Tables_' + #DATETIME_STAMP + '.csv' --the -t below is used for the csv file
--Create the CSV file report with all of the data. The #Statement variable must be used to use variables in the xp_cmdshell command.
SET #Statement = '"SELECT * FROM sys.tables" queryout "'+#REPORT_FILE+'" -c -t"," -r"\n" -S"CurrentServerName\Databasename" -T' --The -S must be used with the -T
SET #Command = 'bcp '+#Statement+' '
EXEC master..xp_cmdshell #Command

Run a SQL Script Against MySQL using Powershell

I have a Powershell script that backs up my MySQL DB's each night using mysqldump. This all works fine but I would like to extend the script to update a reporting db (db1) from the backup of the prod db (db2). I have written the following test script but it does not work. I have a feeling the problem is the reading of the sql file to the CommandText but I am not sure how to debug.
[system.reflection.assembly]::LoadWithPartialName("MySql.Data")
$mysql_server = "localhost"
$mysql_user = "root"
$mysql_password = "password"
write-host "Create coonection to db1"
# Connect to MySQL database 'db1'
$cn = New-Object -TypeName MySql.Data.MySqlClient.MySqlConnection
$cn.ConnectionString = "SERVER=$mysql_server;DATABASE=db1;UID=$mysql_user;PWD=$mysql_password"
$cn.Open()
write-host "Running backup script against db1"
# Run Update Script MySQL
$cm = New-Object -TypeName MySql.Data.MySqlClient.MySqlCommand
$sql = Get-Content C:\db2.sql
$cm.Connection = $cn
$cm.CommandText = $sql
$cm.ExecuteReader()
write-host "Closing Connection"
$cn.Close()
Any assistance would be appreciated. Thanks.
This line:
$sql = Get-Content C:\db2.sql
Returns an array of strings. When that gets assigned to something expecting a string then PowerShell will concatenate the array of strings into a single string using the contents of the $OFS (output field separator) variable. If this variable isn't set, the default separator is a single space. Try this instead and see if it works:
$sql = Get-Content C:\db2.sql
...
$OFS = "`r`n"
$cm.CommandText = "$sql"
Or if you're on PowerShell 2.0:
$sql = (Get-Content C:\db2.sql) -join "`r`n"