I connect to a ODBC DSN to collect data from a COBOL database which works for most of the tables.
Now I've a problem with a table that has "wrong" dates in some columns. The date is 0000-00-00 which doesn't seems to be allowed in powershell datatable.
Does anyone know away around this problem? since the original data cannot be changed.
The code:
$conn = New-Object Data.Odbc.OdbcConnection
$conn.ConnectionString= "dsn=DSN_name;uid=;pwd=;"
$conn.open()
$result = (new-Object Data.Odbc.OdbcCommand("SELECT * from 'TaBLE_name';",$conn)).ExecuteReader()
$tableresult = new-object "System.Data.DataTable"
try {
$tableresult.load($result)
}
Catch {
Write-Host "ERROR : Unable to run: `n$Error[0]"
}
Finally {
$conn.close()
}
The error:
ERROR : Unable to run: Exception calling "Load" with "1" argument(s): "Year, Month, and Day parameters describe an un-representable DateTime."[0]
The goal in the end is to write the $tableresult to an Mysql Database.
Related
I'm a bit new to PowerShell, and I've got a new requirement to get Data out of a MySQL database and into an Oracle one. The strategy I chose was to output to a CSV and then import the CSV into Oracle.
I wanted to get a progress bar for the export from MySQL into CSV, so I used the data reader to achieve this. It works, and begins to export, but somewhere during the export (around record 5,000 of 4.5mil -- not consistent) it will throw an error:
Exception calling "Read" with "0" argument(s): "Fatal error encountered during data read." Exception calling "Close" with "0" argument(s): "Timeout in IO operation" Method invocation failed because [System.Management.Automation.PSObject] does not contain a method named 'op_Addition'. Exception calling "ExecuteReader" with "0" argument(s): "The CommandText property has not been properly initialized."
Applicable code block is below. I'm not sure what I'm doing wrong here, and would appreciate any feedback possible. I've been pulling my hair out on this for days.
Notes: $tableObj is a custom object with a few string fields to hold table name and SQL values. Not showing those SQL queries here, but they work.
Write-Host "[INFO]: Gathering data from MySQL select statement..."
$conn = New-Object MySql.Data.MySqlClient.MySqlConnection
$conn.ConnectionString = $MySQLConnectionString
$conn.Open()
#
# Get Count of records in table
#
$countCmd = New-Object MySql.Data.MySqlClient.MySqlCommand($tableObj.SqlCount, $conn)
$recordCount = 0
try{
$recordCount = $countCmd.ExecuteScalar()
} Catch {
Write-Host "[ERROR]: (" $tableObj.Table ") Error getting Count."
Write-Host "---" $_.Exception.Message
Exit
}
$recordCountString = $recordCount.ToString('N0')
Write-Host "[INFO]: Count for table '" $tableObj.Table "' is " $recordCountString
#
# Compose the command
#
$cmd = New-Object MySql.Data.MySqlClient.MySqlCommand($tableObj.SqlExportInit, $conn)
#
# Write to CSV using DataReader
#
Write-Host "[INFO]: Data gathered into memory. Writing data to CSV file '" $tableObj.OutFile "'"
$counter = 0 # Tracks items selected
$reader=$cmd.ExecuteReader()
$dataRows = #()
# Read all rows into a hash table
while ($reader.Read())
{
$counter++
$percent = ($counter/$recordCount)*100
$percentString = [math]::Round($percent,3)
$counterString = $counter.ToString('N0')
Write-Progress -Activity '[INFO]: CSV Export In Progress' -Status "$percentString% Complete" -CurrentOperation "($($counterString) of $($recordCountString))" -PercentComplete $percent
$row = #{}
for ($i = 0; $i -lt $reader.FieldCount; $i++)
{
$row[$reader.GetName($i)] = $reader.GetValue($i)
}
# Convert hashtable into an array of PSObjects
$dataRows += New-Object psobject -Property $row
}
$conn.Close()
$dataRows | Export-Csv $tableObj.OutFile -NoTypeInformation
EDIT: Didn't work, but I also added this line to my connection string: defaultcommandtimeout=600;connectiontimeout=25 per MySQL timeout in powershell
Using #Carl Ardiente's thinking, the query is timing out, and you have to set the timeout to something insane to fully execute. You simply have to set the timeout value for your session before you start getting data.
Write-Host "[INFO]: Gathering data from MySQL select statement..."
$conn = New-Object MySql.Data.MySqlClient.MySqlConnection
$conn.ConnectionString = $MySQLConnectionString
$conn.Open()
# Set timeout on MySql
$cmd = New-Object MySql.Data.MySqlClient.MySqlCommand("set net_write_timeout=99999; set net_read_timeout=99999", $conn)
$cmd.ExecuteNonQuery()
#
# Get Count of records in table
#
...Etc....
Not that I've found the solution, but none of the connection string changes worked. Manually setting the timeout didn't seem to help either. It seemed to be caused from too many rows returned, so I broke up the function to run in batches, and append to a CSV as it goes. This gets rid of the IO / timeout error.
i've coded an ActiveDirectory logging system a couple of years ago...
it never become a status greater than beta but its still in use...
i got an issue reported and found out what happening...
they are serveral filds in such an ActiveDirectory Event witch are UserInputs, so i've to validate them! -- of course i didnt...
so after the first user got the brilliant idea to use singlequotes in a specific foldername it crashed my scripts - easy injection possible...
so id like to make an update using prepared statements like im using in PHP and others.
Now this is a Powershell Script.. id like to do something like this:
$MySQL-OBJ.CommandText = "INSERT INTO `table-name` (i1,i2,i3) VALUES (#k1,#k2,#k3)"
$MySQL-OBJ.Parameters.AddWithValue("#k1","value 1")
$MySQL-OBJ.Parameters.AddWithValue("#k2","value 2")
$MySQL-OBJ.Parameters.AddWithValue("#k3","value 3")
$MySQL-OBJ.ExecuteNonQuery()
This would work fine - 1 times.
My Script runs endless as a Service and loops all within a while($true) loop.
Powershell clams about the param is already set...
Exception calling "AddWithValue" with "2" argument(s): "Parameter
'#k1' has already been defined."
how i can reset this "bind" without closing the database connection?
id like the leave the connection open because the script is faster without closing and opening the connections when a event is fired (10+ / sec)
Example Code
(shortend and not tested)
##start
function db_prepare(){
$MySqlConnection = New-Object MySql.Data.MySqlClient.MySqlConnection
$MySqlConnection.ConnectionString = "server=$MySQLServerName;user id=$Username;password=$Password;database=$MySQLDatenbankName;pooling=false"
$MySqlConnection.Open()
$MySqlCommand = New-Object MySql.Data.MySqlClient.MySqlCommand
$MySqlCommand.Connection = $MySqlConnection
$MySqlCommand.CommandText = "INSERT INTO `whatever` (col1,col2...) VALUES (#va1,#va2...)"
}
while($true){
if($MySqlConnection.State -eq 'closed'){ db_prepare() }
## do the event reading and data formating stuff
## bild some variables to set as sql param values
$MySQLCommand.Parameters.AddWithValue("#va1",$variable_for_1)
$MySQLCommand.Parameters.AddWithValue("#va2",$variable_for_2)
.
.
.
Try{ $MySqlCommand.ExecuteNonQuery() | Out-Null }
Catch{ <# error handling #> }
}
Change your logic so that the db_prepare() method initializes a MySql connection and a MySql command with parameters. Set the parameter values for pre-declared parameter names in loop. Like so,
function db_prepare(){
# ...
# Add named parameters
$MySQLCommand.Parameters.Add("#val1", <datatype>)
$MySQLCommand.Parameters.Add("#val2", <datatype>)
}
while($true) {
# ...
# Set values for the named parameters
$MySQLCommand.Parameters.SetParameter("#val1", <value>)
$MySQLCommand.Parameters.SetParameter("#val2", <value>)
$MySqlCommand.ExecuteNonQuery()
# ...
}
Trying to execute a MySql query from Powershell ver 4, I get this error:
Could not run MySQL Query Exception calling "Fill" with "2" argument(s): "Fatal error encountered during command execution."
Here's the SQL I'm trying from Powershell. The SQL is in variable $updateReportQuery:
$updateReportQuery = "call count_changed_mark(#count_changed_mark);"
MySQLQuery -ConnectionString $connection -query $updateReportQuery
For reference, here is a snippet from the MySQLQuery function used with other queries no problem:
# Run MySQL Querys
Write-Verbose "Run MySQL Querys"
$command = New-Object MySql.Data.MySqlClient.MySqlCommand($query, $connection)
$dataAdapter = New-Object MySql.Data.MySqlClient.MySqlDataAdapter($command)
$dataSet = New-Object System.Data.DataSet
$recordCount = $dataAdapter.Fill($dataSet, "data")
$dataSet.Tables["data"] | Format-Table
return $dataSet.Tables[“data”] #returns response as an object
When I output the SQL from the function to write-host then copy/paste the SQL in a SQL console, it runs fine. I only get the error from Powershell.
Thanks in advance for your help.
Problem is with the user variable. Must add Allow User Variables=True; to connection string.
Example:
$connection = "Server=localhost;Uid=youruserid;Pwd=yourpassword;database=yourdatabase;Allow User Variables=True;"
I want to encrypt all stored procedures in existing database
so i tried with shell scripts
$db = (new-Object Microsoft.SqlServer.Management.Smo.Server("SQL_instance")).Databases.Item("[DB_name]")
Foreach ($sp in $db.StoredProcedures){
if(!$sp.IsSystemObject){
if (!$sp.IsEncrypted){
$sp.TextMode = $false;
$sp.IsEncrypted = $true;
$sp.TextMode = $true;
try
{
$sp.Alter();
}
catch{
Write-Host "$sp.Name fail to encrypted."
}
}
}
}
it is working on one of my local server but not on client's server
Source for this
i am getting following error
You script works fine for me - so I would suspect that you may need to run powershell in administrator mode or elevate its privileges..
I am updating a few records in database. And all the processing is done in eval block.
The problem is that even if the records are successfully updated , I still see a exception being raised.
To debug the exception, I tried printing it using Data Dumper but the exception is blank.
Can any one please help me identify what is this error and why is it thrown every time ?
Environment Details (Perl 5.8 and Unix SUSE)
Dump from Data Dumper:
$VAR1 = '
';
I am using various internal APIs, to update these records.. so I have modified my code to look similar:
sub main{
eval{
DB->updateRecord($value)
};
if($#){
Mail->SendMail(__PACKAGE__,$#):
}
}
package DB;
sub updateRecord{
my ($self , $value) = #_;
my $query = "update set column_value = $value ..<update query> ";
API->processQuery($query );
}
Does your code use warnings;?
The symptom your describing indicated that in your code you are passing die the string "\n". My guess would be that in your source you have a line that is trying to die with an error message but your error message was not initialized. It could be something like
my $error;
if (some_test()) {
$error = 'Some String';
}
if (some_other_test()) {
die "$error\n";
}
If some_test() passes but some_other_test() fails the die will report an error containing only a new line. It would also emit an warning if warnings are enabled.
Another possibility is a typo. If you don't use strict; the error variable might not be correct.
my $error = 'Some String';
if ($error) {
#note the typo (transposed ro to or)
die "$erorr\n";
}
Without use strict; this can be an easy mistake to miss.