MySQL timeout in powershell - mysql

I have a MySQL DB setup on my Windows laptop. I'm using a powershell script to connect/query/input to the DB. I have a query I'm trying to run, I can run it in the MySQL Workbench currently it takes 31.032 sec to run and returns 3 rows.
SELECT puz1.sudoku9x9_id, puz1.difficulty, puz2.sudoku9x9_id, puz2.difficulty
FROM sudoku9x9 as puz1
INNER JOIN sudoku9x9 as puz2
WHERE
puz1.clue_9 = puz2.clue_1 AND puz1.region_9 = puz2.region_1 AND
puz1.difficulty/puz2.difficulty BETWEEN .84 AND 1.19 AND
NOT EXISTS (SELECT 1 FROM samurai2x AS a
WHERE
a.puz1_id = puz1.sudoku9x9_id AND a.puz2_id = puz2.sudoku9x9_id)
Powershell Script
$samurai2x = "SELECT puz1.sudoku9x9_id, puz1.difficulty, puz2.sudoku9x9_id, puz2.difficulty FROM sudoku9x9 as puz1 INNER JOIN sudoku9x9 as puz2 WHERE puz1.clue_9 = puz2.clue_1 AND puz1.region_9 = puz2.region_1 AND puz1.difficulty/puz2.difficulty BETWEEN .84 AND 1.19 AND NOT EXISTS (SELECT 1 FROM samurai2x AS a WHERE a.puz1_id = puz1.sudoku9x9_id AND a.puz2_id = puz2.sudoku9x9_id) LIMIT 1"
Invoke-MySqlQuery -Query $samurai2x | ForEach {
$diff = ([INT]$_.'difficulty' + [INT]$_.'difficulty1') / 2
Invoke-MySqlQuery -Query "INSERT INTO samurai2x(difficulty, puz1_id, puz2_id) VALUES ('$diff', '$($_.'sudoku9x9_id')', '$($_.'sudoku9x9_id1')')"
}
When I run the powershell script, it times out. So I looked into changing the timeout options. I first ran
SET GLOBAL connect_timeout=31536000;
SET GLOBAL wait_timeout=2147483;
SET GLOBAL interactive_timeout=31536000;
The numbers are the max allowed from the MySQL documentation. That did not cut it! So I edited the my.ini file and added
[mysqld]
connect_timeout=31536000
wait_timeout=2147483
interactive_timeout=31536000
I restarted the MySQL service. Still the same issue!
When the script connects to the DB it displays
ServerThread : 26
DataSource : localhost
ConnectionTimeout : 15
Database : sudoku
UseCompression : False
State : Open
ServerVersion : 5.7.17-log
ConnectionString : server=localhost;port=3306;user id=root;database=sudoku
IsPasswordExpired : False
Site :
Container :
The ConnectionTimeout has always displayed 15 prior to editing the timeout and after every attempt.
What am I missing here guys? Thanks in advance for the help.
EDIT
# Set MySQL connection info
$username = "root"
$password = cat D:\Sudoku\mysecurestring.txt | convertto-securestring
$dbcred = new-object -typename System.Management.Automation.PSCredential `
-argumentlist $username, $password
# Connect to MySQL server
Connect-MySqlServer -Credential $dbcred -ComputerName localhost -Database sudoku
I'm connecting to the DB based on the steps at this site: Querying MySQL Databases with PowerShell
EDIT
At the bottom is the modified function. I put a comment "Added" above each new line.
New Connection Line
Connect-MySqlServer -Credential $dbcred -ComputerName localhost -Database sudoku -CommandTimeOut 600 -ConnectionTimeOut 25
New Connection Output
ServerThread : 23
DataSource : localhost
ConnectionTimeout : 25
Database : sudoku
UseCompression : False
State : Open
ServerVersion : 5.7.17-log
ConnectionString : server=localhost;port=3306;user id=root;database=sudoku;defaultcommandtimeout=600;connectiontimeout=25
IsPasswordExpired : False
Site :
Container :
Modified Function
function Connect-MySqlServer
{
<#
.SYNOPSIS
Connect to a MySQL Server
.DESCRIPTION
This function will establish a connection to a local or remote instance of
a MySQL Server. By default it will connect to the local instance on the
default port.
.PARAMETER ComputerName
The name of the remote computer to connect to, otherwise default to localhost
.PARAMETER Port
By default this is 3306, otherwise specify the correct value
.PARAMETER Credential
Typically this may be your root credentials, or to work in a specific
database the credentials with appropriate rights to do work in that database.
.PARAMETER Database
An optional parameter that will connect you to a specific database
.PARAMETER TimeOut
By default timeout is set to 15 seconds
.EXAMPLE
Connect-MySqlServer -Credential (Get-Credential)
cmdlet Get-Credential at command pipeline position 1
Supply values for the following parameters:
Credential
ServerThread : 2
DataSource : localhost
ConnectionTimeout : 15
Database :
UseCompression : False
State : Open
ServerVersion : 5.6.22-log
ConnectionString : server=localhost;port=3306;User Id=root
IsPasswordExpired : False
Site :
Container :
Description
-----------
Connect to the local mysql instance as root. This example uses the
Get-Credential cmdlet to prompt for username and password.
.EXAMPLE
Connect-MySqlServer -ComputerName db.company.com -Credential (Get-Credential)
cmdlet Get-Credential at command pipeline position 1
Supply values for the following parameters:
Credential
ServerThread : 2
DataSource : db.company.com
ConnectionTimeout : 15
Database :
UseCompression : False
State : Open
ServerVersion : 5.6.22-log
ConnectionString : server=db.company.com;port=3306;User Id=root
IsPasswordExpired : False
Site :
Container :
Description
-----------
Connect to a remote mysql instance as root. This example uses the
Get-Credential cmdlet to prompt for username and password.
.NOTES
FunctionName : Connect-MySqlServer
Created by : jspatton
Date Coded : 02/11/2015 09:19:10
.LINK
https://github.com/jeffpatton1971/mod-posh/wiki/MySQL#Connect-MySqlServer
#>
[OutputType('MySql.Data.MySqlClient.MySqlConnection')]
[CmdletBinding()]
Param
(
[Parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[pscredential]$Credential,
[Parameter()]
[ValidateScript({ Test-Connection -ComputerName $_ -Quiet -Count 1 })]
[ValidateNotNullOrEmpty()]
[string]$ComputerName = $env:COMPUTERNAME,
[Parameter()]
[ValidateNotNullOrEmpty()]
[int]$Port = 3306,
[Parameter()]
[ValidateNotNullOrEmpty()]
[string]$Database,
# Added
[Parameter()]
[ValidateNotNullOrEmpty()]
[int]$CommandTimeOut = 15,
# Added
[Parameter()]
[ValidateNotNullOrEmpty()]
[int]$ConnectionTimeOut = 20
)
begin
{
$ErrorActionPreference = 'Stop'
if ($PSBoundParameters.ContainsKey('Database')) {
$connectionString = 'server={0};port={1};uid={2};pwd={3};database={4};' -f $ComputerName,$Port,$Credential.UserName, $Credential.GetNetworkCredential().Password,$Database
}
else
{
$connectionString = 'server={0};port={1};uid={2};pwd={3};' -f $ComputerName, $Port, $Credential.UserName, $Credential.GetNetworkCredential().Password
}
# Added
$connectionString = $connectionString + "default command timeout=$CommandTimeOut; Connection Timeout=$ConnectionTimeOut;"
}
process
{
try
{
[MySql.Data.MySqlClient.MySqlConnection]$conn = New-Object MySql.Data.MySqlClient.MySqlConnection($connectionString)
$conn.Open()
$Global:MySQLConnection = $conn
if ($PSBoundParameters.ContainsKey('Database')) {
$null = New-Object MySql.Data.MySqlClient.MySqlCommand("USE $Database", $conn)
}
$conn
}
catch
{
Write-Error -Message $_.Exception.Message
}
}
}

Add default command timeout=60; to the connection string in your Powershell script.
You may also want to set the CommandTimeout property of the MySqlCommand object.

Related

PowerShell: Authentication -> mysql_native_password failed with Access denied

I have find many other topics about that but non of them brings me a step forward. I think it's just a stupid little thing, but unfortunately I can't find the problem and I've been working on it for days ^^
Base:
Windows Server 2016 data center
MariaDB 10.4.10, MySQL.exe 15.1
PowerShell 5.1.14393
PS started as administrator
Attempt 1:
If I log in with "mysql -hlocalhost -uroot -p1totalsecet" from the PS, it works without any problems.
Attempt 2:
If I run a PS (not mine), I get an "Access Denied" for the user root with an identical password. The exact message is:
Exception calling "Open" with 0 argument(s): "Authentication to host 'localhost' for user 'root' using method 'mysql_native_password' failed with message: Access denied for user 'root'#'localhost' (using password: yes)"
The corresponding PS function looks like this:
$Today = (Get-Date).ToString("yyyyMMdd")
$DBErrorLog = "$PSScriptRoot\$Today-DBError.log"
$ConnectionString = "server=" + $MySQLHost + ";port=" + $MySQLPort + ";uid=" + $MySQLUserName + ";pwd=" + $MySQLPassword + ";database=" + $MySQLDatabase + ";SslMode=" + $MySQLSSL + ";Default Command Timeout=" + $MySQLCommandTimeOut + ";Connect Timeout=" + $MySQLConnectTimeout + ";"
$Error.Clear()
Try {
[void][System.Reflection.Assembly]::LoadWithPartialName("MySql.Data")
$Connection = New-Object MySql.Data.MySqlClient.MySqlConnection
$Connection.ConnectionString = $ConnectionString
$Connection.Open()
$Command = New-Object MySql.Data.MySqlClient.MySqlCommand($Query, $Connection)
$DataAdapter = New-Object MySql.Data.MySqlClient.MySqlDataAdapter($Command)
$DataSet = New-Object System.Data.DataSet
$RecordCount = $DataAdapter.Fill($DataSet, "data")
$DataSet.Tables[0]
}
Catch {
Debug "[ERROR] DATABASE ERROR : Unable to run query : $Query $($Error[0])"
}
Finally {
$Connection.Close()
}
}
User and password have been checked a hundred times and are definitely identical (test crosswise using copy and paste).
"local-infile = 1" is already set in "my.ini" under [mysql] and under [mysqld]. There is also a "dontprotecthome.conf" with the content "ProtectHome=false" in the BIN directory, although I think this is only relevant under Linux.
It would be really super nice if someone could help me there; unfortunately I've run out of ideas.
Regards
Micha

How do I connect to MySQL database via Powershell?

I was trying to get Powershell connect to a MySQL database from a remote location.
What I've tried before: (https://community.spiceworks.com/topic/1899615-powershell-command-to-connect-to-remote-sql-server)
Function DBConnection
(
[string]$server,
[string]$database,
[string]$dbuser,
[string]$dbpass,
[Parameter(Mandatory=$true)] [string]$Query,
[switch]$IntegratedSecurity,
[int]$QueryTimeout = 120
)
{
#$secdbpass = ConvertTo-SecureString $dbpass -AsPlainText -Force
if (-not ($IntegratedSecurity))
{
#$secdbpass = ConvertTo-SecureString $dbpass -AsPlainText -Force
#$connString = "Server=$server;Database=$database;user id=$dbuser;password=$secdbpass;Connect Timeout=$QueryTimeout;"
$connString = "Server=$server;Database=$database;user id=$dbuser;password=$dbpass;Connect Timeout=$QueryTimeout;"
}
Else
{ $connString = "Server=$server;Database=$database;Integrated Security=SSPI;Connect Timeout=$QueryTimeout;"
}
$dataAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$dataAdapter.SelectCommand = new-object System.Data.SqlClient.SqlCommand ($query,$connString)
$commandBuilder = new-object System.Data.SqlClient.SqlCommandBuilder $dataAdapter
$dt = New-Object System.Data.DataTable
[void]$dataAdapter.fill($dt)
$dt
}
DBConnection -server 'xxx.xxx.xxx.xxx:3306' -database 'xxx' -dbuser 'xxx' -dbpass 'xxx' -Query 'INSERT INTO xxx (xxx, xxx) VALUES ("xxx", "xxx")'
And I got this result:
Exception when calling "Fill" with 1 argument (s): "Network-related or instance-specific error connecting to SQL Server. The server was not found or is inaccessible. Check that the
Instance name is correct and whether SQL Server allows remote connections. (provider: SQL Network Interfaces, error: 25 - connection string invalid) "
In line: 28 characters: 2
+ [void] $ dataAdapter.fill ($ dt)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo: NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId: SqlException
I also tried this solution:
(https://mcpmag.com/articles/2016/03/02/querying-mysql-databases.aspx/)
$password = ConvertTo-SecureString 'xxx' -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential ('xxx',$password)
Connect-MySqlServer -Port "3306" -Credential $credential -ComputerName 'xxx.xxx.xxx.xxx' -Database "xxx"
Invoke-MySqlQuery -Query 'INSERT INTO xxx (xxx, xxx) VALUES ("xxx", "xxx")'
And this was my result:
Connect-MySqlServer: Exception when calling "Open" with 0 argument (s): "Unable to connect to any of the specified MySQL hosts."
In line: 4 characters: 1
+ Connect-MySqlServer -Port "3306" -Credential $ credential -ComputerNa ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo: NotSpecified: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId: Microsoft.PowerShell.Commands.WriteErrorException, Connect-MySqlServer
PS C: \ Users \ Administrator>
How do I connect to a MySQL database via Powershell?

How to store credentials in sql configuration file as environment variables

I've got an SQL configuration file that's something like this:
[client]
database = dev
host = my-host.com
user = dev
password = super-secret-password
default-character-set = utf8
Is there any way I can swap out the plaintext host and password with some sort of environment variable, so I don't have to push it to GitHub directly? To deploy, I've been pushing to GitHub, making a docker image of the code pushed, pulling it onto an AWS server, and running it.
I'd rather not push the plaintext config file directly so I was wondering how to get around this.
You can use Github Secret to store sensitive data for your projects .
Read more about it from here ; Creating encrypted secrets for a repository
Create Env Variable using Github Action:
steps:
- name: Execute script
env:
PASSWORD: ${{ secrets.SCRIPT_CREDENTIALS }}
run: # your script to connect the database here .
for example to use a PHP script you can follow this method :
<?php
$servername = "localhost";
$username = "username";
$password = getenv("PASSWORD");
$conn = new mysqli($servername, $username, $password);
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
echo "Connected successfully";
?>
To make a change on a .cfg file you can also use githubaction like that :
steps:
- name: Edit your config file
env:
PASSWORD: ${{ secrets.SCRIPT_CREDENTIALS }}
run: echo "password = ${{ secrets.SCRIPT_CREDENTIALS }}" >> file.cfg
Update on this for anyone using Django and having a similar issue, I was able to figure it out like this.
Before, my database connection file was set up like this:
DATABASES = {
"default": {
"ENGINE": "django.db.backends.mysql",
"OPTIONS": {
"read_default_file": "local.cnf",
},
}
}
rather than doing this, it's easier to do something like:
DATABASES = {
"default": {
"ENGINE": "django.db.backends.mysql",
'NAME': 'dev',
'USER': 'dev',
'PASSWORD': os.environ['DEV_PASS'],
'HOST': os.environ['DEV_HOST']
}
}
so then you can specify your environment variables as usual.

Connecting to MySQL using encrypted password in PowerShell

I am trying to avoid exposing my password on my connection string while connecting to my MySQL DB via PowerShell but I'm getting the error below:
Exception calling "Open" with "0" argument(s): "Authentication to host 'endpoint' for user 'my_username' using method 'mysql_native_password' failed.
$key = "01000000d08c9ddf0115d1118c7a00c04fc297eb01000000cb490db2b4c9fa44b669d5aef998b678000000000200000000001066000000010000200000003f506a26c5eeddc7620d8f9714d0ccc48968528840746270eba475f13b70a040000000000e80000000020000200000002d6f515408429f78324bc6b3589056810856bd4570fc61c717464e4f42ce691c2000000071d40064ee52a3222e3bcd64a8115d4ae1a3d6dd0daa64e7eab63fb72d5ce6d74000000034b9e0fe42c8fae724eb374a75e250da714372646099dfa076a2329673797ff3e13aedb1b79edcd2685f03802e40d7a43265fe16419acb238966298eda256567"
$password = ConvertTo-SecureString $key
[void][System.Reflection.Assembly]::LoadWithPartialName("MySql.Data")
$cn = New-Object -TypeName MySql.Data.MySqlClient.MySqlConnection
$cn.ConnectionString = "SERVER=$mysql_server;DATABASE=information_schema;UID=$mysql_user;PWD=$password"
$cn.Open()

Creating a Route53 entry for RDS using Terraform

I am attempting to create a Route53 entry for a MySQL RDS instance but having issues with the :3306 at the end of the RDS endpoint returned from Terraform.
resource "aws_db_instance" "mydb" {
allocated_storage = 10
engine = "mysql"
engine_version = "5.6.17"
instance_class = "db.t2.micro"
name = "mydb"
username = "foo"
password = "bar"
db_subnet_group_name = "my_database_subnet_group"
parameter_group_name = "default.mysql5.6"
}
resource "aws_route53_record" "database" {
zone_id = "${aws_route53_zone.primary.zone_id}"
name = "database.example.com"
type = "CNAME"
ttl = "300"
records = ["${aws_db_instance.default.endpoint}"]
}
Terraform puts a :3306 at the end of the endpoint and that gets entered into the Route53 Value of the CNAME.
When I then try to connect to the CNAME database.example.com with the MySQL client I get:
ERROR 2005 (HY000): Unknown MySQL server host 'database.example.com' (0)
Once I remove the :3306 via the AWS route53 console It seems work just fine.
Question is: How do I strip the :3306 from the Terraform RDS endpoint
As well as an endpoint output, Terraform's aws_db_instance resource also outputs address that provides the FQDN of the instance.
So all you need to do is change your aws_route53_record resource to use address instead:
resource "aws_db_instance" "mydb" {
allocated_storage = 10
engine = "mysql"
engine_version = "5.6.17"
instance_class = "db.t2.micro"
name = "mydb"
username = "foo"
password = "bar"
db_subnet_group_name = "my_database_subnet_group"
parameter_group_name = "default.mysql5.6"
}
resource "aws_route53_record" "database" {
zone_id = "${aws_route53_zone.primary.zone_id}"
name = "database.example.com"
type = "CNAME"
ttl = "300"
records = ["${aws_db_instance.mydb.address}"]
}