I want to know the current size of my MySql DB, to get that data I use the following methods:
def self.calculate_total_db_size
sql = "SELECT table_schema AS 'database',
sum( data_length + index_length ) / ( 1024 *1024 ) AS size
FROM information_schema.TABLES
WHERE ENGINE=('MyISAM' || 'InnoDB' )
AND table_schema = '#{get_current_db_name}'"
return perform_sql_query(sql)
end
def self.get_current_db_name
return Rails.configuration.database_configuration[Rails.env]["database"]
end
def self.perform_sql_query(query)
result = []
mysql_res = ActiveRecord::Base.connection.execute(query)
mysql_res.each_hash{ |res| result << res }
return result
end
this works great in my development and staging environment, but from some reason when i run it in production the query doesnt return any value, if i take the MySql query and run it manually on my production DB I get the correct values. why cant i do it through the application in production?
any thoughts?
I added some more logs and that helped me pinpoint the
problem, I was using:
Rails.configuration.database_configuration[Rails.env]["database"]
which returns an empty string when I was in production and not in any other
environment, I guess it is because in my database.yml there's a link to the
development setting under production (what makes the production settings the
same as the dev).
anyway, since i dont want to change my database.yml file i just changed the
way im getting the database name.
now it works great.
I'll wager $1 it's a permissions issue in production. Log into your production server and fire up the db console (go into your rails dir and type rails db to ensure you're using the same user as your app) and try to select from tables in the information_schema database.
The mysql account you're using in production most likely doesn't have access to the records you want in the information_schema database.
Related
What I want to know is, what the heck happens, under the hoods, when I upload data through R and it turns to be way much faster than MySQL Workbench or KNIME?
I work with data and, everyday, I upload data into a MySQL server. I used to upload data using KNIME since it was much faster than uploading with MySQL Workbench (select the table -> "import data").
Some infos: The CSV has 4000 rows and 15 columns. The library I used in R is RMySQL. The node I used in KNIME is database writer.
library('RMySQL')
df=read.csv('C:/Users/my_user/Documents/file.csv', encoding = 'UTF-8', sep=';')
connection <- dbConnect(
RMySQL::MySQL(),
dbname = "db_name",
host = "yyy.xxxxxxx.com",
user = "vitor",
password = "****"
)
dbWriteTable(connection, "table_name", df, append=TRUE, row.names=FALSE)
So, to test, I did the exact same process, using the same file. It took 2 minutes in KNIME and only seconds in R.
Everything happens under the hood! Data upload to DB depends on parameters such as interface between DB and tool, network connectivity, batch size set, memory available for tool and tool data processing speed itself and probably some more. In your case RMySQL package uses batch size of 500 by default and KNIME only 1 so probably that is where the difference comes from. Try setting it to 500 in KNIME and then compare. Have no clue how MySQL Workbench works...
I am currently working on the website that uses ADODB library. In entire website all the queries are written in UPPERCASE.
The problem is when I run the query it doesn't work because of table name which is UPPERCASE. But when I change the table name to lowercase it works.
$sql = "SELECT * FROM MEMBERS where USERNAME = '$username'";
$db = ADONewConnection('mysql');
$db->debug = true;
$db->Connect(DB_HOSTNAME, DB_USERNAME, DB_PASSWORD, DB_NAME);
$resultFriends = $db->Execute($sql);
while ($row = $resultFriends->FetchRow()) {
var_dump($row);
die;
}
Here is the error I get:
ADOConnection._Execute(SELECT * FROM MEMBERS where USERNAME = 'fury', false) % line 1012, file: adodb.inc.php
ADOConnection.Execute(SELECT * FROM MEMBERS where USERNAME = 'fury') % line 15, file: index.php
Bear in mind I don't want to change the scripts. There are 1000 files and 10000 places.
Is there any library or are there any way that I can run this queries without error?
The version for live sire was linux kernel. but the new dev site is ubuntu.
I have done this on ubuntu/ mysql CML and it didn't work.
The solution is I had to reconfigure the mySql database in AWS/rdbs
You have to modify the “lower_case_table_names” parameter for your DB Instance(s). Prior to today, the lower_case_table_names parameter was not modifiable, with a system default of zero (0) or “table names stored as specified and comparisons are case sensitive.” Beginning immediately, values of zero and one (table names are stored in lowercase and comparisons are not case sensitive) are allowed. See the MySQL documentation for more information on the lower_case_table_names parameter.
The lower_case_table_names parameter can be specified via the rds-modify-db-parameter-group API. Simply include the parameter name and specify the desired value, such as in the following example:
rds-modify-db-parameter-group example --parameters "name=lower_case_table_names, value=1, method=pending-reboot" --region us-east-1
Support for modifying parameters via the AWS Management Console is expected to be added later this year.
setting the lower_case_table_names parameter via a custom DB Parameter Group and doing so before creating an associated DB Instance. Changing the parameter for existing DB Instances could cause inconsistencies with point-in-time recovery backups and with Read Replicas.
Amazon RDS
I have a working Qt application that uses MySQL as a Database, but I decided to change the database to Sqlite in order to deploy on android. I created the same DB in Sqlite with the same data and tables structure.
I changed the connection to the Sqlite successfully.
By testing the App (in Desktop) I found that some queries works fine with both Sqlite and MySQL, but in some other places the query doesn't return any data in Sqlite while working fine in MySQL.
Its not a query problem, because I changed the Query to a simple SELECT * FROM TABLE_NAME and I still get the same problem.
Here is a simple code snippet
QSqlQuery qry;
qry.prepare("SELECT * FROM users;");
if( !qry.exec() ){
qDebug()<< qry.lastError().text().toLatin1();
qDebug()<< "data" ;
}
else if (qry.size() < 1) {
qDebug()<< "There is no users" << qry.size();
}
else {
qDebug()<< "It Works !!" << qry.size();
}
While using Sqlite I always get there is no users -1
But in MySQL it returns the right number of the rows in the table.
Any suggestion what the problem might be! Is it related to speed or something?
From Qt's documentation on QSqlQuery:
Returns the size of the result (number of rows returned), or -1 if the size cannot be determined or if the database does not support reporting information about query sizes.
SQLite is actually the one which doesn't provide this information. You can confirm it with:
qDebug() << qry.driver().hasFeature(QSqlDriver::QuerySize);
In case of SQLite you need to iterate through all rows from the results and count the results by yourself.
If you just need to count rows in users table, then it's better to do:
qry.prepare("SELECT count(*) FROM users;");
This will always tell you number of rows as a single-cell result. No need for special features of database. This will work everywhere (unless some database doesn't support count(*) function, then please correct me in comments).
I am trying to query my mysql database. I am using the database cookbook and can setup a connection with my database. I trying to query my database for information so now the question is how do I store than information so I can access it in another resource. Where do the results of the query get stored? This is my recipe:
mysql_database "Get admin users" do
connection mysql_connection_info
sql "Select * from #{table_name}"
action :query
end
Thanks in advance
If you don't have experience with Ruby, this might be really confusing. There's no way to "return" the result of a provider from a Chef resource. The mysql_database is a Chef::Recipe DSL method that gets translated to Chef::Provider::Database::Mysql at runtime. This provider is defined in the cookbook.
If you take some time to dive into that provider, you'll can see how it executes queries, using the db object. In order to get the results of a query, you'll need to create your own connection object in the recipe and execute a command against it. For example
require 'mysql'
db = ::Mysql.new('host', 'username', 'password', nil, 'port', 'socket') # varies with setup
users = db.query('SELECT * FROM users')
#
# You might need to manipulate the result into a more manageable data
# structure by splitting on a carriage return, etc...
#
# Assume the new object is an Array where each entry is a username.
#
file '/etc/group' do
contents users.join("\n")
end
I find using good old Chef::Mixin:ShellOut / shell_out() fairly sufficient for this job and it's DB agnostic (assuming you know your SQL :) ). It works particularly well if all you are querying is one value; for multiple rows/columns you will need to parse the SQL query results. You need to hide row counts, column headers, eat preceding white-space, etc. from your result set to just get the query results you want. For example, below works on SQL Server :
Single item
so = shell_out!("sqlcmd ... -Q \"set nocount on; select file_name(1)\" -h-1 -W")
db_logical_name = so.stdout.chop
Multiple rows/columns (0-based position of a value within a row tells you what this column is)
so = shell_out!("sqlcmd ... -Q \"set nocount on; select * from my_table\" -h-1 -W")
rows_column_data = so.stdout.chop
# columns within rows are space separated, so can be easily parsed
I have succesfully installed mysql-proxy and understood the way working and lua scripting.
But I am not good at writing lua.
So my problem is the software I use for web hosting keep creating mysql users with OLD_PASSWORD('') in the queries. I think mysql-proxy should easyly replace OLD_PASSWORD('') with new PASSWORD('') value.
I have recently seen some tutorials on youtube some ppl manage to replace queries via phpmyadmin. If there is any other way to replace queries please let me now.
Not: I can not change queries from software because it is encoded and some files exe.
Example query:
INSERT INTO mysql.user (Host, User, Password) VALUES ('localhost', 'safaf', OLD_PASSWORD('123456'))
Expected query :
INSERT INTO mysql.user (Host, User, Password) VALUES ('localhost', 'safaf', PASSWORD('123456'))
After some search on the internet i found the solution.
function read_query( packet )
if string.byte(packet) == proxy.COM_QUERY then
local query = string.sub(packet, 2)
local replacing = false
-- matches "OLD" as first word of the query
if string.match(string.upper(query), 'OLD_PASSWORD') then
query = string.gsub(query,'OLD_PASSWORD', 'PASSWORD')
replacing = true
end
if (replacing) then
proxy.queries:append(1, string.char(proxy.COM_QUERY) .. query )
return proxy.PROXY_SEND_QUERY
end
end
end