I'm making a eggdrop tcl script to write activity of several public IRC channels to a database (over time this will be 10 to 15 channels I think). I have two options how to handle the database connection in mind.
An user says something -> Open a mysql connection to the database -> insert information about what the user said -> close the connection
Start the bot -> Open a mysql connection to the database -> Insert information when there is channel activity -> Wait for more information etc.
I think it's better to use case 1, but when there is much channel activity I think opening and closing a connection every time will cause a massive server load and slows things down drastically after a while.
What's the best way to do this?
If you want to keep the connection open just call
mysql::ping $dbhandle
from time to time.
This can be done with something like this:
proc keepMySqlOpen {dbhandle} {
mysql::ping $dbhandle
after 2000 [list keepMySqlOpen $dbhandle]
}
....
set dbh [mysql::open ...]
keepMySqlOpen $dbh
...
An other option is just to use mysql::ping before accessing the db, which should according to the mysqltcl manual reconnect if necessary. This might be the best of both worlds (let the connection time out if there is not much activity, keep it open otherwise).
Related
I have created a discord bot that interacts with a mysql database but when you run a command that uses the UPDATE query it doesnt execute the update query but executes sleep , meaning the data in the DB isnt chnaged.
(from comment)
#client.command()
async def SetJob(ctx, uid: str, rank: str):
disout = exec("UPDATE users SET 'job'='{0}' WHERE identifier='{1}'".format(rank,uid))
if ctx.message.author == client.user:
return
if ctx.message.author.id not in whitelisted:
await ctx.send(embed=discord.Embed(title="You are not authorized to use this bot", description='Please contact Not Soviet Bear to add you to the whitelisted members list', color=discord.Color.red()))
return
else:
await ctx.send(embed=discord.Embed(title="Job Change", description="Job changed to '{0}' for Identifier'{1}'".format(rank,uid), color=discord.Color.blue()))
I assume your "bot" is periodically doing SHOW PROCESSLIST? Well, the UPDATE probably finished so fast that it did not see the query.
The Sleep says that the connection is still sitting there, but doing nothing. (There is no "sleep command"; "Sleep" indicates that no query is running at the instant.)
So, perhaps the question is "why did my update not do anything?". In order to debug that (or get help from us),
Check for errors after running the update. (You should always do this.)
Figure out the exact text of the generated SQL. (Sometimes there is an obvious syntax error or failure to escape, say, quotes.)
MySQL scenario:
When I execute "SELECT" queries in MySQL using multiple threads I get the following message: "Commands out of sync; you can't run this command now", I found that this is due to the limitation of having to wait "consume" the results to make another query.
C ++ example:
void DataProcAsyncWorker :: Execute ()
{
std :: thread (& DataProcAsyncWorker :: Run, this) .join ();
}
void DataProcAsyncWorker :: Run () {
sql :: PreparedStatement * prep_stmt = c-> con-> prepareStatement (query);
...
}
Important:
I can't help using multiple threads per query (SELECT, INSERT, ETC) because the module I'm building that is being integrated with NodeJS "locks" the thread until the result is already obtained, for this reason I need to run in the background (new thread) and resolve the "promise" containing the result obtained from MySQL
Important:
I am saving several "connections" [example: 10], and with each SQL call the function chooses a connection.
This is:
1. A connection pool that contains 10 established connections, Ex:
for (int i = 0; i <10; i ++) {
Com * c = new Com;
c-> id = i;
c-> con = openConnection ();
c-> con-> setSchema ("gateway");
conns.push_back (c);
}
2. The problem occurs when executing> = 100 SELECT queries per second, I believe that even with the connection balance 100 connections per second is a high number and the connection "ex: conns.at (50)" is in process and was not consumed
My question:
A. Does PostgreSQL have this limitation as well? Or in PostgreSQL there is also such a limitation?
B. Which server using SQL commands is recommended for large SQL queries per second without the need to "open new connections", that is:
In a conns.at (0) connection I can execute (through 2 simultaneous threads) SELECT commands.
Additional:
1. I can even create a larger number of connections in the pool, but when I simulate a number of queries per second greater than the number of pre-set connections I will get the error: "Commands out of sync", the the only solution I found was mutex, which is bad for performance
I found that PostgreSQL looks great with this (queue / queue), in a very efficient way, unlike MySQL where I need to call "_free_result", in PostgreSQL, I can run multiple queries on the same connection without receiving the error: "Commands out of sync ".
Note: I did the test using libpqxx (library for connection / queries to the PostgreSQL server in C) and it really worked like a wonder without giving me a headache.
Note: I don't know if it allows multi-thread execution or the execution is done synchronously on the server side for each connection, the only thing I know is that there is no such error in postgresql.
I'm using SQL Magic to connect to a db2 instance. However, I can't seem to find the syntax anywhere on how to close the connection when I'm done querying the database.
you cannot explicitly close a connection using Jupyter SQL Magic. In fact, that is one of the shortcoming of using Jupyter SQL Magic to connect to DB2. You need to close your session to close the Db2 connection. Hope this helps.
This probably isn't very useful, and to the extent it is it's probably not guaranteed to work in the future. But if you need a really hackish way to close the connection, I was able to do it this way (for a postgres db, I assume it's similar for db2):
In[87]: connections = %sql -l
Out[87]: {'postgresql://ngd#node1:5432/graph': <sql.connection.Connection at 0x7effdbcf6b38>}
In[88]: conn = connections['postgresql://ngd#node1:5432/graph']
In[89]: conn.session.close()
In[90]: %sql SELECT 1
...
StatementError: (sqlalchemy.exc.ResourceClosedError) This Connection is closed
[SQL: SELECT 1]
[parameters: [{'__name__': '__main__', '__doc__': 'Automatically created module for IPython interactive environment', '__package__': None, '__loader__': None, '__s ... (123202 characters truncated) ... stgresql://ngd#node1:5432/graph']", '_i28': "conn = connections['postgresql://ngd#node1:5432/graph']\nconn.session.close()", '_i29': '%sql SELECT 1'}]]
A big problem is--if you want to reconnect, that doesn't seem to work. Even after running %reload_ext sql, and trying to connect again, it still thinks the connection is closed when you try to use it. So unless someone knows how to fix that behavior, this is only useful for disconnecting if you don't want to re-connect again (to the same db with the same params) before restarting the kernel.
You can also restart the kernel.
This is the most simple way I've found to close all connections at the end of the session. You must restart the kernel to be able to re-establish the connection.
connections = %sql -l
[c.session.close() for c in connections.values()]
sorry for being to late but I've just started with working with SQL Magic and got annoyed with the constant errors appearing. It's a bit of a awkward patch but this helped me use it.
def multiline_qry(qry):
try:
%sql {qry}
except Exception as ex:
if str(type(ex).__name__) != 'ResourceClosedError':
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
print (message)
qry = '''DROP TABLE IF EXISTS EMPLOYEE;
CREATE TABLE EMPLOYEE(firstname varchar(50),lastname varchar(50));
INSERT INTO EMPLOYEE VALUES('Tom','Mitchell'),('Jack','Ryan');
'''
multiline_qry(qry)
log out the notebook first if you want to close the connection.
I am using the Mariaex.start_link method to establish a connection with MySQL database and it returns me a pid. I was wondering what's the best practice to manage these pids, i.e. close and create new ones every time? keep 1, 2, ... n pid(s) around as needed?
Also how would I close that connection or kill that pid? I tried Process.exit with :normal which doesn't stop it and I tried it with :kill but I get an error probably from Mariaex and it doesn't seem clean to kill it that way.
Thanks!
You might refer to Ecto codebase to see how it handles this case.
Basically, it starts a connection, executes a query and stops the Mariaex GenServer immediately after:
with {:ok, conn} <- Mariaex.start_link(opts) do
value = Ecto.Adapters.MySQL.Connection.execute(conn, sql, [], opts)
GenServer.stop(conn)
value
end
I've connected to a MySQL database using Perl DBI. I would like to find out which database I'm connected to.
I don't think I can use:
$dbh->{Name}
because I call USE new_database and $dbh->{Name} only reports the database that I initially connected to.
Is there any trick or do I need to keep track of the database name?
Try just executing the query
select DATABASE();
From what I could find, the DBH has access to the DSN that you initially connected with, but not after you made the change. (There's probably a better way to switch databases.)
$dbh->{Name} returns the db name from your db handle.
If you connected to another db after connected with your dbh, using mysql query "USE db_name", and you did not setup a new perl DBI db handle, of course, $dbh->{Name} will return the first you previously connected to... It's not spontaneic generation.
So to get the connected db name once the db handle is set up - for DBI mysql:
sub get_dbname {
my ($dbh) = #_;
my $connected_db = $dbh->{name};
$connected_db =~ s/^dbname=([^;].*);host.*$/$1/;
return $connected_db;
}
You can ask mysql:
($dbname) = (each %{$dbh->selectrow_hashref("show tables")}) =~ /^Tables_in_(.*)/;
Update: obviously select DATABASE() is a better way to do it :)
When you create a connection object it is for a certain database. In DBI's case anyway. I I don't believe doing the SQL USE database_name will affect your connection instance at all. Maybe there is a select_db (My DBI is rusty) function for the connection object or you'll have to create a new connection to the new database for the connection instance to properly report it.
FWIW - probably not much - DBD::Informix keeps track of the current database, which can change if you do operations such as CREATE DATABASE. The $dbh->{Name} attribute is specified by the DBI spec as the name used when the handle is established. Consequently, there is an Informix-specific attribute $dbh->{ix_DatabaseName} that provides the actual current database name. See: perldoc DBD::Informix.
You could consider requesting the maintainer(s) of DBD::MySQL add a similar attribute.