I'm using the wonderful ipython-sql
But when I get some data from the database I always see the connection string printed out. See image.
How can I stop the connection string from being shown?
The extension can handle multiple connections. If you omit the connection string from the magic statement, it wants to tell you which connection the query was applied to (even if you only had one open). But if you include the connection string, then the output won't have the connection string, your statement would need to look like:
%sql postgresql://rl_odin_admin:***#localhost:5432/odin select * from heimdall.vw_time_series_type_v1;
To save typing, you can assign the connection string to a variable, and then use $ in the magic statement, like so:
con = 'postgresql://rl_odin_admin:***#localhost:5432/odin'
%sql $con select * from heimdall.vw_time_series_type_v1;
In April 2019 they added a configuration option called displaycon which you can set to False.
%config SqlMagic.displaycon = False
As of 2020-02-20, though, this version isn't in PyPI. To get it to work, I had to clone the repo and install from the source.
! git clone https://github.com/catherinedevlin/ipython-sql.git
! pip install ipython-sql/
# Successfully installed ipython-sql-0.4.0 prettytable-0.7.2 sqlparse-0.3.0
Related
I'm using SQL Magic to connect to a db2 instance. However, I can't seem to find the syntax anywhere on how to close the connection when I'm done querying the database.
you cannot explicitly close a connection using Jupyter SQL Magic. In fact, that is one of the shortcoming of using Jupyter SQL Magic to connect to DB2. You need to close your session to close the Db2 connection. Hope this helps.
This probably isn't very useful, and to the extent it is it's probably not guaranteed to work in the future. But if you need a really hackish way to close the connection, I was able to do it this way (for a postgres db, I assume it's similar for db2):
In[87]: connections = %sql -l
Out[87]: {'postgresql://ngd#node1:5432/graph': <sql.connection.Connection at 0x7effdbcf6b38>}
In[88]: conn = connections['postgresql://ngd#node1:5432/graph']
In[89]: conn.session.close()
In[90]: %sql SELECT 1
...
StatementError: (sqlalchemy.exc.ResourceClosedError) This Connection is closed
[SQL: SELECT 1]
[parameters: [{'__name__': '__main__', '__doc__': 'Automatically created module for IPython interactive environment', '__package__': None, '__loader__': None, '__s ... (123202 characters truncated) ... stgresql://ngd#node1:5432/graph']", '_i28': "conn = connections['postgresql://ngd#node1:5432/graph']\nconn.session.close()", '_i29': '%sql SELECT 1'}]]
A big problem is--if you want to reconnect, that doesn't seem to work. Even after running %reload_ext sql, and trying to connect again, it still thinks the connection is closed when you try to use it. So unless someone knows how to fix that behavior, this is only useful for disconnecting if you don't want to re-connect again (to the same db with the same params) before restarting the kernel.
You can also restart the kernel.
This is the most simple way I've found to close all connections at the end of the session. You must restart the kernel to be able to re-establish the connection.
connections = %sql -l
[c.session.close() for c in connections.values()]
sorry for being to late but I've just started with working with SQL Magic and got annoyed with the constant errors appearing. It's a bit of a awkward patch but this helped me use it.
def multiline_qry(qry):
try:
%sql {qry}
except Exception as ex:
if str(type(ex).__name__) != 'ResourceClosedError':
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
print (message)
qry = '''DROP TABLE IF EXISTS EMPLOYEE;
CREATE TABLE EMPLOYEE(firstname varchar(50),lastname varchar(50));
INSERT INTO EMPLOYEE VALUES('Tom','Mitchell'),('Jack','Ryan');
'''
multiline_qry(qry)
log out the notebook first if you want to close the connection.
I am currently working on the website that uses ADODB library. In entire website all the queries are written in UPPERCASE.
The problem is when I run the query it doesn't work because of table name which is UPPERCASE. But when I change the table name to lowercase it works.
$sql = "SELECT * FROM MEMBERS where USERNAME = '$username'";
$db = ADONewConnection('mysql');
$db->debug = true;
$db->Connect(DB_HOSTNAME, DB_USERNAME, DB_PASSWORD, DB_NAME);
$resultFriends = $db->Execute($sql);
while ($row = $resultFriends->FetchRow()) {
var_dump($row);
die;
}
Here is the error I get:
ADOConnection._Execute(SELECT * FROM MEMBERS where USERNAME = 'fury', false) % line 1012, file: adodb.inc.php
ADOConnection.Execute(SELECT * FROM MEMBERS where USERNAME = 'fury') % line 15, file: index.php
Bear in mind I don't want to change the scripts. There are 1000 files and 10000 places.
Is there any library or are there any way that I can run this queries without error?
The version for live sire was linux kernel. but the new dev site is ubuntu.
I have done this on ubuntu/ mysql CML and it didn't work.
The solution is I had to reconfigure the mySql database in AWS/rdbs
You have to modify the “lower_case_table_names” parameter for your DB Instance(s). Prior to today, the lower_case_table_names parameter was not modifiable, with a system default of zero (0) or “table names stored as specified and comparisons are case sensitive.” Beginning immediately, values of zero and one (table names are stored in lowercase and comparisons are not case sensitive) are allowed. See the MySQL documentation for more information on the lower_case_table_names parameter.
The lower_case_table_names parameter can be specified via the rds-modify-db-parameter-group API. Simply include the parameter name and specify the desired value, such as in the following example:
rds-modify-db-parameter-group example --parameters "name=lower_case_table_names, value=1, method=pending-reboot" --region us-east-1
Support for modifying parameters via the AWS Management Console is expected to be added later this year.
setting the lower_case_table_names parameter via a custom DB Parameter Group and doing so before creating an associated DB Instance. Changing the parameter for existing DB Instances could cause inconsistencies with point-in-time recovery backups and with Read Replicas.
Amazon RDS
I am pretty new to SSIS (my first real project) and I have built a package that successfully performs some insert operations on a database. I want to deploy it to multiple environments, so I have created a connection manager whose connection string will be passed in from the command line which will in turn be defined in a command file.
The command that I run from the command line is ...
DTExec /F InitialDatabseImport.dtsx /CommandFile InitialCommandFile.config.txt
In the command file (InitialCommandFile.config.txt) I have the following variable assignment ...
/SET \Package.Variables[User::DBConnectionString].Properties[Value];"\"Data Source=(localdb)\ProjectsV12;Initial Catalog=DatabaseName;Integrated Security=SSPI;Connect Timeout=30;Auto Translate=False;"\"
Running this I get the following error ...
Option "Source=(localdb)\ProjectsV12;Initial" is not valid.
It seems that is delimiting the connection string on spaces rather than semicolons.
I have tried a number of configurations with quotes. Here I changed the quotes in front of "Data" to something that seemed to make more sense ...
/SET \Package.Variables[User::DBConnectionString].Properties[Value];\""Data Source=(localdb)\ProjectsV12;Initial Catalog=MarketingPrefsDB;Integrated Security=SSPI;Connect Timeout=30;Auto Translate=False;"\"
which returned the error ...
The argument "\Package.Variables[User::DBConnectionString].Properties[Value];\"Data" has mismatched quotes.
And this one, I wrapped the entire assignment in quotes ...
/SET "\Package.Variables[User::DBConnectionString].Properties[Value];\"Data Source=(localdb)\ProjectsV12;Initial Catalog=MarketingPrefsDB;Integrated Security=SSPI;Connect Timeout=30;Auto Translate=False;\"";
which returned the error ...
Argument ""\Package.Variables[User::DBConnectionString].Properties[Value];\"Data" for option "set" is not valid.
Has anyone else who has seen this problem been able to figure it out?
Thanks,
G
UPDATE - 2014/07/21
When I put the SET command directly in the command line I am getting a slightly different error ...
Command Line ...
C:\Project>DTExec /F QuaeroInitialImport.dtsx /SET \Package.Variables[User::DBConn
ectionString].Properties[Value];"Data Source=servername;Initial Catalog=DatabaseName;Integrated Security=SSPI;Connect Timeout=30;Auto Translate=False;"
Error Message
Argument ""\Package.Variables[User::DBConnectionString].Properties[Value];\"Data Source=servername;Initial Catalog=DatabaseName;Integrated Security=SSPI;Connect Timeout=30;Auto Translate=False;" for option "set" is not valid.
I've met similar issue and just share my work-around in case others reach here thru google.
My case is:
Deploying the package on SQL Server 2014, and I use some package parameter in the package and met this issue. I've met this if I don't use package parameter on another server running SQL Server 2014 also (both server are running the same version of SQL Server 2014).
I fix this by replacing all spaces in the keywords of the connection space with an underscore (_). The connection work for me on my server like:
Data_Source=xxx.xxx.xxx.xxx,xxxx;Initial_Catalog=ABC_DEF;Provider=SQLNCLI11.1;Persist_Security Info=True;Auto_Translate=False;User_ID=userabc;
Please note even I have forgotten to replace some space in some keyword, it still works.
So I finally change it as below:
Data_Source=xxx.xxx.xxx.xxx,xxxx;Initial_Catalog=ABC_DEF;Provider=SQLNCLI11.1;Persist_Security_Info=True;Auto_Translate=False;User_ID=userabc;
So... it doesn't make me comfortable in this case as I am not sure actually if it is working on a correcting settings... you still have to verify it yourself.
I have coded a Ruby IRC bot which is on github (/ninjex/rubot) which is having some conflicting output with MySQL on a dedicated server I just purchased.
Firstly we have the connection to the database in the MySQL folder (in .gitignore) which looks similar to the following code block.
#con = Mysql.new('localhost', 'root', 'pword', 'db_name')
Then we have an actual function to query the database
def db_query
que = get_message # Grabs query from user i.e,./db_query SELECT * FROM words
results = #con.query(que) # Send query through the connection i.e, #con.query("SELECT * FROM WORDS")
results.each {|x| chan_send(x)} # For each row returned, send it to the channel via
end
On my local machine, when running the command:
./db_query SELECT amount, user from words WHERE user = 'Bob' and word = 'hello'
I receive the output in IRC in an Array like fashion: ["17", "Bob"] Where 17 is amount and Bob is the user.
However, using this same function on my dedicated server results in an output like: 17Bob I have attempted many changes in the code, as well as try to parse the data into it's own variable, however it seems that 17Bob is coming out as a single variable, making it impossible to parse into something like an array, which I could then use to send the data correctly.
This seems odd to me on both my local machine and the dedicated server, as I was expecting the output to first send 17 to the IRC and then Bob like:
17
Bob
For all the functions and source you can check my github /Ninjex/rubot, however you may need to install some gems.
A few notes:
Make sure you are sanitizing query via get_message. Or you are opening yourself up to some serious security problems.
Ensure you are using the same versions of the mysql gem, ruby and MySql. Differences in any of these may alter the expected output.
If you are at your wits end and are unable to resolve the underlying issue, you can always send a custom delimiter and use it to split. Unfortunately, it will muck up the case that is actually working and will need to be stripped out.
Here's how I would approach debugging the issue on the dedicated machine:
def db_query
que = get_sanitized_message
results = #con.query(que)
require 'pry'
binding.pry
results.each {|x| chan_send(x)}
end
Add the pry gem to your Gemfile, or gem install pry.
Update your code to use pry: see above
This will open up a pry console when the binding.pry line is hit and you can interrogate almost everything in your running application.
I would take a look at results and see if it's an array. Just type results in the console and it will print out the value. Also type out results.class. It's possible that query is returning some special result set object that is not an array, but that has a method to access the result array.
If results is an array, then the issue is most likely in chan_send. Perhaps it needs to be using something like puts vs print to ensure there's a new line after each message. Is it possible that you have different versions of your codebase deployed? I would also add a sleep 1 within the each block to ensure that this is not related to your handling of messages arriving at the same time.
I've connected to a MySQL database using Perl DBI. I would like to find out which database I'm connected to.
I don't think I can use:
$dbh->{Name}
because I call USE new_database and $dbh->{Name} only reports the database that I initially connected to.
Is there any trick or do I need to keep track of the database name?
Try just executing the query
select DATABASE();
From what I could find, the DBH has access to the DSN that you initially connected with, but not after you made the change. (There's probably a better way to switch databases.)
$dbh->{Name} returns the db name from your db handle.
If you connected to another db after connected with your dbh, using mysql query "USE db_name", and you did not setup a new perl DBI db handle, of course, $dbh->{Name} will return the first you previously connected to... It's not spontaneic generation.
So to get the connected db name once the db handle is set up - for DBI mysql:
sub get_dbname {
my ($dbh) = #_;
my $connected_db = $dbh->{name};
$connected_db =~ s/^dbname=([^;].*);host.*$/$1/;
return $connected_db;
}
You can ask mysql:
($dbname) = (each %{$dbh->selectrow_hashref("show tables")}) =~ /^Tables_in_(.*)/;
Update: obviously select DATABASE() is a better way to do it :)
When you create a connection object it is for a certain database. In DBI's case anyway. I I don't believe doing the SQL USE database_name will affect your connection instance at all. Maybe there is a select_db (My DBI is rusty) function for the connection object or you'll have to create a new connection to the new database for the connection instance to properly report it.
FWIW - probably not much - DBD::Informix keeps track of the current database, which can change if you do operations such as CREATE DATABASE. The $dbh->{Name} attribute is specified by the DBI spec as the name used when the handle is established. Consequently, there is an Informix-specific attribute $dbh->{ix_DatabaseName} that provides the actual current database name. See: perldoc DBD::Informix.
You could consider requesting the maintainer(s) of DBD::MySQL add a similar attribute.