I'm a total beginner to MySQL, I'm more of a firmware specialist. I'm working on an application where I will be getting GPS coordinates from a microcontroller + cellular device and I would like some way to store the coordinates and do processing on them. I figured a database hosted on a server made the most sense, which is what has brought me to MySQL.
Basically, I'm wondering what the basic protocol is for sending data to a MySQL server over an internet connection (my device has data). Like how do I connect to the server and publish data to it?
I'm experienced with MQTT and I think I could do TCP as well but I'm looking for a protocol that is not super power-intensive and I can't use anything that requires an operating system, like a python script.
To be clear, I am NOT asking you to tell me every step for how this is done, but basically what protocol and what tools could I use? Anything you can tell me would be appreciated.
I was thinking that I could use the MySQL client C code to help write a driver that could allow me to connect to the server. I'm experienced with writing drivers and the microcontroller I'm using uses C.
You need no direct connection to the DB at all. Your cellular device should be able to establish tcp connection to the ipaddress/port and to send the byte-stream through the connection. It can be the dumb unidirectional protocol with losses.
You need some service that can listen on the other side, that can parse your byte-stream, can fetch the correct packets from it and then send the data to the database. Speaking frankly that service can even be written in linux shell:
nc -lk 1234 | collector.sh
where collector.sh is a script like that:
#!/bin/sh
while read LINE
do
# $LINE parsing and all the staff
mysql -e "INSERT INTO mygps.nmea (lat,lon,dtime) VALUES ($LAT, $LON, $DTIME);"
done <<< /dev/stdin
####
Sure it isn't a best solution but it was really helpful for me at the very beginning. Then you can proceed the gathered data in any desired way.
Build a simple server that communicates with whatever gathered data and then use the server so send the data to MySQL with the help of MySQL connector. Building part of the protocol will quite time consuming. - nbk
If you "can't use anything that requires an operating system" you need some middleware that can run the MySQL client driver to talk to the database, you will then use MQTT to pass data between your sensor and the middleware. If you don't want to write this middleware yourself, something like Node-RED might come handy.
You certainly can reimplement the driver for your MC, though I personally would not want to waste the time on something like this when I can assemble a solution from existing components. Database protocols are typically chatty, synchronous, and sensitive to network quality, and I wouldn't want to waste my MC cycles on that when I can make middleware do that asynchronously. - mustaccio
Simply "reverse ssh port forwarding"? That can be done, I think, with a single ssh command at one (or both) end of the connection. MySQL, by default, needs the client to connect on port 3306 to the server. - rick-james
Related
Is it possible to cache database connections when using PHP like you would in a J2EE container? If so, how?
There is no connection pooling in php.
mysql_pconnect and connection pooling are two different things.
There are many problems connected with mysql_pconnect and first you should read the manual and carefully use it, but this is not connection pooling.
Connection pooling is a technique where the application server manages the connections. When the application needs a connection it asks the application server for it and the application server returns one of the pooled connections if there is one free.
We can do connection scaling in php for that please go through following link: http://www.oracle.com/technetwork/articles/dsl/white-php-part1-355135.html
So no connection pooling in php.
As Julio said apache releases all resources when the request ends for the current reques. You can use mysql_pconnect but you are limited with that function and you must be very careful. Other choice is to use singleton pattern, but none of this is pooling.
This is a good article: https://blogs.oracle.com/opal/highly-scalable-connection-pooling-in-php
Also read this one http://www.apache2.es/2.2.2/mod/mod_dbd.html
Persistent connections are nothing like connection pooling. A persistent connection in php will only be reused if you make multiple db connects within the same request/script execution context. In most typical web dev scenarios you'll max out your connections way faster if you use mysql_pconnect because your script will have no way to get a reference to any open connections on your next request. The best way to use db connections in php is to make a singleton instance of a db object so that the connection is reused within the context of your script execution. This still incurs at least 1 db connect per request, but it's better than making multiple db connects per reqeust.
There is no real db connection pooling in php due to the nature of php. Php is not an application server that can sit there in between requests and manage references to a pool of open connections, at least not without some kind of major hack. I think in theory you could write an app server in php and run it as a commandline script that would just sit there in the background and keep a bunch of db connections open and pass references to them to your other scripts, but I don't know if that would be possible in practice, how you'd pass the references from your commandline script to other scripts, and I sort of doubt it would perform well even if you could pull it off. Anyway that's mostly speculation. I did just notice the link someone else posted to an apache module to allow connection pooling for prefork servers such as php. Looks interesting:
https://github.com/junamai2000/mod_namy_pool#readme
I suppose you're using mod_php, right?
When a PHP file finishes executing all it's state is killed so there's no way (in PHP code) to do connection pooling. Instead you have to rely on extensions.
You can mysql_pconnect so that your connections won't get closed after the page finishes, that way they get reused in the next request.
This might be all that you need but this isn't the same as connection pooling as there's no way to specify the number of connections to maintain opened.
You can use MySQLi.
For more info, scroll down to Connection pooling section # http://www.php.net/manual/en/mysqli.quickstart.connections.php#example-1622
Note that Connection pooling is also dependent on your server (i.e. Apache httpd) and its configuration.
If an unused persistent connection for a given combination of "host, username, password, socket, port and default database can not be found" in the open connection pool, then only mysqli opens a new connection otherwise it would reuse already open available persistent connections, which is in a way similar to the concept of connection pooling. The use of persistent connections can be enabled and disabled using the PHP directive mysqli.allow_persistent. The total number of connections opened by a script can be limited with mysqli.max_links (this may be interesting to you to address max_user_connections issue hitting hosting server's limit). The maximum number of persistent connections per PHP process can be restricted with mysqli.max_persistent.
In wider programming context, it's a task of web/app server however in this context, it's being handled by mysqli directive of PHP itself in a way supporting connection re-usability. You may also implement a singleton class to get a static instance of connection to reuse just like in Java. Just want to remind that java also doesn't support connection pooling as part of its standard JDBC, they're being different module/layers on top of JDBC drivers.
Coming to PHP, the good thing is that for the common databases in the PHP echosystem it does support Persistent Database Connections which persists the connection for 500 requests (config of max_requests in php.ini) and this avoids creating a new connection in each request. So check it out in docs in detail, it solves most of your challenges. Please note that PHP is not so much sophisticated in terms of extensive multi-threading mechanism and concurrent processing together with powerful asynchronous event handling, when compared to strictly object oriented Java. So in a way it is very less effective for PHP to have such in-built mechanism like pooling.
You cannot instantiate connection pools manually.
But you can use the "built in" connection pooling with the mysql_pconnect function.
I would like to suggest PDO::ATTR_PERSISTENT
Persistent connections are links that do not close when the execution of your script ends. When a persistent connection is requested, PHP checks if there's already an identical persistent connection (that remained open from earlier) - and if it exists, it uses it. If it does not exist, it creates the link.
Connection pooling works at MySQL server side like this.
If persistence connection is enabled into MySQL server config then MySQL keep a connection open and in sleep state after requested client (php script) finises its work and die.
When a 2nd request comes with same credential data (Same User Name, Same Password, Same Connection Parameter, Same Database name, Maybe from same IP, I am not sure about the IP) Then MySQL pool the previous connection from sleep state to active state and let the client use the connection. This helps MySQL to save time for initial resource for connection and reduce the total number of connection.
So the connection pooling option is actually available at MySQL server side. At PHP code end there is no option. mysql_pconnect() is just a wrapper that inform PHP to not send connection close request signal at the end of script run.
For features such as connection pooling - you need to install swoole extension first: https://openswoole.com/
It adds async features to php.
After that its trivial to add mysql and redis connection pooling:
https://github.com/open-smf/connection-pool
Some PHP frameworks come with pooling built-in: https://hyperf.wiki/2.2/#/en/pool
Can I Use My Browser to Access MySQL server by sending requests in URL?
I have Tried giving URL of my Server and port to which it is bound send the user name and password in URL as properties and value
http://127.0.0.1:3306/?user=root&password=password
But it gives error like (J���
5.5.28����/,-<&Hv;�ÿ÷�€����������ca>OR08fzTsi�mysql_native_password�!��ÿ„#08S01Got packets out of order)
Does using Any protocol Work?
You are attempting to access a server bound on a port that does not speak the protocol of http, but the socket open request is naturally granted.
Mysql responds in non-http and non-html output, and your browser displays the gobbly-gook. As your browser is naturally not a mysql client with baked in mysql library calls to deal with that handshake, the train just went off the tracks.
Instead of trying to craft an extension to mysql to perform this, it is best to redirect one's focus to the likes of PHP, asp.net, a java back-end middle-ware, etc.
As for passing values like you are in the URL, I suggest you read This Blog Here and jump down to the text showing:
Doesn’t look too bad? Let’s take a look at the URL:
A_URL?domain=&subdomain=sdjflsdhkfhds&name=asdasdf&email=aaaaa#letthemeatspam.com&pass1=ThisIsMyPassword&pass2=ThisIsMyPassword&aggree=yes&error_multiple=&error_domain=&error_subdomain=&error_name=&error_email=&error_pass=2&error_tos=&error_number=&error_js=&error_disposable=1&error_bad_gmail=
Not only is that information sitting in your users cache, it is visible along the way between the user and server (perhaps half a dozen hops) and is logged. If that doesn't raise an eyebrow of concern, I don't know what more to say.
You cannot use your browser to access MySQL without installing a browser-based tool such as phpMyAdmin. I do not recommend this.
Instead, just use the MySQL command line client. From a shell prompt:
mysql -h127.0.0.1 -uroot -p
You will be prompted for your password.
I am trying to learn about databases and my most current challenge is connecting from my iOS machine using qt5 into a database through the QODBC driver. After a bit of goggle-ing sure enough I found a bunch of threads on the subject and I now realise that in order to get rid of my
"QODBC3: Unable to connect" "[iODBC][Driver Manager]Data source name not found and no default driver specified. Driver could not be loaded"
error I must specify a connection string like such
db->setDatabaseName("Driver={SQL Server Native Client 11.0};Server=*<ip/host>*;Database=*<database>*;Uid=*<userid>*;Pwd=*<password>*;"
But I still can't make this work, because I am not sure what should I put as a Driver value. Where is this driver, or what if my database server is on another computer inside my Lan?
The QODBC driver adds a windows DSN. They are quite clever about preventing access from processes not running on the Quickbooks computer unless you buy their $499 server version.
I access the QODBC driver from within PHP command line, I first allow the odbc php module and use this in my PHP script:
$qb = odbc_connect("Quickbooks Data", "", "");
I did experiment with running a PHP script as a service, connected to a port which would then listen for incoming queries and reply with the result.
This worked, but ultimately we moved Quickbooks out of the workflow we use interacting with customers. Quickbooks is too unreliable for mission critical processes (example: taking money from customers); multiple times something would happen, Windows or UPS would update, and then our automated workflow for shuffling customer data and charging credit cards would fail.
I saw the following statement on StackOverflow and was wondering about its meaning:
If you connect via 'localhost', the connection will automatically be established via the MySQL socket, which is really cheap anyways.
The discussion thread was pretty old, so I didn't want to comment on it.
Basically what I understand is, that using 'localhost' when connecting to your mysql database has certain advantages - such as "automatically established connections via MySQL socket". What does that mean exactly?
Currently I'm using
mysql_connect("73.21.24.201", [...]);
(changed to a random IP Address)
Does it make any difference? Can I change it to "localhost" without having to worry about it? (The mysql server is obviously on the same server/ip address as my website/application)
When you connect to 'localhost' you'll connect using a Unix socket, which is just a communications channel for the local processes to use. The big advantage of this is that you can disable networking completely in MySQL, and negate any processing overhead and security risks that go along with that.
When MySQL starts, it creates a socket file (typically at a place like /var/lib/mysql/mysql.sock) that your client program needs to be able to find. On a typical PHP (you didn't say, but I'm assuming) setup, it should know where to find this socket. If not, check /etc/my.cnf and /etc/php.ini to make sure the values match.
And finally, if that is PHP, stop using mysql_*() functions in PHP right now! They have been deprecated for years and are inefficient and insecure.
I'd like to connect to a MySQL server (AWS RDS, if you must know) via SSL. I'm familiar with the basic mechanics involved.
All the methods I've seen so far involve running a SQL query to determine that the current connection uses SSL.
My concern is that this is a "chicken and egg" problem: I would need to send my credentials to the server potentially in plain text, so I can run a query on the server to determine that it was not plain text.
Is there a way to run the SHOW STATUS LIKE 'Ssl_cipher'; query without first authenticating?
This seems like a pretty basic requirement (I'd like to know that my password will be encrypted BEFORE sending it over the wire) but it seems difficult to find the required information.
I've read the following resources:
https://kb.berkeley.edu/page.php?id=23112
http://dev.mysql.com/doc/refman/5.1/en/ssl-options.html
http://dev.mysql.com/doc/refman/5.0/en/creating-ssl-certs.html
MySQL native authentication was never sending password over network without regard of SSL usage (it always sends result of function of password and salt returned from Server instead).
In recent versions it may be beneficial to send clear text password - but that may happen only if you explicitly provide option (on client) --enable-cleartext-plugin (or similar).