I am trying to build a socket in TCL using the socket -server accept <port> command and then using the vwait forever loop - The simplest possible socket.
http://wiki.tcl.tk/15315
I am able to connect to the server fine but my issue is -- How do I close the socket when it is no longer needed without having to exit the application.
Some context on my application:
It is a Synopsys tool with a TCL shell.
I am planning on building a GUI using TK and ideally, I would like to develop it in Python for scalability reasons (plus the TK interface through Synopsys TCL shell is not the regular TCL/TK interface).
When the forever event loop is running, the shell is constantly listening - Making the application's own TCL user prompt unavailable - I am not expecting that the shell will be available when some command over the socket is running, but I do expect the shell to be available once the command over the socket is completed. (I understand that this will be a little complex to implement....but just putting the question out)
When I try using the exit command passed through the socket, the application (the entire one) is closed.
Is there any command that I can pass over the socket to close the socket only and not close the entire application?
Please let me know if more details are needed.
A call to exit will exit the interpreter and cause the process to close. You close sockets by using the close function on the channel. In a server application with a connected client there are two sockets. Once you have dealt with all transactions from the client your server code should call close on the client socket. This was provided to your code in the callback function you passed in when creating the server socket. The server socket also needs to be closed at some point. How you do that depends on your application and platform. On a unix system you might use an extension to trap a signal to call this close function. Or you might close the server in response to some input from a control socket or standard input channel. The interpreter will not be reading and parsing input unless you add code to do this. Using fileevent you can arrange to read from multiple channels and using info complete you can read from stdin and evaluate the input to get a REPL loop going for your server. An implementation of this can be found online and I imagine the wiki has some examples as well.
Related
I'm a total beginner to MySQL, I'm more of a firmware specialist. I'm working on an application where I will be getting GPS coordinates from a microcontroller + cellular device and I would like some way to store the coordinates and do processing on them. I figured a database hosted on a server made the most sense, which is what has brought me to MySQL.
Basically, I'm wondering what the basic protocol is for sending data to a MySQL server over an internet connection (my device has data). Like how do I connect to the server and publish data to it?
I'm experienced with MQTT and I think I could do TCP as well but I'm looking for a protocol that is not super power-intensive and I can't use anything that requires an operating system, like a python script.
To be clear, I am NOT asking you to tell me every step for how this is done, but basically what protocol and what tools could I use? Anything you can tell me would be appreciated.
I was thinking that I could use the MySQL client C code to help write a driver that could allow me to connect to the server. I'm experienced with writing drivers and the microcontroller I'm using uses C.
You need no direct connection to the DB at all. Your cellular device should be able to establish tcp connection to the ipaddress/port and to send the byte-stream through the connection. It can be the dumb unidirectional protocol with losses.
You need some service that can listen on the other side, that can parse your byte-stream, can fetch the correct packets from it and then send the data to the database. Speaking frankly that service can even be written in linux shell:
nc -lk 1234 | collector.sh
where collector.sh is a script like that:
#!/bin/sh
while read LINE
do
# $LINE parsing and all the staff
mysql -e "INSERT INTO mygps.nmea (lat,lon,dtime) VALUES ($LAT, $LON, $DTIME);"
done <<< /dev/stdin
####
Sure it isn't a best solution but it was really helpful for me at the very beginning. Then you can proceed the gathered data in any desired way.
Build a simple server that communicates with whatever gathered data and then use the server so send the data to MySQL with the help of MySQL connector. Building part of the protocol will quite time consuming. - nbk
If you "can't use anything that requires an operating system" you need some middleware that can run the MySQL client driver to talk to the database, you will then use MQTT to pass data between your sensor and the middleware. If you don't want to write this middleware yourself, something like Node-RED might come handy.
You certainly can reimplement the driver for your MC, though I personally would not want to waste the time on something like this when I can assemble a solution from existing components. Database protocols are typically chatty, synchronous, and sensitive to network quality, and I wouldn't want to waste my MC cycles on that when I can make middleware do that asynchronously. - mustaccio
Simply "reverse ssh port forwarding"? That can be done, I think, with a single ssh command at one (or both) end of the connection. MySQL, by default, needs the client to connect on port 3306 to the server. - rick-james
I reset my MySQL password using cmd commands from this guide.
cd "C:\Program Files\MySQL\MySQL Server 8.0\bin"
mysqld
--defaults-file="C:\\ProgramData\\MySQL\\MySQL Server 8.0\\my.ini"
--init-file=C:\\Users\\<username>\\resetMYSQL.txt
resetMYSQL contains a command to change password
ALTER USER 'root'#'localhost' IDENTIFIED BY '<myNewPassword>';
Now, it turned out I have to keep running the mysqld command above to start my server or else I'll get this error. If I run the command again I can flawlessly run my server until I terminate it.
This is the error. It's ECONNREFUSED.
I think my password is already reset, because apart from the command line, I also run ALTER USER in Workbench after I got access to it while the cmd is running.
Btw, before this I can just start my server using Node and Sequelize and didn't have to start server elsewhere or set anything beforehand
I'm not sure what is the problem here, so I don't know which keyword I should look up on Google. I googled the error code, but those cases seems unrelated to mine.
I'd be glad if you can explain me what's going on.
The nodejs error message you showed us, a ECONNREFUSED message with a traceback, shows the your mysql database server program was not running when your nodejs program tried to connect to it. Nodejs reaches out to MySQL via TCP/IP. TCP/IP responds "I don't know any MySQL." Specifically, it responds "ECONNREFUSED on port 3306," meaning "nothing on this machine accepts connections on MySQL's port."
nodejs does not start the mysql software for you. It connects to it and uses it.. MySQL has to be running already for that work.
Ordinarily, software like mysql runs in the form of a operating system service; a background process that runs all the time on the machine to await requests).
And, ordinarily, you don't provide init files to MySQL to do things like change passwords, except just once, if you must, to rescue something broken. In your case it looks like you forgot your MySQL password, so you needed to use an init file to rescue yourself. Once the password is reset, stop using that init file.
Explaining how to make MySQL run as a service on your machine is beyond the scope of a Stack Overflow answer. But the installers for MySQL, on almost every operating system, set it up to run as a service automatically. It's generally useless otherwise.
Is it possible to cache database connections when using PHP like you would in a J2EE container? If so, how?
There is no connection pooling in php.
mysql_pconnect and connection pooling are two different things.
There are many problems connected with mysql_pconnect and first you should read the manual and carefully use it, but this is not connection pooling.
Connection pooling is a technique where the application server manages the connections. When the application needs a connection it asks the application server for it and the application server returns one of the pooled connections if there is one free.
We can do connection scaling in php for that please go through following link: http://www.oracle.com/technetwork/articles/dsl/white-php-part1-355135.html
So no connection pooling in php.
As Julio said apache releases all resources when the request ends for the current reques. You can use mysql_pconnect but you are limited with that function and you must be very careful. Other choice is to use singleton pattern, but none of this is pooling.
This is a good article: https://blogs.oracle.com/opal/highly-scalable-connection-pooling-in-php
Also read this one http://www.apache2.es/2.2.2/mod/mod_dbd.html
Persistent connections are nothing like connection pooling. A persistent connection in php will only be reused if you make multiple db connects within the same request/script execution context. In most typical web dev scenarios you'll max out your connections way faster if you use mysql_pconnect because your script will have no way to get a reference to any open connections on your next request. The best way to use db connections in php is to make a singleton instance of a db object so that the connection is reused within the context of your script execution. This still incurs at least 1 db connect per request, but it's better than making multiple db connects per reqeust.
There is no real db connection pooling in php due to the nature of php. Php is not an application server that can sit there in between requests and manage references to a pool of open connections, at least not without some kind of major hack. I think in theory you could write an app server in php and run it as a commandline script that would just sit there in the background and keep a bunch of db connections open and pass references to them to your other scripts, but I don't know if that would be possible in practice, how you'd pass the references from your commandline script to other scripts, and I sort of doubt it would perform well even if you could pull it off. Anyway that's mostly speculation. I did just notice the link someone else posted to an apache module to allow connection pooling for prefork servers such as php. Looks interesting:
https://github.com/junamai2000/mod_namy_pool#readme
I suppose you're using mod_php, right?
When a PHP file finishes executing all it's state is killed so there's no way (in PHP code) to do connection pooling. Instead you have to rely on extensions.
You can mysql_pconnect so that your connections won't get closed after the page finishes, that way they get reused in the next request.
This might be all that you need but this isn't the same as connection pooling as there's no way to specify the number of connections to maintain opened.
You can use MySQLi.
For more info, scroll down to Connection pooling section # http://www.php.net/manual/en/mysqli.quickstart.connections.php#example-1622
Note that Connection pooling is also dependent on your server (i.e. Apache httpd) and its configuration.
If an unused persistent connection for a given combination of "host, username, password, socket, port and default database can not be found" in the open connection pool, then only mysqli opens a new connection otherwise it would reuse already open available persistent connections, which is in a way similar to the concept of connection pooling. The use of persistent connections can be enabled and disabled using the PHP directive mysqli.allow_persistent. The total number of connections opened by a script can be limited with mysqli.max_links (this may be interesting to you to address max_user_connections issue hitting hosting server's limit). The maximum number of persistent connections per PHP process can be restricted with mysqli.max_persistent.
In wider programming context, it's a task of web/app server however in this context, it's being handled by mysqli directive of PHP itself in a way supporting connection re-usability. You may also implement a singleton class to get a static instance of connection to reuse just like in Java. Just want to remind that java also doesn't support connection pooling as part of its standard JDBC, they're being different module/layers on top of JDBC drivers.
Coming to PHP, the good thing is that for the common databases in the PHP echosystem it does support Persistent Database Connections which persists the connection for 500 requests (config of max_requests in php.ini) and this avoids creating a new connection in each request. So check it out in docs in detail, it solves most of your challenges. Please note that PHP is not so much sophisticated in terms of extensive multi-threading mechanism and concurrent processing together with powerful asynchronous event handling, when compared to strictly object oriented Java. So in a way it is very less effective for PHP to have such in-built mechanism like pooling.
You cannot instantiate connection pools manually.
But you can use the "built in" connection pooling with the mysql_pconnect function.
I would like to suggest PDO::ATTR_PERSISTENT
Persistent connections are links that do not close when the execution of your script ends. When a persistent connection is requested, PHP checks if there's already an identical persistent connection (that remained open from earlier) - and if it exists, it uses it. If it does not exist, it creates the link.
Connection pooling works at MySQL server side like this.
If persistence connection is enabled into MySQL server config then MySQL keep a connection open and in sleep state after requested client (php script) finises its work and die.
When a 2nd request comes with same credential data (Same User Name, Same Password, Same Connection Parameter, Same Database name, Maybe from same IP, I am not sure about the IP) Then MySQL pool the previous connection from sleep state to active state and let the client use the connection. This helps MySQL to save time for initial resource for connection and reduce the total number of connection.
So the connection pooling option is actually available at MySQL server side. At PHP code end there is no option. mysql_pconnect() is just a wrapper that inform PHP to not send connection close request signal at the end of script run.
For features such as connection pooling - you need to install swoole extension first: https://openswoole.com/
It adds async features to php.
After that its trivial to add mysql and redis connection pooling:
https://github.com/open-smf/connection-pool
Some PHP frameworks come with pooling built-in: https://hyperf.wiki/2.2/#/en/pool
I'm running into a problem trying to use MySQL in conjunction with Emacs. Let me start by giving the relevant version information:
Local OS: Windows XP, SP3
Terminal emulator/SSH client: Putty 0.58
Remote OS (as reported by uname -a): Linux 2.6.19.7 (also happens on another installation whose Linux is 2.6.18)
Emacs version (running on remote OS): 21.3.1 (also happens on another installation whose Emacs is 21.4.1)
MySQL version (running on remote OS): 4.1.22 (also happens on another installation whose MySQL is 5.0.91)
I always have a shell buffer running inside Emacs, and I'd like to run the command 'mysql' so I can run SQL queries. But if I type
mysql -uuserid -ppassword
it just hangs. When I kill the mysql process, it reports:
bash: [XXXXX: 2] tcsetattr: Invalid argument
(where XXXXX is the process id of the bash shell that's the parent of the mysql process that was killed.)
Years ago I wrote C code that called tcsetattr() to set flags for the tty device driver. I remember it being based on the idea that a tty was connected via RS-232, and many of the attributes you set via tcsetattr() controlled characteristics of the RS-232 connection - baud rate, parity, etc. Obviously, for pseudo-ttys, most of that is irrelevant. It looks like the 'mysql' executable is trying to do some hardware control that doesn't work in the context of an Emacs shell buffer.
As I investigated further, I discovered that Emacs has a Lisp function named sql-mysql which sounds like it runs MySQL queries directly. That sounded good to me. But when I tried it I ran into different problems.
What little documentation I could find on how to use M-x sql-mysql was rather incomplete. It didn't give any information on how to run a query. http://www.emacswiki.org/emacs/SqlMode says,
"Type M-x sql-help RET to read the online help."
so I did that. The help suggests running:
M-x sql-mysql
to invoke MySQL, and then says,
"Once you have the SQLi buffer, you can enter SQL statements in the buffer. The output generated is appended to the buffer and a new prompt is generated."
The buffer never displayed a prompt. I can't click on the items in Emacs' menubar because I'm connecting via ssh (client is putty), and putty doesn't seem to transmit mouse clicks. So I can't get at the "SQL" or "In/Out" menus in Emacs' menubar.
I tried sending a C-c (by typing C-c C-c) to the *SQL* buffer, expecting that would cause it to exit, just like a shell buffer. Much to my surprise, IT DISPLAYED THE OUTPUT OF THE QUERY I'D ENTERED before reporting:
Ctrl-C -- exit!
Process SQL exited abnormally with code 1
I then noticed that M-x sql-help also says:
"If you have a really complex SQL statement or if you are writing a procedure, you can do this in a separate buffer. Put the new buffer in `sql-mode' by calling M-x sql-mode. The name of this buffer can be anything. The name of the major mode is SQL."
"In this SQL buffer (SQL mode), you can send the region or the entire buffer to the interactive SQL buffer (SQLi mode). The results are appended to the SQLi buffer without disturbing your SQL buffer."
So I created a second buffer, named it "my-sql-buffer" and ran M-x sql-mode. When I type the query:
show tables;
into that buffer and terminate it with RET, nothing happens in the *SQL* buffer. I tried positioning my cursor at the end of the query and typing C-c C-c. The minibuffer at the bottom displayed:
"Sent string to buffer *SQL*."
but nothing appeared in the *SQL* buffer. I typed C-c C-c a few more times. Each time the minibuffer redisplayed:
"Sent string to buffer *SQL*."
After typing C-c C-c several times, suddenly output appeared in the *SQL* buffer. It was the output from:
show tables;
repeated 5 times! Except the last time stopped in the middle. This behavior suggests that something is buffering the output. I checked how many bytes had been output (C-x h M-C-| wc -c) and it had output 4096 bytes! Since 4K is a common buffer size, this confirms my suspicion that something is buffering the output. But I can't figure out what's doing the buffering and how to turn the buffering off.
Can anyone help me solve one or more of the following problems:
How to run the mysql executable from a bash prompt inside emacs and not have it hang when it calls tcsetattr()?
How to run M-x sql-mysql, type a query into the *SQL* buffer and actually get it to run without making the SQL process exit?
How to flush the output to *SQL* after each query when I type a query into a second buffer that's in sql-mode?
advTHANKSance for your help.
Mark Rosenthal
You got to add (setq sql-mysql-options '("-C" "-t" "-f" "-n")) to your _emacs to make sql-mysql work on Windows properly. M-x describe-variable sql-mysql-options to read the docs.
I'm curious if it is possible to map a UNIX socket on to an INET socket. The situation is simply that I'd like to connect to a MySQL server. Unfortunately it has INET sockets disabled and therefore I can only connect with UNIX sockets. The tools I'm using/writing have to connect on an INET socket, so I'm trying to see if I can map one on to the other.
It took a fair amount of searching but I did find socat, which purportedly does what I'm looking for. I was wondering if anyone has any suggestions on how to accomplish this. The command-line I've been using (with partial success) is:
socat -v UNIX-CONNECT:/var/lib/mysql/mysql.sock TCP-LISTEN:6666,reuseaddr
Now I can make connections and talk to the server. Unfortunately any attempts at making multiple connections fail as I need to use the fork option but this option seems to render the connections nonfunctional.
I know I can tackle the issue with Perl (my preferred language), but I'd rather avoid writing the entire implementation myself. I familiar with the IO::Socket libraries, I am simply hoping anyone has experience doing this sort of thing. Open to suggestions/ideas.
Thanks.
Reverse the order of your arguments to socat, and it works.
socat -v tcp-l:6666,reuseaddr,fork unix:/var/lib/mysql/mysql.sock
This instructs socat to
Listen on TCP port 6666 (with SO_REUSEADDR)
Wait to accept a connection
When a connection is made, fork. In the child, continue the steps below. In the parent, go to 2.
Open a UNIX domain connection to the /var/lib/mysql/mysql.sock socket.
Transfer data between the two endpoints, then exit.
Writing it the other way around
socat -v unix:/var/lib/mysql/mysql.sock tcp-l:6666,reuseaddr,fork
doesn't work, because this instructs socat to
Open a UNIX domain connection to the /var/lib/mysql/mysql.sock socket.
Listen on TCP port 6666 (with SO_REUSEADDR)
Wait to accept a connection
When a connection is made, spawn a worker child to transfer data between the two addresses.
The parent continues to accept connections on the second address, but no longer has the first address available: it was given to the first child. So nothing useful can be done from this point on.
Yes, you can do this in Perl.
Look at perlipc, IO::Select, IO::Socket and Beej's Guide to Network Programming.
You might want to consider doing it in POE - it's asynchronous library for dealing with events, so it looks like great for the task.
It is not 100% relevant, but I use POE to write proxy between stateless protocol (HTTP) and statefull protocol (telnet session, and more specifically - MUD session), and it was rather simple - You can check the code in here: http://www.depesz.com/index.php/2009/04/08/learning-poe-http-2-mud-proxy/.
In the comments somebody also suggested Coro/AnyEvent - I haven't played with it yet, but you might want to check it.