I set the server time 36 hrs faster than current time on google compute engine instance.
But it was found that the server time had been synced by google_clock_skew_daemon to current time several hours ago.
I would very appreciated if someone give advice to avoid server time being synced by this daemon.
Thanks a lot.
Stop the service of the daemon. On a Linux distribution with systemd, you can stop the service as follows:
sudo systemctl stop google-clock-sync-manager.service
sudo systemctl disable google-clock-sync-manager.service
You should then be able to set the system time as you wish, without any interference of Google's clock skew daemon.
Related
im trying to restart my mysql server. The server is on a Kubernetes/fedora container. I have tried to use # /etc/init.d/mysqld restart and # systemctl restart mysqld. The problem is that there is no files in the init.d.
When running # /etc/init.d/mysqld restart bash says No such file, obviously as there is no such file. When running # systemctl restart mysqld it responds "bash: systemctl: Command not found"
The mysql-servier is running fine and i can log into it, however i cant restart it. Please help.
To restart a server on Kubernetes you simply need to delete the pod with kubectl delete pod <id>. If you didn't create pod manually, but rather with a deployment, it will restart and come back online automatically.
Deleting a pod is a correct way of shutting down servers. First Kubernetes will send mysql a TERM signal that will politely ask it to shutdown. Then after some time (configurable) it will shoot it with KILL if it doesn't comply.
The mysql-servier is running fine and i can log into it, however i cant restart it.
You have 2 options and both have it's 'dangers' to address:
More likely, container is started with either CMD given in docker image or command directive in kubernetes manifest that is actually starting your mysql. In that case, regardless of the way you manage to do termination (during restart) of running instance of mysql on that container you will also - terminate the whole container. Kubernetes will then restart that container, or whole pod.
Less likely, but possible is that container was started with some other 'main' command, and mysql is started as part of a separate script. In that case inspecting docker file or kubernetes manifest will give you details about start/stop procedure and only in that case you can restart mysql without actually killing the container it is running onto.
Dangers: data persistence. If you don't have proper data persistence in place killing running container (either through restart or refresh) will also destroy any ephemeral data with it.
Additional note: have you tried service mysql status and service mysql restart?
Two mysql(5.6.20) instances in two docker containers (1.8.32),
master and slave build semi-synchronous replication with each other,
then users do some dml or ddl operating in master always。
after ten days or more, all the clients which connect to slave will hang
gdb -p/strace slave mysqld process hangs
pstack/perf top -p slave mysqld process show nothing
kill -9 will not kill the mysqld process
docker stop will not stop the docker container
what tools or methods can help locating the problem?
I had the same occur today. In my case, using docker compose to bring up mysql and a range of consumers, using the current "latest" mysql image from docker hub. (5.7.16-1debian8)
I've launched a number of these, and within a week I've seen a couple of instances where mysql has well over 100 threads, all the memory on the host is consumed, and the containers are hung. I can't stop anything, I can't even reboot. Only a power cycle of the VM recovers.
I'll try and monitor. I suspect it depends highly on infrastructure load (slow VM host results in slow queries backing up). The solution is more likely to be mysql tuning and a docker bug.
When I run mysqld, it has a whole lot of information about what it's doing.
As I understand, this is not the correct way to run a mysql server and you should use service mysql start instead (on older servers at least).
Any searches for mysqld log come up with logs for queries, I want to know what the program is doing as it starts. (I'm trying to set up mariadb 10.1.14 with galera replication)
I want to be able to run service mysql start and then watch what's happening in the background.
I am trying to setup a django website on EC2, basically I want to start MySQL server, and Uwsgi after reboot.
In order to make MySQL start on reboot, I did:
sudo cp /opt/mysql/server-5.6/support-files/mysql.server /etc/init.d/
sudo update-rc.d mysql.server defaults
In order to make Uwsgi start on reboot, I created a file /etc/init/uwsgi.conf:
description "ubuntu uwsgi instance"
start on runlevel [2345]
stop on runlevel [06]
exec uwsgi --ini /home/ubuntu/uwsgi.ini
However the problem is that I will need mysql to start first, right now it looks like Uwsgi starts first, and tried to connect to mysql, which fails, and mysql never gets started.
Could anyone help me on how to solve this issue?
Thanks in advance
When your computer starts up, it doesn't run the init.d scripts directly. Instead, depending on what's called the "runlevel", it runs the scripts in /etc/rcN.d (where N is the runlevel). You can determine the current runlevel with the runlevel command; mine returns 2 in normal operation. That means that when the computer started up, it ran the scripts in /etc/rc2.d. The contents of rc2.d are just symlinks to scripts in /etc/init.d, named according to whether they should be started or stopped, and the order they should be run.
Use the runlevel command to find out what runlevel your computer is at (probably 2), then look in /etc/rc2.d for a link named smthing like uwsgi, which will be a symlink to /etc/init.d/uwsgi, and rename it to zzz999 - or whatever it takes to get it to sort after the other entries - that will cause it to run last.
There's more information about init.d scripts and runlevels at https://www.linux.com/news/enterprise/systems-management/8116-an-introduction-to-services-runlevels-and-rcd-scripts
Even if you start MySQL before uWSGI you're not assured it will be available when uWSGI is managing requests.
At start MySQL does some checks on database, loads InnoDB indexes, recover from transaction log or it may even hang.
You shouldn't rely on that approach.
Instead add application logic that ensures you correctly handle unavailability of database, i.e. retrying or showing an error page to the user asking to retry.
When the system runs out of memory, ubuntu 12.04 kills the mysql process:
Out of memory: Kill process 17074 (mysqld) score 146 or sacrifice child
So the process ends up killed.
This happens at peaks of server load and mainly because of apache getting wild and eating the remaining available memory. Possible approaches could be:
Change somewhere somehow the priority of mysql, so it's not killed (probably a bad fix as something else will be killed)
Monitor the status of mysql and restart automatically whenever it's killed (the one I'm thinking about, but don't know how to do it).
How do you see it?
Abrupt termination of a database server is a very serious crash. You need to avoid this in a production system, because it may not restart cleanly.
The database server is a shared resource, and should almost never terminate in an unplanned fashion in production. The only thing that should cause unplanned termination is a catastrophic hardware or power failure. Most properly configured production data base servers have an unplanned termination once every ten years or less frequently. Seriously.
What to do?
Fix your apache configuration. Limit the number of worker threads and processes it can use, so it can't run wild. Learn how to do this. It's vital. See here: http://httpd.apache.org/docs/current/mod/mpm_common.html#maxrequestworkers
Fix the defects in your web app that are causing your apache to run wild.
If you can, move your mysqld server to a different server machine from apache, so the two don't contend for the same hardware resources.
Configure your mysqld to limit the number of connections it will accept from apache worker threads or other clients. Your web app probably handles the situation where a worker thread needs to wait for a connection. See here. http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html#sysvar_max_connections
Are you on an EC2 micro instance? You need to do some serious tuning. See here: http://ubuntuforums.org/showthread.php?t=1979049
You can check mysql status every minute (with cron) and restart if it is crashed:
* * * * * service mysql status | grep running || service mysql restart