opennebula libvirt live migration failire - libvirt

From the sunstone GUI I tried performing a live migration from luther to blackout but the migration failed. Kindly see below excerpt from oned.log. Thanks for assistance and leads in advance.
Wed Feb 12 13:15:34 2014 [VMM][I]: Successfully execute transfer manager driver operation: tm_premigrate.
Wed Feb 12 13:15:34 2014 [VMM][I]: ExitCode: 0
Wed Feb 12 13:15:34 2014 [VMM][I]: Successfully execute network driver operation: pre.
Wed Feb 12 13:15:35 2014 [VMM][I]: Command execution fail: /var/tmp/one/vmm/kvm/migrate 'one-4' 'blackout' 'luther' 4 luther
Wed Feb 12 13:15:35 2014 [VMM][E]: migrate: Command "virsh --connect qemu:///system migrate --live one-4 qemu+ssh://blackout/system" failed: error: unable to connect to server at 'blackout:49152': No route to host
Wed Feb 12 13:15:35 2014 [VMM][E]: Could not migrate one-4 to blackout
Wed Feb 12 13:15:35 2014 [VMM][I]: ExitCode: 1
Wed Feb 12 13:15:35 2014 [VMM][I]: Failed to execute virtualization driver operation: migrate.
Wed Feb 12 13:15:35 2014 [VMM][E]: Error live migrating VM: Could not migrate one-4 to blackout
Wed Feb 12 13:15:35 2014 [LCM][I]: Fail to live migrate VM. Assuming that the VM is still RUNNING (will poll VM).
Wed Feb 12 13:56:36 2014 [LCM][I]: New VM state is SAVE_STOP
Wed Feb 12 13:56:44 2014 [VMM][I]: ExitCode: 0

iptables was the culprit. Simple turn of iptables or add libvirt default port ranges

Related

/usr/sbin/mysql code=exited status 203/EXEC ERROR

I don't know exactly what I did wrong, but it's likely some 'chown' operation that I did. I was trying to allow the user&group mysql:mysql access to a /media/usb drive, but may have inadvertently changed something else.
When I do sudo systemctl start mysql.service I get an error. Upon examining with sudo systemctl status mysqld, I get the following:
mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: activating (start-post) (Result: exit-code) since Fri 2020-06-19 08:11:01 EDT; 19s ago
Process: 15459 ExecStart=/usr/sbin/mysqld (code=exited, status=203/EXEC)
Process: 15444 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
Main PID: 15459 (code=exited, status=203/EXEC); : 15460 (mysql-systemd-s)
Tasks: 2
Memory: 2.4M
CPU: 175ms
CGroup: /system.slice/mysql.service
└─control
├─15460 /bin/bash /usr/share/mysql/mysql-systemd-start post
└─15687 sleep 1
Jun 19 08:11:01 apil-dlrig systemd[1]: Starting MySQL Community Server...
Jun 19 08:11:01 apil-dlrig systemd[1]: mysql.service: Main process exited, code=exited, status=203/EXEC
When I check ownership on /var/lib/mysql, I get the following, which seems reasonable. I.e. user mysql has full ownership on this folder.
apil#apil-dlrig:~$ sudo ls -la /var/lib/mysql
total 176212
drwx------ 7 mysql mysql 4096 Jun 19 07:34 .
drwxr-xr-x 79 root root 4096 Oct 30 2019 ..
-rw-r----- 1 mysql mysql 56 Oct 20 2019 auto.cnf
-rw------- 1 mysql mysql 1680 Nov 22 2019 ca-key.pem
-rw-r--r-- 1 mysql mysql 1112 Nov 22 2019 ca.pem
-rw-r--r-- 1 mysql mysql 1112 Nov 22 2019 client-cert.pem
-rw------- 1 mysql mysql 1676 Nov 22 2019 client-key.pem
-rw-r--r-- 1 mysql mysql 0 May 5 06:38 debian-5.7.flag
drwxr-x--- 2 mysql mysql 4096 Jun 6 13:44 foo
-rw-r----- 1 mysql mysql 665 Jun 19 07:34 ib_buffer_pool
-rw-r----- 1 mysql mysql 79691776 Jun 19 07:34 ibdata1
-rw-r----- 1 mysql mysql 50331648 Jun 19 07:34 ib_logfile0
-rw-r----- 1 mysql mysql 50331648 Oct 20 2019 ib_logfile1
-rw-r----- 1 mysql mysql 155 Jun 16 07:23 keyring_backup
drwxr-x--- 2 mysql mysql 4096 May 5 06:38 mysql
-rw-r--r-- 1 mysql mysql 6 May 5 06:38 mysql_upgrade_info
drwxr-x--- 2 mysql mysql 4096 May 5 06:38 performance_schema
-rw------- 1 mysql mysql 1680 Nov 22 2019 private_key.pem
drwxr-x--- 2 mysql mysql 4096 Jun 16 07:25 prod
-rw-r--r-- 1 mysql mysql 452 Nov 22 2019 public_key.pem
-rw-r--r-- 1 mysql mysql 1112 Nov 22 2019 server-cert.pem
-rw------- 1 mysql mysql 1680 Nov 22 2019 server-key.pem
drwxr-x--- 2 mysql mysql 12288 Nov 22 2019 sys
The /etc/systemd/system/multi.user.wants.targets/mysql.service looks as the following. Nothing should've changed here, i.e. it is as default as MySQL comes.
# MySQL systemd service file
[Unit]
Description=MySQL Community Server
After=network.target
[Install]
WantedBy=multi-user.target
[Service]
User=mysql
Group=mysql
PermissionsStartOnly=true
ExecStartPre=/usr/share/mysql/mysql-systemd-start pre
ExecStart=/usr/sbin/mysqld
ExecStartPost=/usr/share/mysql/mysql-systemd-start post
TimeoutSec=600
Restart=on-failure
RuntimeDirectory=mysqld
RuntimeDirectoryMode=755
Wondering what could be going wrong. Any help would be appreciated.
Thanks
If you look at the original error, the issue is: ExecStart=/usr/sbin/mysqld (code=exited, status=203/EXEC). Looks like some kind of execution privilege for the mysqld file? Checked it too:
ls -la /usr/sbin/mysqld which returned
-rw-r--r-- 1 root root 24585896 Apr 30 10:52 /usr/sbin/mysqld
So the issue (I thought) was that user root didn't have the execute permission. Looks at the first three letters rw-. The last dash means no execution privilege.
So I simply ran the following chmod 777 /usr/sbin/mysqld, after which the ownership returns as
-rwxrwxrwx 1 root root 24585896 Apr 30 10:52 /usr/sbin/mysqld
Now, systemctl start mysql.service runs just fine.
It's amazing how simply the process of writing a question on stackoverflow actually helps me solve a problem 80% of the time. Thanks again, folks.

How to enable kvm/qemu logs?

I'm working in company where by they are using kvm virtualisation
[root#601 log]# virsh list --all --title
Id Name State Title
----------------------------------------------------------------------------------
2 reporting-pilosa07 running 10.3.6.172
3 reporting-pilosa09 running 10.3.6.173
4 reporting-pilosa11 running 10.3.6.174
5 reporting-pilosa13 running 10.3.6.175
6 reporting-pilosa05 running 10.3.6.171
the VMs are running, but time to time, they dead for some reason and I would like to look at individual VM logs
[root#601 qemu]# ls -ltr
total 32
-rw------- 1 root root 2341 Oct 15 2018 reporting-pilosa07.log
-rw------- 1 root root 2341 Oct 15 2018 reporting-pilosa09.log
-rw------- 1 root root 2341 Oct 15 2018 reporting-pilosa11.log
-rw------- 1 root root 2341 Oct 15 2018 reporting-pilosa13.log
-rw------- 1 root root 4885 Nov 12 2018 reporting-pilosa05.log
-rw------- 1 root root 7181 Jul 25 04:14 offlineonboarder02.log
[root#601 qemu]# pwd
/var/log/libvirt/qemu
The logs are not logging since a year back. Where can I enable back the logs so that I can observe the cause to why the VMs went dead?
Thanks.

Openshift site shuts down unexpectedly

I have a site hosted on OpenShift. Every so often, maybe once every two weeks, the server appears to randomly shut down the site. I get a 503 error on the page and the server log says:
DEBUG: Sending SIGTERM to child...
Thu Aug 06 2015 11:27:21 GMT-0400 (EDT): Received SIGTERM - terminating sample app ...
Thu Aug 06 2015 11:27:21 GMT-0400 (EDT): Node server stopped.
==> app-root/logs/mongodb.log <==
Thu Aug 6 11:27:19.208 [signalProcessingThread] shutdown: going to close sockets...
Thu Aug 6 11:27:19.208 [signalProcessingThread] shutdown: waiting for fs preallocator...
Thu Aug 6 11:27:19.208 [signalProcessingThread] shutdown: lock for final commit...
Thu Aug 6 11:27:19.208 [signalProcessingThread] shutdown: final commit...
Thu Aug 6 11:27:19.212 [signalProcessingThread] shutdown: closing all files...
Thu Aug 6 11:27:19.213 [signalProcessingThread] closeAllFiles() finished
Thu Aug 6 11:27:19.215 [signalProcessingThread] journalCleanup...
Thu Aug 6 11:27:19.215 [signalProcessingThread] removeJournalFiles
Thu Aug 6 11:27:19.269 [signalProcessingThread] shutdown: removing fs lock...
Thu Aug 6 11:27:19.314 dbexit: really exiting now
I can restart the app and it works fine again for a while but I would prefer to find the source of the issue and solve it. The error doesn't mean much to me. Where should I start looking for the origin of the SIGTERM?

loading * no listener in gwan linux 32-bit

GWAN error on Centos 7 version
I tried both version 64 bit and 32 bit version , I am using latest version from GWAN site.
Any help would be appreciated.
[Fri Nov 21 18:02:32 2014 GMT] user: root (uid:0), group: root (uid:0)
[Fri Nov 21 18:02:32 2014 GMT] system fd_max: 110,032
[Fri Nov 21 18:02:32 2014 GMT] program fd_max: 110,032
[Fri Nov 21 18:02:32 2014 GMT] updated fd_max: 500,000
[Fri Nov 21 18:02:32 2014 GMT] Available network interfaces (2):
[Fri Nov 21 18:02:32 2014 GMT] 127.0.0.1
[Fri Nov 21 18:02:32 2014 GMT] 10.21.150.98
[Fri Nov 21 18:02:32 2014 GMT] * no listener in /opt/gwan_linux32-bit
[Fri Nov 21 18:17:57 2014 GMT] ------------------------------------------------
[Fri Nov 21 18:17:57 2014 GMT] G-WAN 4.3.14 32-bit (Mar 14 2013 07:34:53)
[Fri Nov 21 18:17:57 2014 GMT] ------------------------------------------------
[Fri Nov 21 18:17:57 2014 GMT] Local Time: Fri, 21 Nov 2014 13:17:57 GMT-5
[Fri Nov 21 18:17:57 2014 GMT] RAM: (1.11 GiB free + 0 shared + 0 buffers) / 3.53 GiB total
[Fri Nov 21 18:17:57 2014 GMT] Physical Pages: 1.11 GiB / 3.53 GiB
[Fri Nov 21 18:17:57 2014 GMT] DISK: 19.14 GiB free / 19.55 GiB total
The problem is described in your log file:
no listener in /opt/gwan_linux32-bit
This is likely to be a file permission issue. Also, try to rename this folder as gwan32 as old versions had trouble with names matching the naming patterns used by listeners.
Note: the version you are using is quite old: G-WAN 4.3.14 (March 14 2013), registered users have access to far more recent releases that add features and patches for system (OS versions, LIBC versions, Hypervisors...) incompatibilities.

Delay after RCTP command on GMail

I want to create a SMTP validation on registration at my website, but I got a strange behavior on GMail SMTP servers.
The validation looks like sending emails: after establishing connection with SMTP server I send commands EHLO, HELO, MAIL FROM, RCPT TO and QUIT.
Commands like HELO and MAIL FROM not take many time, but command RCPT TO can take about 3 seconds on GMail. For example, on several others servers this command takes only 200-300ms.
Also, if I check an existing email address, RCTP command takes only 600-900ms, but this delay always takes more then 3 seconds for unexisting address.
I think, this delay is an antispam behavior, but how I can bypass it?
My domain already has correct TXT-SPF and SPF records. My server not listed in any spam lists.
I'm not seeing the delay that you're reporting after RCPT TO. I run a mail server that logs the transcript with the remote MTA for every delivery. Looking at a few recent deliveries to gmail addresses, I see that the server for gmail responded with a 250 OK almost immediately after RCPT TO from my side. See below. (email addresses and IP's anonymized to protect the innocent!).
Tue Jul 23 16:01:52 2013 GMT DNS MX record lookup for gmail.com:
Tue Jul 23 16:01:52 2013 GMT MX1=gmail-smtp-in.l.google.com, MX2=alt1.gmail-smtp-in.l.google.com, MX3=alt2.gmail-smtp-in.l.google.com
Tue Jul 23 16:01:52 2013 GMT Attempting SMTP connection from smtp.myserver.com [x.x.x.x] to gmail-smtp-in.l.google.com [173.194.64.27] on port 25
Tue Jul 23 16:01:52 2013 GMT Delivery id 813435, process id 21352
Tue Jul 23 16:01:52 2013 GMT received from remotehost: 220 mx.google.com ESMTP b5si18818338obh.145 - gsmtp
Tue Jul 23 16:01:52 2013 GMT smtp.myserver.com sent: EHLO smtp.myserver.com
Tue Jul 23 16:01:52 2013 GMT received from remotehost: 250-mx.google.com at your service, [x.x.x.x]
250-SIZE 35882577
250-8BITMIME
250-STARTTLS
250-ENHANCEDSTATUSCODES
250 PIPELINING
Tue Jul 23 16:01:52 2013 GMT smtp.myserver.com sent: STARTTLS
Tue Jul 23 16:01:52 2013 GMT received from remotehost: 220 2.0.0 Ready to start TLS
Tue Jul 23 16:01:52 2013 GMT starting TLS...
Tue Jul 23 16:01:52 2013 GMT TLS Enabled.
Tue Jul 23 16:01:52 2013 GMT smtp.myserver.com sent: EHLO smtp.myserver.com
Tue Jul 23 16:01:52 2013 GMT received from remotehost: 250-mx.google.com at your service, [x.x.x.x]
250-SIZE 35882577
250-8BITMIME
250-ENHANCEDSTATUSCODES
250 PIPELINING
Tue Jul 23 16:01:52 2013 GMT smtp.myserver.com sent: MAIL FROM: <sender#sender.com>
Tue Jul 23 16:01:52 2013 GMT received from remotehost: 250 2.1.0 OK b5si18818338obh.145 - gsmtp
Tue Jul 23 16:01:52 2013 GMT smtp.myserver.com sent: RCPT TO: <recipient#gmail.com>
Tue Jul 23 16:01:52 2013 GMT received from remotehost: 250 2.1.5 OK b5si18818338obh.145 - gsmtp
Tue Jul 23 16:01:52 2013 GMT smtp.myserver.com sent: DATA
Tue Jul 23 16:01:52 2013 GMT received from remotehost: 354 Go ahead b5si18818338obh.145 - gsmtp
Tue Jul 23 16:01:52 2013 GMT [sending message...] (14871 bytes)
Tue Jul 23 16:01:52 2013 GMT [message sent]
Tue Jul 23 16:01:53 2013 GMT received from remotehost: 250 2.0.0 OK 1374595312 b5si18818338obh.145 - gsmtp