Dashboard Error : Connection to the hypervisor is broken on host
# tail -f /var/log/nova/nova-compute.log
2016-12-05 09:48:17.077 1920 INFO nova.virt.libvirt.driver [req-c2a6d49a-ffa9-4364-b5a0-c9dd58a36f45 b9cb3442c99a4d9fb37d821f378f9329 22e36611b69e412ab6a49e4c453336f3 - - -] Connection event '0' reason 'Failed to connect to libvirt'
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host [req-c2a6d49a-ffa9-4364-b5a0-c9dd58a36f45 b9cb3442c99a4d9fb37d821f378f9329 22e36611b69e412ab6a49e4c453336f3 - - -] Connection to libvirt failed: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host Traceback (most recent call last):
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/host.py", line 528, in get_connection
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host conn = self._get_connection()
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/host.py", line 515, in _get_connection
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host wrapped_conn = self._get_new_connection()
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/host.py", line 467, in _get_new_connection
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host wrapped_conn = self._connect(self._uri, self._read_only)
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/host.py", line 321, in _connect
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host libvirt.openAuth, uri, auth, flags)
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host rv = execute(f, *args, **kwargs)
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host six.reraise(c, e, tb)
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host rv = meth(*args, **kwargs)
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/dist-packages/libvirt.py", line 105, in openAuth
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host if ret is None:raise libvirtError('virConnectOpenAuth() failed')
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
2016-12-05 09:48:17.096 1920 ERROR nova.virt.libvirt.host
2016-12-05 09:48:17.252 1920 INFO nova.virt.libvirt.driver [req-c2a6d49a-ffa9-4364-b5a0-c9dd58a36f45 b9cb3442c99a4d9fb37d821f378f9329 22e36611b69e412ab6a49e4c453336f3 - - -] Connection event '0' reason 'Failed to connect to libvirt'
2016-12-05 09:48:17.267 1920 ERROR nova.virt.libvirt.host [req-c2a6d49a-ffa9-4364-b5a0-c9dd58a36f45 b9cb3442c99a4d9fb37d821f378f9329 22e36611b69e412ab6a49e4c453336f3 - - -] Connection to libvirt failed: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
:: 로그검색
root@XXX-XXX-XXX:/var/log# grep -r libvirtd *
apport.log:ERROR: apport (pid 25365) Mon Dec 5 10:54:28 2016: executable: /usr/sbin/libvirtd (command line "/usr/sbin/libvirtd -d -l")
apport.log:ERROR: apport (pid 25365) Mon Dec 5 10:54:33 2016: wrote report /var/crash/_usr_sbin_libvirtd.0.crash
installer/syslog:Oct 25 05:50:37 user-setup: adduser: The group `libvirtd' does not exist.
installer/cdebconf/templates.dat:Default: adm cdrom dip lpadmin plugdev sambashare debian-tor libvirtd lxd
kern.log:Dec 5 10:54:28 hl-217-112 kernel: [1025731.929562] libvirtd[1694]: segfault at ca37 ip 00007effe0ab7acb sp 00007ffc59e0b610 error 4 in libc-2.23.so[7effe09d2000+1bf000]
syslog:Dec 5 10:54:28 hl-217-112 libvirtd[1694]: internal error: End of file from monitor
syslog:Dec 5 10:54:28 hl-217-112 kernel: [1025731.929562] libvirtd[1694]: segfault at ca37 ip 00007effe0ab7acb sp 00007ffc59e0b610 error 4 in libc-2.23.so[7effe09d2000+1bf000]
syslog:Dec 5 10:57:50 hl-217-112 libvirtd[26052]: libvirt version: 1.3.1, package: 1ubuntu10.5 (Stefan Bader <stefan.bader@canonical.com> Thu, 06 Oct 2016 13:07:20 +0200)
syslog:Dec 5 10:57:50 hl-217-112 libvirtd[26052]: hostname: hl-217-112
syslog:Dec 5 10:57:50 hl-217-112 libvirtd[26052]: 모니터 소켓 연결 실패: 그런 프로세스가 없음
syslog:Dec 5 10:57:50 hl-217-112 libvirtd[26052]: 지원되지 않는 설정: Security driver apparmor not enabled
syslog:Dec 5 10:59:40 hl-217-112 libvirtd[26052]: internal error: End of file from monitor
syslog:Dec 5 10:59:40 hl-217-112 libvirtd[26052]: Cannot open log file: '/var/log/libvirt/qemu/instance-0000002b.log': 장치나 자원이 동작 중
syslog.1:Dec 5 03:25:10 hl-217-112 libvirtd[1694]: iohelper reports:
syslog.1:Dec 5 03:25:10 hl-217-112 libvirtd[1694]: Cannot open log file: '/var/log/libvirt/qemu/instance-0000002b.log': 장치나 자원이 동작 중
:: livbirtd 데몬 죽어있음....
# ps uax |grep libvirtd
# service libvirt-bin start
root 26052 0.3 0.0 1381972 30524 ? Sl 10:57 0:00 /usr/sbin/libvirtd -d -l
:: 특정 인스턴스 상태 변경시 에러발생 --> DB 수동으로 업데이트
MariaDB [nova]> update instances set vm_state='active' where display_name='VM_22' AND vm_state='error';
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0
