EVE-NG - unable to login to web interface - SQL server does not start

[mathewfer]

Hi

I am unable to login to EVE-NG comm edition web interface and I see root HD is nto full but SQL server has errors

Any idea how to fix this?

root@eve-ng:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.9G 0 3.9G 0% /dev
tmpfs 799M 15M 785M 2% /run
/dev/mapper/eve--ng--vg-root 110G 76G 30G 72% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/sda1 472M 83M 365M 19% /boot
root@eve-ng:~#

root@eve-ng:~# /etc/init.d/mysql start
[....] Starting mysql (via systemctl): mysql.serviceJob for mysql.service failed because the control process exited with error code. See "systemctl status mysql.service" and "journalctl -xe" for details.
failed!
root@eve-ng:~#

root@eve-ng:~# apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
libegl1-mesa libgbm1 libgl1-mesa-dri libwayland-egl1-mesa linux-headers-generic
linux-headers-virtual linux-image-virtual linux-virtual
0 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
2 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Setting up mysql-server-5.7 (5.7.24-0ubuntu0.16.04.1) ...
Job for mysql.service failed because the control process exited with error code. See "systemctl stat us mysql.service" and "journalctl -xe" for details.
invoke-rc.d: initscript mysql, action "start" failed.
* mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Wed 2018-10-24 10:00:48 EEST; 6ms ago
Process: 25783 ExecStartPost=/usr/share/mysql/mysql-systemd-start post (code=exited, status=0/SUCC ESS)
Process: 25782 ExecStart=/usr/sbin/mysqld (code=exited, status=2)
Process: 25774 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCES S)
Main PID: 25782 (code=exited, status=2)

Oct 24 10:00:48 eve-ng systemd[1]: Failed to start MySQL Community Server.
Oct 24 10:00:48 eve-ng systemd[1]: mysql.service: Unit entered failed state.
Oct 24 10:00:48 eve-ng systemd[1]: mysql.service: Failed with result 'exit-code'.
dpkg: error processing package mysql-server-5.7 (--configure):
subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of mysql-server:
mysql-server depends on mysql-server-5.7; however:
Package mysql-server-5.7 is not configured yet.

dpkg: error processing package mysql-server (--configure):
dependency problems - leaving unconfigured
Errors were encountered while processing:
mysql-server-5.7
mysql-server
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@eve-ng:~#

[Uldis (UD)]

all simple you had screwed your SQL DB.
Usually it is happening when you had powered off PC while VM ware and eve VM was running...
There is solution but please visit EVE Live chat...
http://www.eve-ng.net/live-helpdesk

Use google account or create new for join

[mathewfer]

Hi,

I have Google account/username same as the login name to this forum but I can not login to live-help desk. I can login to Google mail but not to your live-help desk. The login error says "User not found or incorrect password".

Can I please know the process to get access to live-help desk?

Mathew

[Uldis (UD)]

Live Helpdesk is not the same account as it is here.
Use your google email logon for chatnot this account.

Or you can create new chat account..

UD

[mathewfer]

Hi UD,

I tried multiple times and multiple PCs but I can not login to helpdesk. My Google ID works if I directly login to Gmail but not through LiveHelpdesk.
This happens after I normal server shutdown - graceful shutdown and powered up gain - unable to logon to EVE.

is there way to that you can let me know the steps to recover the SQL server to login?

Please help me.

See the errors;

root@eve-ng:~# tail -55 /var/log/mysql/error.log
InnoDB: End of page dump
2018-10-30T23:26:53.338513Z 0 [Note] InnoDB: Uncompressed page, stored checksum in field1 2910029977, calculated checksums for field1: crc32 2910029977/955636607, innodb 2283032670, none 3735928559, stored checksum in field2 1878234640, calculated checksums for field2: crc32 2910029977/955636607, innodb 208590535, none 3735928559, page LSN 0 30522823, low 4 bytes of LSN at page end 30112653, page number (if stored to page already) 48, space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a system page
2018-10-30T23:26:53.338523Z 0 [ERROR] [FATAL] InnoDB: The page in the doublewrite buffer is corrupt. Cannot continue operation. You can try to recover the database with innodb_force_recovery=6
2018-10-31 01:26:53 0x7fde0f38e740 InnoDB: Assertion failure in thread 140591714854720 in file ut0ut.cc line 942
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/ ... overy.html
InnoDB: about forcing recovery.
23:26:53 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.

key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=0
max_threads=151
thread_count=0
connection_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 76387 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x3b)[0xe93b9b]
/usr/sbin/mysqld(handle_fatal_signal+0x489)[0x78a8e9]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fde0dee4390]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7fde0d29d428]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7fde0d29f02a]
/usr/sbin/mysqld[0x7601d0]
/usr/sbin/mysqld(_ZN2ib5fatalD1Ev+0x145)[0x10dd345]
/usr/sbin/mysqld(_Z17buf_dblwr_processv+0x1152)[0x1126772]
/usr/sbin/mysqld(_Z35recv_recovery_from_checkpoint_startm+0x2536)[0xfada46]
/usr/sbin/mysqld(_Z34innobase_start_or_create_for_mysqlv+0x38cb)[0x1084aab]
/usr/sbin/mysqld[0xf45538]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x51)[0x7d80c1]
/usr/sbin/mysqld[0xc71ab5]
/usr/sbin/mysqld(_Z40plugin_register_builtin_and_init_core_sePiPPc+0x318)[0xc74ed8]
/usr/sbin/mysqld[0x782cef]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0xa49)[0x7846d9]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fde0d288830]
/usr/sbin/mysqld(_start+0x29)[0x77ab99]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
root@eve-ng:~#

root@eve-ng:~# journalctl -xe
Oct 31 01:52:18 eve-ng mysqld[9498]: mysqld: [ERROR] Fatal error in defaults handling. Program aborted!
Oct 31 01:52:18 eve-ng systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
Oct 31 01:52:18 eve-ng kernel: audit: type=1400 audit(1540943538.730:129): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/9498/sta
Oct 31 01:52:18 eve-ng kernel: audit: type=1400 audit(1540943538.730:130): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/s
Oct 31 01:52:18 eve-ng kernel: audit: type=1400 audit(1540943538.730:131): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/9498/sta
Oct 31 01:52:48 eve-ng systemd[1]: Failed to start MySQL Community Server.
-- Subject: Unit mysql.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/li ... temd-devel
--
-- Unit mysql.service has failed.
--
-- The result is failed.
Oct 31 01:52:48 eve-ng systemd[1]: mysql.service: Unit entered failed state.
Oct 31 01:52:48 eve-ng systemd[1]: mysql.service: Failed with result 'exit-code'.
Oct 31 01:52:48 eve-ng systemd[1]: mysql.service: Service hold-off time over, scheduling restart.
Oct 31 01:52:48 eve-ng systemd[1]: Stopped MySQL Community Server.
-- Subject: Unit mysql.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/li ... temd-devel
--
-- Unit mysql.service has finished shutting down.
Oct 31 01:52:48 eve-ng systemd[1]: Starting MySQL Community Server...
-- Subject: Unit mysql.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/li ... temd-devel
--
-- Unit mysql.service has begun starting up.
Oct 31 01:52:48 eve-ng mysql-systemd-start[9648]: my_print_defaults: [ERROR] Found option without preceding group in config file /etc/mysql/my.cnf at line 24
Oct 31 01:52:48 eve-ng mysql-systemd-start[9648]: my_print_defaults: [ERROR] Fatal error in defaults handling. Program aborted!
Oct 31 01:52:48 eve-ng audit[9655]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/9655/status" pid=9655 comm="mysqld" request
Oct 31 01:52:48 eve-ng audit[9655]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/system/node/" pid=9655 comm="mysqld"
Oct 31 01:52:48 eve-ng audit[9655]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/9655/status" pid=9655 comm="mysqld" request
Oct 31 01:52:48 eve-ng mysqld[9655]: mysqld: [ERROR] Found option without preceding group in config file /etc/mysql/my.cnf at line 24!
Oct 31 01:52:48 eve-ng mysqld[9655]: mysqld: [ERROR] Fatal error in defaults handling. Program aborted!
Oct 31 01:52:48 eve-ng systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
Oct 31 01:52:48 eve-ng kernel: audit: type=1400 audit(1540943568.978:132): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/9655/sta
Oct 31 01:52:48 eve-ng kernel: audit: type=1400 audit(1540943568.978:133): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/s
Oct 31 01:52:48 eve-ng kernel: audit: type=1400 audit(1540943568.978:134): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/9655/sta



Mathew

[Uldis (UD)]

not only google account can be used for chat.
Any other email as well.
Let me know your username in the chat I will check it, activate.
Do not mess chat with this forum logins, they are diff.
Uldis

[Uldis (UD)]

here you are. It is dangerous commands and can end for you with EVE re install.

#######stop sql processess###########
systemctl stop mysql

#########remove lock for libs##############
rm -f /var/lib/dpkg/lock

##############hard erase of sql and related packages################
export TERM=xterm
apt-get --purge autoremove $(dpkg -l | grep 'mysql' | awk '{print $2}')

When prompted, confirm with yes

rm -rf /var/lib/mysql

rm -rf /etc/mysql


##########Install eve-ng again##############

apt-get install eve-ng

[mathewfer]

Hi UD,

Thank you for your reply and the steps.

Your steps are followed exactly and I could get it to work now. I can login and run all my LABs successfully.
I also backup up the below 3 folders, in case, it runs into a crash and to re-build the server.

/opt/unetlab/addons/
/opt/unetlab/labs/
/opt/unetlab/tmp/

Mathew