日期:2014-05-16  浏览次数:20829 次

mysql数据启动大量报错且无法启动(The InnoDB memory heap is disabled,Status: NOT_KILLED)故障排查

mysql数据启动大量报错且无法启动(The InnoDB memory heap is disabled,Status: NOT_KILLED)故障排查

?
周一上班巡检发现公司的有一组mysql主从同步进程报警,登录报警服务器,进行查看排查步骤如下:
?
1,首先登录保障服务器,执行mysql命令登录mysql,发现报错,提示SOCK无法连接详如下:
?
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
?
2.执行 lsof -n -i:3306 查看发现mysql监听端口不在!
?
3. 启动数据库;
/etc/init.d/mysqld start
?
提示启动成功!
?
4.再次查看mysql监听端口,执行 lsof -n -i:3306?
?
查看发现mysql监听端口还是不在!呀郁闷了,这种现象还是第一次见,同时感觉到问题有些严重。
?
5.查看mysql进程,发现此时 mysql 进程竟然还在!!!!!!!!! 真实坑。。。
?
ps -ef |grep mysql ? ??
?
mysql ? ?19143 ? ? 1 ?0 12:26 pts/6 ? ?00:00:01 /mysql/bin/mysqld --basedir=/mysql --datadir=/data/3306/data?
?
6.此时查看一下mysql系统日志?
?
tail -f /var/log/mysqlerror.log?
?
发现mysql在不停打印一下错误日志,篇幅还挺大,详细报错内容如下:
?
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
121210 12:25:40 mysqld_safe Number of processes running now: 0
121210 12:25:40 mysqld_safe mysqld restarted
121210 12:25:41 InnoDB: The InnoDB memory heap is disabled
121210 12:25:41 InnoDB: Mutexes and rw_locks use GCC atomic builtins
121210 12:25:41 InnoDB: Compressed tables use zlib 1.2.3
121210 12:25:41 InnoDB: Initializing buffer pool, size = 2.0G
121210 12:25:41 InnoDB: Completed initialization of buffer pool
121210 12:25:41 InnoDB: highest supported file format is Barracuda.
121210 12:25:41 ?InnoDB: Waiting for the background threads to start
121210 12:25:42 InnoDB: 1.1.8 started; log sequence number 7102612557
121210 12:25:42 [Note] Semi-sync replication initialized for transactions.
121210 12:25:42 [Note] Semi-sync replication enabled on the master.
121210 12:25:42 [Note] Recovering after a crash using mysql-bin
121210 12:25:42 [Note] Starting crash recovery...
121210 12:25:42 [Note] Crash recovery finished.
121210 12:25:42 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
121210 12:25:42 [Note] ? - '0.0.0.0' resolves to '0.0.0.0';
121210 12:25:42 [Note] Server socket created on IP: '0.0.0.0'.
121210 12:25:42 [Warning] Neither --relay-log nor --relay-log-index were used; so replication may break when this MySQL server acts as a slave and has his hostname changed!! Please use '--relay-log=mysql04-relay-bin' to avoid this problem.
121210 12:25:42 [Note] Slave I/O thread: Start semi-sync replication to master 'rsync@172.17.2.13:3306' in log 'mysql-bin.000605' at position 607146584
121210 12:25:42 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000605' at position 607113383, relay log './mysql04-relay-bin.000285' position: 607113529
121210 12:25:42 [Note] Event Scheduler: Loaded 0 events
121210 12:25:42 [Note] /mysql/bin/mysqld: ready for connections.
Version