MongoDB StartUp Issues with
"systemctl status mongod.service" and "journalctl -xe"
We recently had issue where one of our replicaset member went offline for some time and fell behind in syncing.
To come around the situation, we decide the clean out the node and start the service so that it will sync automatically from other replicaset members.
However after cleaning out the data directories the mongo refused to start.
sh-4.2$ pbrun /etc/init.d/mongod start
Starting mongod (via systemctl): Job for mongod.service failed because the control process exited with error code. See "systemctl status mongod.service" and "journalctl -xe" for details. [FAILED]
-sh-4.2$ pbrun /etc/init.d/mongod status
â mongod.service - SYSV: Mongo is a scalable, document-oriented database.
Loaded: loaded (/etc/rc.d/init.d/mongod)
Active: failed (Result: exit-code) since Fri 2016-11-11 10:09:20 MST; 2min 49s ago
Docs: man:systemd-sysv-generator(8)
Process: 6552 ExecStop=/etc/rc.d/init.d/mongod stop (code=exited, status=0/SUCCESS)
Process: 10514 ExecStart=/etc/rc.d/init.d/mongod start (code=exited, status=1/FAILURE)
Main PID: 2156 (code=exited, status=0/SUCCESS)
Nov 11 10:09:20 hqiqlmemdb01 systemd[1]: Starting SYSV: Mongo is a scalable, document-oriented database....
Nov 11 10:09:20 hqiqlmemdb01 runuser[10522]: pam_unix(runuser:session): session opened for user mongod by (uid=0)
Nov 11 10:09:20 hqiqlmemdb01 runuser[10522]: pam_unix(runuser:session): session closed for user mongod
Nov 11 10:09:20 hqiqlmemdb01 mongod[10514]: Starting mongod: [FAILED]
Nov 11 10:09:20 hqiqlmemdb01 systemd[1]: mongod.service: control process exited, code=exited status=1
Nov 11 10:09:20 hqiqlmemdb01 systemd[1]: Failed to start SYSV: Mongo is a scalable, document-oriented database..
Nov 11 10:09:20 hqiqlmemdb01 systemd[1]: Unit mongod.service entered failed state.
Nov 11 10:09:20 hqiqlmemdb01 systemd[1]: mongod.service failed.
-
While checking the mongod.conf file, I spoted the mongodb.key value. This file was there for authentication.
security:
authorization: enabled
keyFile: /data/mongodb/pqmrsme001/mongodb.key
While cleaning up Data directory, we also wiped out that file. so I copied the file back from backup and placed it in data directory.
After that process started just fine.
-sh-4.2$ pbrun /etc/init.d/mongod start
Starting mongod (via systemctl): [ OK ]
-sh-4.2$ pbrun /etc/init.d/mongod status
mongod.service - SYSV: Mongo is a scalable, document-oriented database.
Loaded: loaded (/etc/rc.d/init.d/mongod)
Active: active (running) since Fri 2016-11-11 10:28:44 MST; 6s ago
Docs: man:systemd-sysv-generator(8)
Process: 6552 ExecStop=/etc/rc.d/init.d/mongod stop (code=exited, status=0/SUCCESS)
Process: 11733 ExecStart=/etc/rc.d/init.d/mongod start (code=exited, status=0/SUCCESS)
Main PID: 11745 (mongod)
CGroup: /system.slice/mongod.service
11745 /usr/bin/mongod -f /etc/mongod.conf
