Server fails after installing certbot; MPM run failed, exiting
I set up a new server running Ubuntu 18.04 with 4GB of RAM with Apache2,PHP,MariaDB with Fail2ban and OSSEC. When I installed certbot to add an SSL cert from Let's Encrypt, Apache2 crashes.
Even when I restart the server, Apache will still crash.
Apache error log:
[Wed Jun 27 06:45:25.613316 2018] [mpmprefork:notice] [pid 3547] AH00163: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.0g configured -- resuming normal operations
[Wed Jun 27 06:45:25.613342 2018] [core:notice] [pid 3547] AH00094: Command line: '/usr/sbin/apache2'
[Wed Jun 27 06:45:31.656454 2018] [mpmprefork:emerg] pid 7203Identifier removed: AH00144: couldn't grab the accept mutex
[Wed Jun 27 06:45:31.656663 2018] [mpmprefork:emerg] pid 7202Identifier removed: AH00144: couldn't grab the accept mutex
[Wed Jun 27 06:45:31.659122 2018] [mpmprefork:emerg] pid 7201Identifier removed: AH00144: couldn't grab the accept mutex
[Wed Jun 27 06:45:32.623283 2018] [core:alert] [pid 3547] AH00050: Child 7201 returned a Fatal error… Apache is exiting!
[Wed Jun 27 06:45:32.623371 2018] [:emerg] [pid 3547] AH02818: MPM run failed, exiting
[Wed Jun 27 08:06:41.048660 2018] [mpmprefork:notice] [pid 1436] AH00163: Apache/2.4.29 (Ubuntu) OpenSSL/1.1.0g configured -- resuming normal operations
[Wed Jun 27 08:06:41.050591 2018] [core:notice] [pid 1436] AH00094: Command line: '/usr/sbin/apache2'
[Wed Jun 27 08:16:57.484364 2018] [mpmprefork:emerg] pid 1586Identifier removed: AH00144: couldn't grab the accept mutex
[Wed Jun 27 08:16:57.484407 2018] [mpmprefork:emerg] pid 1449Identifier removed: AH00144: couldn't grab the accept mutex
[Wed Jun 27 08:16:57.487201 2018] [mpmprefork:emerg] pid 1625Identifier removed: AH00144: couldn't grab the accept mutex
[Wed Jun 27 08:16:57.487605 2018] [mpmprefork:emerg] pid 1447Identifier removed: AH00144: couldn't grab the accept mutex
[Wed Jun 27 08:16:57.489578 2018] [mpmprefork:emerg] pid 1446Identifier removed: AH00144: couldn't grab the accept mutex
[Wed Jun 27 08:16:57.489626 2018] [mpmprefork:emerg] pid 1631Identifier removed: AH00144: couldn't grab the accept mutex
[Wed Jun 27 08:16:57.492276 2018] [mpmprefork:emerg] pid 1630Identifier removed: AH00144: couldn't grab the accept mutex
[Wed Jun 27 08:16:57.636986 2018] [core:alert] [pid 1436] AH00050: Child 1446 returned a Fatal error… Apache is exiting!
[Wed Jun 27 08:16:57.637053 2018] [:emerg] [pid 1436] AH02818: MPM run failed, exiting
What OSSEC says:
[:emerg] [pid 1436] AH02818: MPM run failed, exiting
[core:alert] [pid 1436] AH00050: Child 1446 returned a Fatal error… Apache is exiting!
system systemd[1]: apache2.service: Failed with result 'exit-code'.
system systemd[1]: apache2.service: Main process exited, code=exited, status=1/FAILURE
Syslog error report:
Jun 27 08:16:45 system systemd[1]: Created slice User Slice of [username redacted].
Jun 27 08:16:45 system systemd[1]: Starting User Manager for UID 1000…
Jun 27 08:16:45 system systemd[1]: Started Session c2 of user [username redacted].
Jun 27 08:16:45 system systemd[1646]: Listening on GnuPG cryptographic agent and passphrase cache (access for web browsers).
Jun 27 08:16:45 system systemd[1646]: Listening on GnuPG network certificate management daemon.
Jun 27 08:16:45 system systemd[1646]: Listening on GnuPG cryptographic agent (ssh-agent emulation).
Jun 27 08:16:45 system systemd[1646]: Reached target Timers.
Jun 27 08:16:45 system systemd[1646]: Listening on GnuPG cryptographic agent and passphrase cache (restricted).
Jun 27 08:16:45 system systemd[1646]: Reached target Paths.
Jun 27 08:16:45 system systemd[1646]: Listening on GnuPG cryptographic agent and passphrase cache.
Jun 27 08:16:45 system systemd[1646]: Reached target Sockets.
Jun 27 08:16:45 system systemd[1646]: Reached target Basic System.
Jun 27 08:16:45 system systemd[1646]: Reached target Default.
Jun 27 08:16:45 system systemd[1646]: Startup finished in 78ms.
Jun 27 08:16:45 system systemd[1]: Started User Manager for UID 1000.
Jun 27 08:16:57 system systemd[1]: Stopping User Manager for UID 1000…
Jun 27 08:16:57 system systemd[1646]: Stopped target Default.
Jun 27 08:16:57 system systemd[1646]: Stopped target Basic System.
Jun 27 08:16:57 system systemd[1646]: Stopped target Sockets.
Jun 27 08:16:57 system systemd[1646]: Closed GnuPG cryptographic agent (ssh-agent emulation).
Jun 27 08:16:57 system systemd[1646]: Closed GnuPG cryptographic agent and passphrase cache (restricted).
Jun 27 08:16:57 system systemd[1646]: Closed GnuPG network certificate management daemon.
Jun 27 08:16:57 system systemd[1646]: Stopped target Paths.
Jun 27 08:16:57 system systemd[1646]: Closed GnuPG cryptographic agent and passphrase cache.
Jun 27 08:16:57 system systemd[1646]: Stopped target Timers.
Jun 27 08:16:57 system systemd[1646]: Closed GnuPG cryptographic agent and passphrase cache (access for web browsers).
Jun 27 08:16:57 system systemd[1646]: Reached target Shutdown.
Jun 27 08:16:57 system systemd[1646]: Starting Exit the Session…
Jun 27 08:16:57 system systemd[1646]: Received SIGRTMIN+24 from PID 1764 (kill).
Jun 27 08:16:57 system systemd[1]: Stopped User Manager for UID 1000.
Jun 27 08:16:57 system systemd[1]: Removed slice User Slice of [username redacted].
Jun 27 08:16:57 system systemd[1]: apache2.service: Main process exited, code=exited, status=1/FAILURE
Jun 27 08:16:57 system systemd[1]: apache2.service: Failed with result 'exit-code'.
2 Replies
This may be a bug that sometimes occurs with Apache, though that's just speculation without knowing more about your setup. There's a common fix, however. In the following file:
/etc/apache2/apache2.conf
Try setting the following directive:
Mutex posixsem
After making any changes be sure to restart Apache:
service apache2 restart
If you're still running into trouble, try purging Apache and it's configuration files and reinstalling. I'd recommend taking backups of any configurations in case you need to reference them or restore them later.
apt-get purge apache2