Lish is unresponsive, but I can SSH in

Linode Staff

When I load the Lish console, I see some messages from when my Linode booted, but I can't enter any input, and it doesn't show a login prompt.

However, I can login via SSH and run commands normally there. My services are otherwise working normally.

How can I troubleshoot the Lish issue? I'm running Ubuntu 16.04.

1 Reply

It sounds like the getty for the serial console for your Linode isn't running. Here's some background on gettys and serial consoles:

About getty

getty is short for 'get tty', and it is a process that manages connections to a terminal (also known as a tty):

getty (Unix)

Your server can have different terminals. On Linode, Lish accesses the serial console/terminal for your server, and the serial console for KVM Linodes is labelled ttyS0.

If you run the following command from your SSH connection, you should see the getty process on ttyS0:

root@localhost:~ ps aux | grep  [t]tyS0
root      3325  0.0  0.2  15756  2144 ttyS0    Ss+  06:20   0:00 /sbin/agetty --keep-baud 115200 38400 9600 ttyS0 vt220

Your serial console getty lets you invoke the login process on that terminal. As well, if you enter your username (but not the password) at the Lish console and then run ps aux | grep [t]tyS0 on your separate SSH connection, you will see that the getty process disappears, and in its place is the login process running on ttyS0:

root@localhost:~ ps aux | grep  [t]tyS0
root      3325  0.0  0.3  68744  3500 ttyS0    Ss+  06:20   0:00 /bin/login --

This is because getty exits and hands off control to the login process. After a successful login, a bash shell is run on ttyS0. If the user exits the bash shell or the login fails, the getty process will automatically respawn.

The init system for your deployment is responsible for setting up a getty process. Ubuntu 16.04 uses systemd to start and manage your services.

On systemd, there are two ways to set up the serial console:

  1. Pass console=ttyS0 as an option to the kernel when booting. Linode does this for you when you use the kernel provided by our Manager. Or, you can configure your boot loader (Grub) to pass this option. systemd should then automatically invoke /lib/systemd/system/serial-getty@.service with this console.

  2. You can manually enable the serial console:

systemctl enable serial-getty@ttyS0.service

These subjects are also covered in the following article:

Gettys on Serial Consoles (and Elsewhere)

Troubleshooting your unresponsive Lish console

From your SSH connection, you should check to see if a getty is running on ttyS0:

ps aux | grep [t]tyS0

If you don't see any such processes, then this is probably why Lish is unresponsive.

You can also search your system logs and look for related messages. The following output is what you would normally see when Lish is responding as expected after booting your Linode:

root@localhost:~ journalctl -b | grep ttyS0
May 07 06:20:25 localhost kernel: Command line: root=/dev/sda console=tty1 console=ttyS0 ro  devtmpfs.mount=1
May 07 06:20:25 localhost kernel: Kernel command line: root=/dev/sda console=tty1 console=ttyS0 ro  devtmpfs.mount=1
May 07 06:20:25 localhost kernel: console [ttyS0] enabled
May 07 06:20:25 localhost kernel: 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
May 07 06:20:26 localhost systemd[1]: Found device /dev/ttyS0.
May 07 06:20:27 localhost systemd[1]: Started Serial Getty on ttyS0.

You can check to see if your serial-getty@.service is enabled:

root@localhost:~ systemctl list-unit-files | grep serial-getty@.service
serial-getty@.service                      enabled

If this is disabled, you can try manually enabling it:

systemctl enable serial-getty@ttyS0.service
systemctl start serial-getty@ttyS0.service

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct