NSLU2-Linux
view · edit · print · history

The Beta release of OpenSlug supports netconsole. This kernel module copies all the helpful messages the kernel outputs over the network to another system running a syslog daemon capable of recieving network logging.

This means that if OpenSlug dies (this is a beta release) with a kernel panic it is possible to see what happened. If you do not run netconsole there will be no record by default.

There are other ways of keeping a permanent record on a remote machine using syslog - simply edit /etc/syslog.conf to use remote as the DESTINATION and set the host name. netconsole has the advantage of starting earlier in the system boot and the disadvantage that it only logs kernel messages. This page only describes how to set up netconsole.

What you need - preparation steps

You must have a system that supports remote syslog. This is pretty much standard on UN*X systems, however it is normally disabled by default. If you have a Windows system, check http://www.kiwisyslog.com/ for a freeware syslog daemon (untested). To switch on remote logging, check the man page for syslogd (e.g. man 8 syslogd) and search for the -r option. (If you're using syslog-ng, check the bottom of this page). Use ps to see if the currently running syslogd has the relevant option, or look in the configuration files. If necessary change the system startup so that syslogd is alway run with this option.

The machine you use should have a static IP address, otherwise you will have to alter the slug configuration every time the remote machine IP changes. One way round this is to use a more sophisticated startup on the slug, but you cannot supply a host name to the kernel module - it will use an apparently random address.

On the same machine obtain the information for the network interface:

  1. Run ifconfig. This lists the interfaces. Ignore lo, the real interface will probably be called eth0 or something like that.
  2. Find the entry for inet addr, record the IP address (if you need to, you probably know this already.)
  3. Find the hardware address of the interface. This is a set of six hexadecimal numbers separated by :, for example 00:30:1B:B3:67:C1 ifconfig output should include it as HWaddr? probably on the first line for the interface.
  4. Find the service (port) number of the syslog service on your computer. Look in /etc/services for syslog, it will have a line for syslog and a decimal number followed by a protocol (udp). It will probably be 514. It it isn't in there it probably needs to be added.

You should have three pieces of information, IP address, hardware address and a port (or service) number. If there is some reason why you can't configure syslog appropriately it is also possible to run netcat on the machine:

netcat -u -l -p 6666

netcat may be called nc (Debian and others) or nc6 (gentoo). This command uses 6666 as the port number (this happens to be the default netconsole uses!).
For fedora core the command to use is:

nc -u -l 6666

How to set it up, step by step.

netconsole is now installed by default in OpenSlug and UcSlugC images. On very old builds you may need to install the netconsole module from the feed. Then ssh in to your slug and install netconsole. It's a kernel module so:

ipkg install kernel-module-netconsole

Now just modprobe the netconsole module into the kernel with the appropriate parameters:

modprobe netconsole netconsole=@/,<port>@<ip>/<hardware>

For example:

modprobe netconsole netconsole=@/,514@192.168.1.19/00:30:1B:B3:67:C1

Run logread to make sure that worked:

logread

The tail of the log should have seven lines from netconsole giving all the information you just entered. If there is a problem it will be reported there - see the last section.

Go to the remote machine and make sure that it gets the message. It should immediately report the netconsole startup with a copy of the lines from the slug. If not double check the slug lines closely - something is probably wrong. See the last section.

Making it permanent

One you have a modprobe line which works make it permanent by entering it into the slug system configuration. Create a new configuration file /etc/rcS.d/S41netconsole and make it a shell script with just the modprobe line and an exit. Something like:

#!/bin/sh
modprobe netconsole netconsole=@/,514@192.168.1.19/00:30:1B:B3:67:C1
exit 0

Make sure it is executable:

chmod +x /etc/rcS.d/S41netconsole

Now netconsole will start on every boot, just after the network comes up.

Troubleshooting

The output of logread should be sufficient to track down problems on the slug. If it ends with network logging started the slug believes everything is ok. To debug remote problems change the netconsole parameters. To do that re-insert the module by removing it:

rmmod netconsole

Then repeat the modprobe command with the different parameters.

If the remote machine does not respond try removing the hardware address:

modprobe netconsole netconsole=@/,514@192.168.1.19/

The disadvantage of this is that netconsole then uses the broadcast hardware address, which causes extra load on other machines.

If it still doesn't work try netcat, use a different port - use 6666 which is the default. On the remote machine, using the correct name for netcat (nc6 on my gentoo machine, nc on others (like Fedora)):

nc6 -u -l -p 6666

You can run that as a regular user, not root. Then on the slug:

rmmod netconsole
modprobe netconsole netconsole=@/,@192.168.1.19/

You just need the IP address!! This should just work. You don't have to run the netcat all the time - netconsole does not care if the log messages are not received - however if the logger is not running when the kernel crashes there is no way of getting the messages.

Other configurations

If you need to netcat from a windows box, try this url for some hints Netcat - The TCP/IP Swiss Army Knife - http://m.nu/program/util/netcat/netcat.html

Configuration options that work with syslog-ng (Syslog-ng doesn't use -r, like classic syslogd) source src { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); unix-stream("/chroot/dhcp/dev/log"); udp(ip("192.168.1.2") port(514)); };

view · edit · print · history · Last edited by repvik.
Based on work by eFfeM, jbowler, repvik, jstueve, and Hannes.
Originally by jbowler.
Page last modified on February 05, 2006, at 06:37 PM