Graphs for my Linode.

The status API isn't really intended for regular polling (in fact, it has been specifically requested that people NOT poll it every few minutes). Your best bet would be to use one of the myriad of stats gathering systems out there. I use munin, but there's plenty of choices. This will also give you the advantage of tracking statistics that are actually useful, e.g. memory usage, swap vs. regular disk I/O, etc.

Thanks, I'll check into that.

Actually, the graphs I'm most interested in is CPU Utilization and Host Load; I'm using my Linode for streaming audio, and while I've done the math, I'd like to see how well my streaming system performs whilst also being a good neighbor on the host.

Is there any way, with Xen, to get the host load for your Linode's host?

A thought:

/proc/loadavg

Is this the loadavg for my specific linode, or the loadavg for the xen host? Is there a way to get the loadavg for the host from /proc? If so that would do exactly what I want on the top of my admin pages: a bar graph of Linode load and a bar graph of Host Load.

Is there a non-apche-based Munin style tool? It looks like Munin requires Apache (according to the installation instructions I found), and I'm running Lighttpd. I'd rather not have to install both, and I need some lighttpd features.

There's no reason you'd need apache to run munin. The munin "collector" process just generates a set of png images and html pages. Should be no problem to get running under lighttpd.

Easy question first: The munin server just spits out a bunch of HTML and PNG files every 5 minutes, which can then be statically served by a web server of your choice. Easy breezy. The parts that interface with the web server are statistics-gathering plugins on the munin client (aka munin-node); obviously, if you don't run Apache, you won't be using the Apache plugins :-)

shameless plug: http://github.com/rtucker/munin-lighttpd

Harder question: The strange thing about Xen is there is no host. On the old UML platform, there was a clear and obvious delineation between the "guests" (Linodes) and the "host" (the thing running the ~40 Linodes as individual processes). The closest we get to that is "dom0", which is, essentially, a guest with some additional privileges above and beyond the normal "domU" guests.

In practice, the dom0 instance is the "gatekeeper" between the domU instances and the hardware, specifically the disks and the network interfaces. This is from where the Linode platform does all of the stuff it needs to do, like launching Linodes, setting up virtual network interfaces, managing disk I/O priorities, etc.

Unlike the UML platform, however, the dom0 doesn't have the guests "within" it. Under UML, the host load meter was a decent metric of what was going on for the whole host, since each Linode was its own process and thus impacted the load average, showed in "top" and "ps", etc. With Xen, the host load meter just looks at the dom0's performance, which is pinned at "idle" unless there's a lot of platform-related activity going on (deploys, backups, mikegrb's seti@home clients, etc).

In other words, anything you can do from within your Linode is unlikely to get that needle above "idle", so monitoring it will be really boring.

And yup, /proc/loadavg is just your Linode's load average. Remember, your kernel has almost no idea it's virtualized. Heck, with the newer/experimental paravirtualized kernels, it's so realistic your clock will drift unless you run ntpd. (Weird, huh?)

@hoopycat:

With Xen, the host load meter just looks at the dom0's performance, which is pinned at "idle" unless there's a lot of platform-related activity going on (deploys, backups, jsmith's distributed.net clients, etc).
FTFY

I installed Munin according to the instructions here:

http://en.gentoo-wiki.com/wiki/Munin

Except for the Apache specific things.

But my /munin/ directory… just never gets anything in it. I've given it an hour. When I run:

/usr/bin/munin-cron

I get:

No data available from munin-update yet at /usr/libexec/munin/munin-html line 158.

It seems I missed something important? Any ideas?

Duhh… I fixed it, I think.

The step I skipped was:

sudo -u munin munin-node-configure --shell

Which is kinda important.

EDIT: It looks like the cron job is never run. Any ideas why?

I attempted to install your lighttpd plugin, hoopycat. I installed it into /usr/libexec/munin/plugins, set it as executable, added the symlinks per the comment into /etc/munin/plugins, run munin-update, run munin-cron… but I don't see anything about lighttpd in the munin graphs (which are now working).

ALSO NOTE: You have to install corefonts in Gentoo to get the graphs to display.

I'm not too sure with the Gentoo way of it… under Ubuntu, it's just "apt-get install munin" on the server and "apt-get install munin-node" on the client and all the cron job stuff just happens. The cron job looks like it just calls munin-cron every 5 minutes.

There's also munin.conf, which tells the server (that's munin-cron) where the clients are (that's munin-node). Ideally, it'll ship with 127.0.0.1 in there, but you never know…

I believe you're the first person other than me to try the lighttpd plugin :-) Some quick troubleshooting…

1) You did restart munin-node, right?

2) telnet localhost 4949, then "list". You should see lighttpdaccesses, among others, in there. "config lighttpdaccesses" should return a block of text starting with graphtitle, and "fetch lighttpdaccesses" should return "data.value something".

This works in general for all Munin plugins, and is a good troubleshooting step when adding/modifying new plugins.

(Apologies if I'm missing something -- in a bit of a hurry, which is never a good condition in which to write a forum post)