Salt Stack for monitoring

I just have to believe Salt Stack is a great low-overhead and simple way to monitor servers.

One way you can start with this is by using the PsUtil interface module that comes with Salt now.

I had come across this slide deck. It seemed to involved for my liking, but something about the Salt Scheduler and this mention of commands that start with “ps.” caught my eye.

Well, Salt has a built-in module that will return various PsUtil outputs. You have to have PsUtil installed on each minion though… specifically the Python version. Here’s how I did it on CentOS 6.5:

yum install python-psutil.x86_64

you could do this concurrently on all Salt minions by typing this on the salt master:

salt ‘*’ cmd.run ‘python-psutil.x86_64 -y’

Or, you could make a global Salt State that applies to all minions that looks something like this (I put this in a folder called globalpackageinstalls/init.sls):

python-psutil.x86_64:

pkg.installed:

– pkg_verify: True

and then in your top.sls you would put something like this:

base:

‘*’:

– globalpackageinstalls

Whatever the case, once the python version of psutils is installed, you can run any of the commands for all minions at once. Making use of the “Returners” feature should allow you to get the results into something like ElasticSearch.

salt ‘*’ ps.cpu_percent

salt ‘*’ ps.disk_partition_usage

This is great and all, but alerting is sort of what you really really want in order to be proactive. I’m not sure how to do that *easily*. But maybe alerts are never really all that easy to set up and manage… bleh