Difference between revisions of "Monitoring (moved)"

From OpenHatch wiki
Jump to navigation Jump to search
Line 58: Line 58:
* See also [[Emergency operations for the openhatch server]]
* See also [[Emergency operations for the openhatch server]]
* See also the page about the [[Login team]]

Revision as of 19:21, 20 April 2011

This is a page about improving or modifying OpenHatch.

We call that "Hacking OpenHatch," and there is a whole category of pages about that.


The basics

  • linode.openhatch.org is the main OpenHatch box, which runs the website.
  • linode2.openhatch.org is the secondary server for OpenHatch. It hosts the Hudson continuous integration server, as well as Nagios!
  • The Nagios configuration is owned by a user called nagios on linode2.openhatch.org.


  • We use ssh keys for login.
  • If you want SSH access to that account, file a bug requesting it, and attach an SSH key. You should hear back within 2 days; if you don't hear back by then, try to find paulproteus or jesstess on IRC.
  • Then you can do:
ssh nagios@linode2.openhatch.org
  • You'll know it's working if you are logged in. If you see a "Password:" prompt, then it is not working.


Making changes

In brief, here's what you need to know:

  • Edit files in ~nagios/
  • Once you know what changes you want to make, create a local branch with those changes:
git checkout -b my_changes
  • As you make changes, make meaningful commits. Also, tell "git commit" to use your identity:
git commit --author="Some Body <some.body@example.com>"
  • After you have made the changes, ask someone to review them and merge the changes to master.
  • Rationale: If you stick to the above process, it is fairly easy to roll back to the "master" branch of the Nagios configuration.
  • History: We came up with this process during issue332.

Viewing the web interface, and handling the daemon

  • On linode2, ~nagios/secrets/ contains the mailman and Nagios web interface passwords.
  • View the Nagios web interface at http://linode2.openhatch.org/nagios3/
  • To restart the Nagios daemon, run
sudo /etc/init.d/nagios3 restart

In case of emergency


  1. Send Nagios notifications to IRC (#openhatch-auto?)?
  2. Make the Nagios web interface world-viewable.
  3. Version the monitoring configurations.
  4. Send SMS alerts to people who want them.
  5. Add historical trending (Munin)?