Difference between revisions of "Monitoring (moved)"
Jump to navigation
Jump to search
imported>Paulproteus |
imported>Mdaniel (deprecation process in favor of sphinx and readthedocs) |
||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| + | {{Hacking OpenHatch}} |
||
| − | {{Hacking_OpenHatch}} |
||
| + | We are updating our documentation system. This page is now included in our project package, and is automatically generated by sphinx at openhatch.readthedocs.org [[http://openhatch.readthedocs.org/en/latest/internals/monitoring.html Monitoring] |
||
| − | == Monitoring == |
||
| − | |||
| − | === The basics=== |
||
| − | * <code>linode.openhatch.org</code> is the main OpenHatch box, which runs the website. |
||
| − | * <code>linode2.openhatch.org</code> is the secondary server for OpenHatch. It hosts the Hudson continuous integration server, as well as Nagios! |
||
| − | * The Nagios configuration is owned by a user called ''nagios'' on ''linode2.openhatch.org''. |
||
| − | |||
| − | === Access === |
||
| − | |||
| − | * We use ssh keys for login. |
||
| − | * If you want SSH access to that account, file a bug requesting it, and attach an SSH key. You should hear back within 2 days; if you don't hear back by then, try to find paulproteus or jesstess on IRC. |
||
| − | * Then you can do: |
||
| − | |||
| − | ssh nagios@linode2.openhatch.org |
||
| − | |||
| − | * You'll know it's working if you are logged in. If you see a "Password:" prompt, then it is not working. |
||
| − | |||
| − | === Notifications === |
||
| − | |||
| − | * Nagios notifications go to [http://lists.openhatch.org/mailman/listinfo/monitoring monitoring@lists.openhatch.org]. Anyone can subscribe to this list or read its archives. |
||
| − | |||
| − | === Making changes === |
||
| − | |||
| − | In brief, here's what you need to know: |
||
| − | |||
| − | * Edit files in ~nagios/ |
||
| − | * Once you know what changes you want to make, create a local branch with those changes: |
||
| − | git checkout -b my_changes |
||
| − | * As you make changes, make meaningful commits. Also, tell "git commit" to use your identity: |
||
| − | git commit --author="Some Body <some.body@example.com>" |
||
| − | * After you have made the changes, ask someone to review them and merge the changes to ''master''. |
||
| − | * '''Rationale''': If you stick to the above process, it is fairly easy to roll back to the "master" branch of the Nagios configuration. |
||
| − | * '''History''': We came up with this process during [https://openhatch.org/bugs/issue332 issue332]. |
||
| − | |||
| − | === Viewing the web interface, and handling the daemon === |
||
| − | |||
| − | * On <code>linode2</code>, <code>~nagios/secrets/</code> contains the mailman and Nagios web interface passwords. |
||
| − | * View the Nagios web interface at <code>http://linode2.openhatch.org/nagios3/</code> |
||
| − | * To restart the Nagios daemon, run |
||
| − | |||
| − | sudo /etc/init.d/nagios3 restart |
||
| − | |||
| − | ===In case of emergency=== |
||
| − | |||
| − | * See [[Emergency operations for the openhatch server]]. People with ssh keys set up for the Linode Shell (Lish) can reboot the box and have other limited emergency capabilities. |
||
| − | |||
| − | == TODOs == |
||
| − | |||
| − | # Send Nagios notifications to IRC (<code>#openhatch-auto</code>?)? |
||
| − | # Make the Nagios web interface world-viewable. |
||
| − | # Version the monitoring configurations. |
||
| − | # Send SMS alerts to people who want them. |
||
| − | # Add historical trending (Munin)? |
||
| − | |||
| − | == Related == |
||
| − | |||
| − | * See also [[Emergency operations for the openhatch server]] |
||
| − | * See also the page about the [[Login team]] |
||
Latest revision as of 16:41, 25 June 2012
This is a page about improving or modifying OpenHatch.
We call that "Hacking OpenHatch," and there is a whole category of pages about that.
We are updating our documentation system. This page is now included in our project package, and is automatically generated by sphinx at openhatch.readthedocs.org [Monitoring