Need Help Configuring Nagios?

Our tech support team is happy to help you with any questions you might have. Contact us on our online support forum at https://support.nagios.com/forum/

Nagios XI Makes Monitoring Easier:

Nagios XI is the easy-to-use, enterprise version of Nagios that features:

  • Web-Based Configuration provides advanced configuration features
  • Monitoring Wizards make it easy to monitor new devices, applications, and services
  • Customizable Dashboards allow for per-user customization
  • Integrated Performance Graphs provide trending and capacity planning information
  • Advanced Reports provide data insight and exporting capabilities
  • Data Visualizations enable powerful analysis of patterns and problems
  • Nagios Core Import functionality makes it easy to migrate from Nagios Core
  • ... and many other features

Download a free 60-day trial of Nagios XI or give the online demo a spin.

Inquire today and let our Quickstart team help you get started with Nagios XI

Up ToUp To: Contents
See Also See Also: Host Checks, Service Checks, Event Handlers, Notifications

Introduction

The current state of monitored services and hosts is determined by two components:

  • The status of the service or host (i.e. OK, WARNING, UP, DOWN, etc.)
  • Tye type of state the service or host is in

There are two state types in Nagios Core - SOFT states and HARD states. These state types are a crucial part of the monitoring logic, as they are used to determine when event handlers are executed and when notifications are initially sent out.

This document describes the difference between SOFT and HARD states, how they occur, and what happens when they occur.

Service and Host Check Retries

In order to prevent false alarms from transient problems, Nagios Core allows you to define how many times a service or host should be (re)checked before it is considered to have a "real" problem. This is controlled by the max_check_attempts option in the host and service definitions. Understanding how hosts and services are (re)checked in order to determine if a real problem exists is important in understanding how state types work.

Soft States

Soft states occur in the following situations:

  • When a service or host check results in a non-OK or non-UP state and the service check has not yet been (re)checked the number of times specified by the max_check_attempts directive in the service or host definition. This is called a soft error.
  • When a service or host recovers from a soft error. This is considered a soft recovery.

The following things occur when hosts or services experience SOFT state changes:

  • The SOFT state is logged.
  • Event handlers are executed to handle the SOFT state.

SOFT states are only logged if you enabled the log_service_retries or log_host_retries options in your main configuration file.

The only important thing that really happens during a soft state is the execution of event handlers. Using event handlers can be particularly useful if you want to try and proactively fix a problem before it turns into a HARD state. The $HOSTSTATETYPE$ or $SERVICESTATETYPE$ macros will have a value of "SOFT" when event handlers are executed, which allows your event handler scripts to know when they should take corrective action. More information on event handlers can be found here.

Hard States

Hard states occur for hosts and services in the following situations:

  • When a host or service check results in a non-UP or non-OK state and it has been (re)checked the number of times specified by the max_check_attempts option in the host or service definition. This is a hard error state.
  • When a host or service transitions from one hard error state to another error state (e.g. WARNING to CRITICAL).
  • When a service check results in a non-OK state and its corresponding host is either DOWN or UNREACHABLE.
  • When a host or service recovers from a hard error state. This is considered to be a hard recovery.
  • When a passive host check is received. Passive host checks are treated as HARD unless the passive_host_checks_are_soft option is enabled.

The following things occur when hosts or services experience HARD state changes:

  • The HARD state is logged.
  • Event handlers are executed to handle the HARD state.
  • Contacts are notified of the host or service problem or recovery.

The $HOSTSTATETYPE$ or $SERVICESTATETYPE$ macros will have a value of "HARD" when event handlers are executed, which allows your event handler scripts to know when they should take corrective action. More information on event handlers can be found here.

Example

Here's an example of how state types are determined, when state changes occur, and when event handlers and notifications are sent out. The table below shows consecutive checks of a service over time. The service has a max_check_attempts value of 3.

TimeCheck #StateState TypeState ChangeNotes
01OKHARDNoInitial state of the service
11CRITICALSOFTYesFirst detection of a non-OK state. Event handlers execute.
22WARNINGSOFTYesService continues to be in a non-OK state. Event handlers execute.
33CRITICALHARDYesMax check attempts has been reached, so service goes into a HARD state. Event handlers execute and a problem notification is sent out. Check # is reset to 1 immediately after this happens.
43WARNINGHARDYesService changes to a HARD WARNING state. Event handlers execute and a problem notification is sent out.
53WARNINGHARDNoService stabilizes in a HARD problem state. Depending on what the notification interval for the service is, another notification might be sent out.
61OKHARDYesService experiences a HARD recovery. Event handlers execute and a recovery notification is sent out.
71OKHARDNoService is still OK.
81UNKNOWNSOFTYesService is detected as changing to a SOFT non-OK state. Event handlers execute.
92OKSOFTYesService experiences a SOFT recovery. Event handlers execute, but notification are not sent, as this wasn't a "real" problem. State type is set HARD and check # is reset to 1 immediately after this happens.
101OKHARDNoService stabilizes in an OK state.