2.1. Agents and Goals
The approach for representing and reasoning about maintenance goals described in this paper, although generic, is most suited to Belief-Desire-Intention (BDI) agents (Rao and Georgeff 1991, 1995). Beliefs represent the information an agent has about itself and the environment (potentially including information about other agents). For example, a soccer playing robot may have the belief that it is located 10 m from the ball, that the ball is located 5 m from the goalkeeper, and that the goalkeeper has seen the ball. From these beliefs, it may determine that it cannot get to the ball before the goalkeeper does. Desires represent states the agent would like to have brought about. For example, the soccer playing robot desires the ball to be in the opponent’s goal. Intentions act as commitments to realizing a particular desire. Many practical agent systems may represent intentions as being goals with plans. A plan represents some method of achieving a goal, and may be represented in many forms. When an agent selects a goal for pursuit, a plan will be instantiated which, when executed, should lead to the satisfaction of the goal.
Goals are similar to desires. They represent states of the world an agent would like to see brought about. A key difference is that the desires of an agent may be inconsistent with one another, where as goals are required to be consistent. The most common type of goals require a particular state of the world to be achieved, termed achievement goals (or perform goals where success is not checked). When an agent achieves a goal, it is aiming to reach a particular state in the world where some condition is satisfied. However, an agent may also aim to keep a particular state true. In the event that the state no longer holds, an agent with a maintenance goal acts to restore that condition. For example, a soccer robot may aim to keep the ball away from the opponent.
A maintenance goal is appropriate in situations such as safety, or where repeated action may be necessary. For example, a moible robot (Pokahr, Braubach, and Lamersdorf 2005a) may use a maintenance goal to ensure that its battery’s charge is always greater than 10%. When this is no longer the case, the maintenance goal activates and causes the robot to find the closest recharger and recharge.
Unlike an achievement goal, a maintenance goal is long-lived, in that it will not be dropped upon success. Success for a maintenance goal is the continued satisfaction of the condition it is maintaining.
There are a variety of ways that this may be achieved, but they can all be described as maintaining a particular state.
One simple approach to maintaining a state is through the use of guarded actions. A guarded action is often employed in conjunction with another goal, for example, an achievement goal. The purpose of a guarded action is to stop the achievement goal in the event that some condition (the guard condition) occurs.
If an agent aims to maintain a state, then this state can be used as the guard. While performing other actions in pursuit of the achievement goal, this guarded state should persist. In the event that it does not, the achievement goal is aborted. For example, (fuel > 10, moveTo(Location10)), indicates that an agent should attempt to achieve the goal moveTo(Location10) so long as the guard condition, fuel > 10 is satisfied. In the event that the guard condition no longer holds, the agent abandons the moveTo(Location10) goal.
This approach achieves a similar effect to maintenance goals, but at the cost of requiring the agent designer to embed the appropriate checks for all maintenance goals into every part of the code where an action may occur. This would result in significantly more complex code, as well as the inconvenience of having to update large sections of the code when a maintenance goal changes. Hence we believe that it is impractical to use guarded actions as a substitute for maintenance goals.
An alternative to guarded actions are reactive maintenance goals. A reactive maintenance goal monitors a maintenance condition, and when this condition is no longer true, only then does it cause the agent to perform some actions, which are intended to make the maintenance condition true once more. If the maintenance condition is never violated, then the reactive maintenance goal will not influence the behavior of the agent.
In some ways, reactive maintenance goals are very similar to achievement goals. Both types of goals are triggered and cause actions to be performed. The important difference between these two forms of goal is that the maintenance goal is not dropped once the maintenance condition has been restored, unlike an achievement goal which is dropped once the desired state has been obtained.
Practical agent systems such as Jadex support reactive maintenance goals. When some maintenance condition is violated, action is triggered, which in turn may lead to the suspension of other goals (and in this way provide behavior similar to that found with guarded actions). Once the maintenance condition is restored, the suspended goals may be resumed. For example, monitoring the battery level in a mobile robot could be realized with a maintenance goal. A reactive maintenance goal to perform such a task would involve having a condition that triggers the maintenance goal if the battery level was less than 10% (for example). Upon activation, the maintenance goal would cause the robot to stop what it was doing and recharge at the nearest base-station. However, this behavior is only triggered when a certain value is reached—it does not take into consideration the current actions or goals of the agent. This can lead the agent to perform inefficiently, or potentially even becoming stranded.
Consider if the agent was located 1m from a base station and its fuel was around 15%. It would not act to recharge at this point. If it was given the task of traveling some long distance, it may trigger the maintenance goal before the journey is complete, causing it to return to the base station. This is an example of inefficient behavior associated with reactive maintenance goals.
One possible improvement to maintenance goals would be to have maintenance goals behave proactively—rather than waiting for the condition to no longer be satisfied, and then reacting to restore it, it may be better to act before the condition becomes unsatisfied.
Hindriks and van Riemsdijk (2007) provide one approach to proactive maintenance goals, which utilized a method similar to a planning mechanism. If an agent determined that its currently selected course of action would cause a violation of one or more of its maintenance goals, it would abort that course of action. Thus, it proactively avoids its maintenance goals from being violated. This does not provide any preventative mechanism however, other than simply not pursuing goals that cause the violation of maintenance conditions. One improvement to this would allow an agent to introduce actions that would allow it to achieve its goals, while ensuring its maintenance conditions remain satisfied (e.g., forcing the agent to refuel its tank to allow it to complete a journey without violating a maintenance condition).
Kaminka Yakir, Erusalimchik, and Cohen-Nov (2007) suggest a multiagent approach to proactive maintenance goals. A team maintenance goal may be to ensure that the distance between two mobile robots never exceeded a certain amount. Proactively, the robots could determine where other robots were heading and thus determine if the distance would exceed its maintenance conditions. If so, the robots would alter their plans to avoid this from occurring. The work presented in this paper focuses on maintenance goals for a single agent. However, we believe that our findings are applicable to many multiagent domains.
Van Riemsdijk et al. (2008) address the issue of the representation and behavior of goals in agent systems. Rather than providing descriptions of various goal types, this work presented a generic representation of goal, which was suitable for a large variety of goals. However, it is noted that the representation presented is insufficient to enabled maintenance goals to be treated in a proactive manner.
The work in this paper builds upon the framework by van Riemsdijk et al. (2008) to provide maintenance goals that exhibit reactive and proactive behavior.
The KAOS goal-oriented requirements engineering framework (Darimont, Delor, Massonet, and van Lamsweerde 1997) includes maintenance goals, as well as avoid goals. Whereas a maintenance goal aims to keep some condition satisfied, the purpose of an avoid goal is to not let some condition become satisfied. By modeling an avoid goal with a maintenance goal (by negating the condition to maintain), it becomes even more apparent how important it is to take the proactive approach concerning maintenance goals. If an agent wishes to avoid a condition from occurring, it will more than likely need to act in advance (i.e., behave proactively) rather than rely only on a reactive approach, particularly for safety-critical applications, or when legal requirements need to be met.
Nakamura, Baral, and Bjäreland (2000), Baral and Eiter (2004), and Baral et al. (2008) focus on defining exactly the behavior of a maintenance goal, utilizing temporal operators. They identify that the temporal operators, always f, is too strong. This does not describe the behavior of a maintenance goal, where it is expected that maintenance conditions may fail, which then need to be repaired. Further, in many cases, it is impossible for an agent to exhibit such a high degree of control over the environment to guarantee always f.
An alternative proposed by Nakamura et al. (2000) is that of always eventually f. This encodes that if f becomes false at any point, it will eventually be (re)satisfied. This encoding is also dismissed by Nakamura et al. as being too strong. It is possible that an agent may be overwhelmed with requests such that it cannot restore the condition it is aiming to maintain. An example presented by Nakamura et al. is that of an agent that monitors a user’s inbox, with the maintenance goal of keeping it empty. The environment is adversarial, keeping the inbox full and thus falsifying the maintenance goal. The agent removes and processes each email at a slower rate than the environment sends email. Therefore, despite the best efforts of the agent, it is unable to maintain the condition of keeping the inbox full. Yet, this behavior would still be described as rational, and attempting to maintain its goals.
Nakamura et al. (2000), Baral and Eiter (2004), and Baral et al. (2008) also define the notion of k-maintainability with respect to maintenance goals. k-maintainability describes the window of opportunity required in order for an agent to perform a number of actions that would restore the maintenance condition. A period less than k does not guarantee that an agent can maintain the desired condition, while given at least k occasionally, will result in the agent attempting to restore the maintenance condition. Nakamura et al. (2000), Baral and Eiter (2004), and Baral et al. (2008) go on to represent and solve the problem of determining k-maintainable controls using a SAT encoding, which is then proven to be polynomial time, and linear time for small k.
2.2. Maintenance Goals in Practical Agent Systems
In this section, we discuss some of the popular families of agent platforms, and how goals, in particular, maintenance goals, behave and are represented in these frameworks.
Due to their shared heritage, the PRS family (PRS, dMARS, and JACK) share many similar notions concerning goals. In this family, goals are not explicitly represented, but are implicitly captured via events. When a particular event is received by the agent, plan selection occurs and a plan appropriate to this event is then pursued. In the default behavior, if the plan begin pursued fails, an alternate plan is selected and the process repeats. The BDI-gap described by Winikoff et al. (2002) is partially a result of not having an explicit notion of goal.
While achievement goals are the most common form of goals, the PRS family includes several alternate goal types including maintain and query goals. The behavior of a maintain goal is to trigger an event (goal) when the maintain condition does not hold during the attempted achievement of the sub-goal. Upon completion of the sub-goal, the agent drops this maintenance goal. Hence, the maintain goal is dynamic, in that it may be adopted and dropped during runtime. If the maintain condition becomes unsatisfied during the execution of the sub-goal, the sub-goal is aborted and recovery actions may be pursued. However, the original goal is dropped, and must be explicitly re-adopted if the agent wishes to continue pursuit of this goal.
The JAM Agent language by Huber (1999) builds upon the UMPRS (Lee et al. 1994) and PRS (Georgeff and Ingrand 1989) implementations of the PRS agent framework. It supports a variety of goal types, including Achieve, Perform, and Maintain. However, the form of maintain goal represented in JAM is reactive in nature.
An example of a JAM agent described by Huber (1999) can be found in Figure 1. After executing the initialize action (which has the highest utility and is thus selected in preference to all other goals), the agent continually performs the wander_lobby action, ensuring that its charge level is always greater than 20% and keeping a safe distance from obstacles.
While wandering the lobby, an agent with this goal set will disregard the maintenance goal of ensuring that its charge level is greater than 20%. It is only when this condition no longer holds, that this maintain goal come into effect and cause the activation of a plan that restores this condition. This is therefore a reactive maintenance goal.
Jadex is a Java-based agent language. Originally built upon JADE (Bellifemine, Poggi, and Rimassa 1999), a FIPA (O’Brien and Nicol 1998) compliant framework for hosting and developing multiagent systems, Jadex provides a framework for developing BDI-based agent systems. Jadex provides a comprehensive collection of goal types, including achieve, maintain, perform and query. Further details of Jadex’s goal types can be found in a comprehensive discussion by Pokahr, Braubach, and Lamersdorf (2003). This discussion also provides considerable detail concerning the representation of goals in the Jadex language.
Any goal in the Jadex framework is in any one of three states at any one time.
This corresponds to the case when this goal can be selected for pursuit, but has not yet been made active. This loosely corresponds to the concept of desire, in that it is a goal the agent would like to pursue, but is currently not taking action towards. This allows goals that are in the Option state to be conflicting.
Once an agent decides to take action towards achieving a goal, the goal is then considered Active. This indicates that the agent is taking steps towards realizing this goal.
A suspended goal indicates a goal that had been Active, but for some reason, cannot be allowed to continue. It is therefore moved to the Suspended state. After a certain condition is met, indicating the goal is once again applicable, it is moved to the Option state where it may be selected when the agent deems it appropriate.
When a maintenance goal is adopted by an agent, it begins in the Option state. In general, it becomes Active when there is no other Active goal that conflicts with it.
Once Active, it monitors particular beliefs of the agent, triggering a plan when its condition is no longer satisfied. In the event that the agent deems that the condition is no longer possible to maintain, the maintenance goal can be moved to Suspended, pending a possible change to the Option state when the maintenance goal is once again applicable.
Goals are explicitly represented in Jadex, which enables more complex deliberation when compared with some other agent families. However, there are no provisions for reasoning about maintenance goals other than simple inhibition links, as discussed by Pokahr et al. (2003). Much like the behavior found in JAM agents, the (default) behavior of JADEX maintenance goals is to act as a trigger to action when the goal’s maintenance condition is no longer satisfied.
We now move to discussing some of the logic-based agent frameworks.
2.2.1. AgentSpeak and Jason One of the origins of AgentSpeak(L) (Rao 1996) is to provide a mechanism for overcoming the “BDI gap,” caused by the fact that practical implemented agent systems often diverged from the theoretical approach.
AgentSpeak(L) aims to formalize the operation of existing practical agent languages, that being PRS (and to some extent, its successor, dMARS) which had “lacked a strong theoretical underpinning”(Rao 1996). It achieves this by representing much of the BDI model in a first-order logical language, containing events and actions.
In Rao (1996), only two forms of goal are considered: achieve-goals, which are the common form of goal found in almost all agent platforms, and test-goals, which allows an agent to determine if it a particular formula is true or false relative to its belief set.
Achievement goals have a context related to them, which must be satisfied before the body can be executed. This context could be utilized to prevent actions from being executed—it does not support any method by which alternate actions could be performed if necessary.
Jason (Bordini and Hübner 2005) is a Java-implementation of AgentSpeak(L), which supports triggering events, to indicate when a plan should be executed. Utilizing this notion could provide support for a form of reactive maintenance goal in Jason.
These rules (and an associated plan that is not listed here) act to cause the agent to believe that its battery is not charged when the level is less than 20%. Dropping this belief can be used to trigger a goal or plan, which is used to direct the agent to recharge. Once the battery level reaches 100%, the second rule causes the adoption of the belief, batterycharged, which can be used to stop the recharging plan.
This behavior is completely reactive, and thus Jason (and AgentSpeak(L)) share the same limitation expressed in discussions concerning other agent systems.
2.2.2. 3APL, GOAL, and Dribble The 3APL family of agent programming languages (Hindriks et al. 1999; Dastani, Dignum, and Meyer 2000) utilize constructs such as an agent’s beliefs and goals, in conjunction with a set of ‘practical reasoning rules’ which revise an agent’s goal set. 3APL also has facilities for creating and modifying plans during the execution of an agent.
An initial extension called GOAL had the planning ability found in 3APL removed, but can use declarative goals in selecting actions to perform. This was to address its inability to determine if goals were completed by alternate means, and to enhance an agent’s ability to reason over its goals.
A later extension called Dribble (van Riemsdijk, van der Hoek, and Meyer 2003) aims to consolidate the procedural and declarative aspects of the agent languages GOAL and 3APL. It features the ability to plan with declarative goals, meaning that it can (in theory) perform more complex reasoning when compared with the original 3APL. However, Dribble is limited in that it is a propositional language (and so excludes variables), limiting its practical use (Dastani et al. 2003).
Incorporating the extensions developed in Dribble, 3APL was extended to include declarative goals and first-order features.
Representing maintenance goals in 3APL can be represented by means of practical reasoning rules to activate certain goals when its maintenance conditions are violated. Maintenance goals can be represented in GOAL with some small modifications (Hindriks and van Riemsdijk 2007), and act mainly to constrain the actions available to an agent, and so, can be considered proactive. However, to achieve proactivity, the agent requires a look-ahead operator (that potentially needs infinite look-ahead) to determine the consequences of its actions. This limits the usefulness of such an approach for practical agents, but it is a useful approach for analysis.
This approach was then extended (Hindriks and van Riemsdijk 2008) to further include support for distinguishing between hard and soft constraints, and preferences, that allow an agent developer to define which goals to pursue in favor of others. This rational action selection architecture (or RASA) realize hard constraints with maintenance goals, and soft constraints via preferences.