We are interested in the development of computer systems that are embedded in the physical or virtual world and that are able to interact autonomously with their environments in an intelligent way. Mobile robots are well-suited to investigate the fundamental challenges of such systems. The key feature of mobile robots is that they embrace all the characteristics of these systems. They have various types of sensors to perceive their environment and they have actuators to change the environment and their position therein. This makes them an ideal tool for developing and testing fundamental techniques needed in intelligent embedded systems. In addition to this, mobile robots themselves will become an important part of our everyday lives. Particularly promising application areas are service robots operating in indoor environments, where they perform cleaning, surveillance and delivery tasks, mobile robots in the health-care sector (e.g. helpmate), where they can be utilized as intelligent wheel-chairs (e.g. UT wheelchair) or personal assistants for the elderly (e.g. Pearl), and mobile robots for education and entertainment (e.g. museum tour-guides, AIBO).
One of the major preconditions for the success of mobile robot systems is the ability to reliably and efficiently act in unstructured and dynamic environments. The key problems in mobile robot navigation arise from the inherent uncertainty in sensing and acting. Over the last years, there has been substantial progress in the development of reliable mobile robot systems. Much of this progress can be attributed to the use of probabilistic methods for dealing with the uncertainties involved in robot navigation. The development of rich yet efficient methods for representing uncertainty and for reasoning under uncertainty has mainly contributed to the success of these systems. Most recently, the introduction of particle filters as a powerful tool for state estimation in mobile robotics has produced another leap in the performance of existing methods. Concurrently with the progress in probabilistic methods, the issue of cooperation between multiple mobile robots has gained increased interest in the robotics community. This interest has been amplified by the RoboCup Challenge, which puts a strong emphasis on the development of teams of collaborating mobile robots.
Despite the tremendous potential benefits of multi-robot collaboration, fundamental issues in the context of probabilistic methods for multi-robot collaboration remain virtually unexplored. A major reason for the lack of substantial progress in this area is the fact that the complexity of probabilistic state estimation and planning problems often grows exponentially in the number of robots. There remains a scientific gap between the success of probabilistic methods in single-robot systems and their successful application to collaborative multi-robot systems: Even though the formulation of optimal solutions to problems involving multiple robots is often possible, it is by far not well understood how they can be solved in an efficient and approximate, yet satisfactory way. It is a major goal of our research to bridge this gap. We will develop probabilistic representations and reasoning mechanisms that allow mobile robots to collaborate and that are efficient enough to fulfill the real-time requirements imposed on teams of mobile robots acting in the real world. We consider the RoboCup Challenge as a unique test bed to develop and evaluate techniques for multi-robot collaboration.
Another key precondition for the success of mobile robot systems is their ability to adapt to changes in their environments and to improve their performance over time. These circumstances create both challenging problems and opportunities for machine learning techniques. Approaches such as reinforcement learning are among the most promising methods for dealing with the extremely complex interactions between mobile robots and their environments. It is due to this complexity and the fact that mobile robot tasks often involve longterm planning and scheduling, however, that reinforcement learning has made only limited contributions to the success of most mobile robot navigation systems. We expect that this will change dramatically due to the RoboCup Challenge and especially the Sony legged robot league. Robot soccer requires the development of a rich repertoire of navigation skills under the constraints of limited world models, limited processing power, and noisy sensors. Reinforcement learning techniques are well-suited to develop such skills, especially when they can be trained in relatively controlled settings. Reinforcement learning can also be applied at higher levels of abstractions, such as the discovery of better policies for switching between actions/behaviors.
To summarize, our research focuses on the development of efficient probabilistic methods for multi-robot systems and on the application of machine learning techniques to improve the efficiency of such systems.
The major components of a robot control system for the RoboCup
Challenge are navigation and ball-handling, state estimation, and
coordination and communication. In this section we will outline our
approach to realizing the individual components.
Previous Sony legged robot soccer competitions showed that good navigation and ball-handling skills are of utmost importance for the success of a robot team. These basic skills can be trained off-line in relatively controlled settings and we will apply reinforcement learning techniques to optimize the robots' behaviors.
State estimation is another important component of the control system, as can be seen from the RoboCup literature. By state estimation we do not only think of the estimating a robot's position within the soccer field, but all sorts of information the robots can collect. This information includes, for example, the relative position of the ball and the other robots. State estimation consists of two sub-problems. First, it is the extraction of extract valuable information from the robot's sensors (mostly vision in this case), and, second, it is the combination of this information over time to achieve a good estimate of the current state.
The first problem will be addressed by applying established techniques from the computer vision community and by adapting them to the specific circumstances of the Sony legged robot league (such as limited processing power). Furthermore, we intend to profit from the extensive experience of other Sony legged robot teams. Once a basic vision system is established, we will apply machine learning techniques to increase the performance of the system.
We will make extensive use of Monte Carlo methods in order to efficiently address the second part of the state estimation problem, namely the integration and representation of information over time. Our Monte Carlo method for mobile robot localization (MCL) has already been applied with great success in several mobile robot systems (including RoboCup). We recently introduced a method for collaborative multi-robot localization. This approach has been shown to significantly improve the localization performance of teams of mobile robots. However, it is based on the assumption that the robots can detect each other and can transfer information (sample sets in this case) about their relative positions. While the first assumption might be reasonable even in the Sony legged robot league, it is not feasible to transfer complete sample sets from one robot dog to the other. Therefore, we will extract compact, symbolic information from Monte Carlo sample sets to be able to transfer this information using the robots' limited communication capabilities. We believe that information extraction from sub-symbolic, real-valued state variables is a very promising field for future research with many potential applications.
An important precondition for successful online coordination of a team of soccer playing robots is that they have estimates about their relative positions. Therefore, this component strongly depends on solutions to the problem of state estimation. Communication can help to enhance both the state estimation and the explicit coordination between robots. We will make extensive use of the communication capabilities of the robots. For example, the goal keeper can watch another robot and give it further information about opponents and the ball position. Please note that we are aware of the fact, that this strongly depends on the capabilities of the vision system, the state estimator, the processing power, and the reliability of the communication channel.
Coordination between robots can also be achieved implicitly (i.e. without communication), for example, by assigning the robots different roles depending on their positions on the soccer field. This strategy will be an important part of our overall coordination design. Another interesting research is issue is the trade-off between sensing and acting. Since actions have a strong impact on the quality of sensor information (e.g. camera information is much less reliable as the robot moves), we will develop coordination mechanisms that take into account the utility of sensor information for different situations and behaviors. These coordination mechanisms will be optimized using reinforcement learning.