Cautionary Tales: Lessons Learned About Unanticipated Behaviors in OC Systems

Abstract

Organic computing (OC) recognizes that the behaviors of deployed systems can be much more interesting than those predicted by simulation. By exploiting self-X properties such as self-organization, self-optimization, self-protection, and context-awareness, OC researchers are developing methods for creating robust, trustworthy systems. This paper presents two examples of unanticipated behaviors that we have observed in our OC test bed of robotic toy cars, an unmodeled phenomenon with surprisingly large effects and an agent behavior that was believed (incorrectly) to be so rare that it could be ignored. We discuss the use of computational reflection as a tool for identifying such situations, consider the challenges posed by the large variety of such situations faced by real systems, and list lessons learned about the importance of test beds for advancing OC research.