In the last few years, the installation of a large number of cameras has led to a need for increased capabilities in video surveillance systems. It has, indeed, been more and more necessary for human operators to be helped in the understanding of ongoing activities in real environments. Nowadays, the technology and the research in the machine vision and artificial intelligence fields allow one to expect a new generation of completely autonomous systems able to reckon the behaviors of entities such as pedestrians, vehicles, and so forth. Hence, whereas the sensing aspect of these systems has been the issue considered the most so far, research is now focused mainly on more newsworthy problems concerning understanding. In this article, we present a novel method for hypothesizing the evolution of behavior. For such purposes, the system is required to extract useful information by means of low-level techniques for detecting and maintaining track of moving objects. The further estimation of performed trajectories, together with objects classification, enables one to compute the probability distribution of the normal activities (e.g., trajectories). Such a distribution is defined by means of a novel clustering technique. The resulting clusters are used to estimate the evolution of objects' behaviors and to speculate about any intention to act dangerously. The provided solution for hypothesizing behaviors occurring in real environments was tested in the context of an outdoor parking lot