Modern imaging techniques in medicine have revolutionized the study of human anatomy and physiology. A
central factor in the success of imaging-based approaches has been the emergence of sophisticated computational methods
for extracting salient information from image data. The utility of image processing has prompted the development of numerous
algorithms for medical data, but these have largely remained research tools and few have been incorporated into a
clinical workflow. A primary cause of this poor track record is the lack of validation: safety and accuracy are in fact two
important keywords when dealing with life-critical systems. In particular, these two aspects have to be taken into careful
attention in medical image segmentation. This in fact represents the first step in the process that starts with the image acquisition
and proceeds to the diagnosis step and therapy definition. Therefore it is important to analyze its possible inaccuracy
sources, since they will affect the whole system.
In literature there are several techniques for the performance evaluation of complex systems. However, most of the proposed
approaches in the field of medical image processing only face the problem of defining different metrics allowing to
assess the accuracy from a purely geometrical and quantitative point of view. In this paper it is provided an overview on
the evaluation methods that have been proposed in literature and the advantages and shortcomings of the underlying design
mechanisms are discussed. Finally, possible future directions for research in performance evaluation in medical image
segmentation are proposed.