Summary: \BeginparA new approach to software reliability estimation is presented that combines operational testing with stratified sampling in order to reduce the number of program executions that must be checked manually for conformance to requirements. Automatic cluster analysis is applied to execution profiles in order to stratify captured operational executions. Experimental results are reported that suggest this approach can significantly reduce the cost of estimating reliability.\Endpar (Provider: ACM) Review: \BeginparMuch work on software testing focuses on synthetic or debug testing, namely, techniques for revealing faults in a program. Much less work has been done so far on operational or acceptance testing{—}techniques for estimating software reliability. This paper makes an important contribution to this second class of problems. It presents an interesting application of stratified sampling to operational testing. The paper clearly presents classical sampling techniques for estimating population parameters, discusses how the techniques can be used to efficiently sample program executions for measuring software reliability, and presents experimental evidence of the validity of the proposed approach. The paper is very well written: nonexperts will be able to understand it with reasonable effort, despite the technical details of the presentation. The paper presents the results honestly and scientifically. It clearly identifies the hypotheses and their implications. It aptly marks the bounds of the techniques and discusses the costs of their introduction.\Endpar \BeginparAn interesting side benefit of the paper is its clarification of domain-based testing and its relation to random testing. The authors present several aspects that will interest the scientific community, in that they will learn how a well-known technique such as stratified sampling can be applied to the domain of operational testing. It may also interest practitioners, who can better understand and tune acceptance testing.\Endpar (Provider: ACM)