The lure of the measurable in design research.

Beginning design research projects by defining success criteria, judged by numerical measurements, is a very attractive idea. But defining a priori success criteria is problematic, as is using numerical metrics to assess the success of a new method or computer tool. The paper points out some pitfalls of using metrics for success. It argues from experience of studying design processes that projects should begin with objectives derived from research questions, but these objectives should be revised as needs and opportunities emerge. Success criteria for of new methods and tools should be derived later from a detailed specification of requirements. Researchers should aim first for understanding their effects, and derive evaluations from that.