Investigating qualitative and quantitative validity of PSYCHLOPS: a novel Patient Reported Outcome Measure in a pilot study of primary care management of insomnia

PSYCHLOPS (‘Psychological Outcome Profiles’) is an idiographic patient reported outcome measure that attempts to evaluate therapeutic progress. We aimed to assess the validity of PSYCHLOPS within a study of insomnia.

Methods

We investigated reliability, content, criterion and construct validity of PSYCHLOPS as part of a cluster randomized pilot study. Other measures used included the Pittsburgh Sleep Quality Index, Insomnia Severity Index, Beck Depression Inventory and sleep diaries. We administered PSYCHLOPS in a study seeking to educate primary care clinicians to deliver problem focused and patient centred therapy for insomnia with adults. The instrument was administered in intervention and control primary care sites. Sixty two participants measuring five and above on the PSQI were included in the study.

Results

The qualitative analysis showed expected responses for sleep problems. We found a positive correlation between PSYCHLOPS and ISI, but not between PSYCHLOPS and PSQI, suggesting partial criterion validity with regards to insomnia impact. We found a positive correlation between PSYCHLOPS and BDI supporting construct validity. However, these validity results rested on a number of assumptions insofar as patients who offered complex free text responses would have to rate their experiences at the beginning, mid way and end of therapy respectively, in order for the tool to create a significant score. This would be difficult for patients to do, which undermines the tool’s validity.

Conclusion

PSYCHLOPS demonstrated aspects of validity supporting further circumspect use and evaluation in practice. A combined qualitative and quantitative analysis added important dimensions to the assessment of validity for this tool. Nonetheless, idiographic measures like PSYCHLOPS provide a much needed patient voice to research and clinical outcome measures.