Creyos has made several new improvements to the Creyos Health report. These updates are meant to answer two important questions that will help make interpreting assessment results even easier:
- Validity Indicator: How do I know if a patient's performance on a task is likely to be valid?
- Meaningful Change Indicator: How can I tell if a change in a patient's results is potentially significant?
Keep reading for additional details. If you have any questions, or would like a personalized walkthrough of any of these updates, please let us know and we'd be happy to assist.
Article Highlights
- Creyos Health reports include a Validity Indicator, which shows whether performance on a task is within reasonable bounds and likely valid, and a Meaningful Change Indicator, which shows whether a change in scores is potentially significant.
- A task score is considered likely valid only when it meets every validity parameter for that task. Double Trouble, for example, has 14 validity conditions.
- Meaningful change is assessed against the patient's first valid score (baseline) and latest valid score. Bounds for every task are quantified from normative data on how performance fluctuates over time.
- Classification terms now use objective language ("Below" or "Above" average) in place of subjective terms such as "Poor" or "Superior."
What's new in the Creyos Health report?
The new features of the upcoming Creyos Health report are highlighted below:
Each of the new features above is detailed in the following sections.
What is the Validity Indicator?
To date, there has been no simple or efficient way to verify that performance on a task truly reflects a patient's cognition at that moment. In rare instances, a patient may not have fully understood the task instructions, or may have been distracted, leading to an inaccurate result.
That's why Creyos Health reports will soon include a Validity Indicator, a new and easy-to-read section that conveys whether performance on a task is within reasonable bounds and therefore likely to be valid, helping you gain confidence that task scores reflect a patient's true cognitive state.
How does the Validity Indicator work?
When a task score does not meet the validity criteria, the result shows as potentially invalid in the statement and trends chart.
How do we determine if a task score is likely to be valid?
Each task has a set of parameters that must all be met for the score to be considered likely valid. For instance, the Digit Span task has the following validity conditions:
| Condition | Requirement |
|---|---|
| Number of attempts | Must be greater than zero |
| Number of correct answers* | Must have one or more correct answers, and fewer than 11 correct answers |
| Test duration* | Must be longer than 40 seconds, but shorter than (or equal to) 362 seconds |
| Max score* | Must be 3 or higher, and less than or equal to 12 |
| Average score* | Must be 3 or higher, and less than or equal to 8.4 |
*99% of task scores in the Cambridge Brain Sciences normative database fall within these bounds.
Other tasks may have some additional criteria. For instance, Double Trouble has 14 different validity conditions, including an analysis of reaction times for each of the different types of problems a patient may encounter during the task.
For more information on how task validity is determined, see the Report Interpretation Guide.
What should be done if a task score is likely to be invalid?
If a report shows a potentially invalid result, it is up to you to evaluate all sources of information at your disposal to decide the appropriate course of action. For instance, you may have a patient complete an assessment that includes three memory tasks, with only one of those memory tasks showing as potentially being invalid. In this case, you may choose to forego re-assessing the patient. However, if all of the memory tasks are flagged as potentially being invalid, you may ask the patient to complete the assessment a second time.
As another example, perhaps you are conducting a standard four-task screen as part of your intake process, in which each of the four tasks targets a different cognitive domain. In this case, even if only one of the tasks returns a "potentially invalid" result, you may wish to ask your patient to repeat either the full assessment, or the one task that was determined likely to be invalid.
These scenarios are examples only, and every situation should be evaluated individually.
What is the Meaningful Change Indicator?
As you likely know, Creyos Health can be a valuable tool to assess changes in cognition over time. However, distinguishing between noise and a meaningful change (e.g., a deterioration resulting from a clinical condition, or perhaps an improvement due to a successful treatment protocol) can be difficult. That's why starting today, you'll be able to objectively determine whether a change in an individual's cognitive performance is significant relative to their latest valid score and first valid score (i.e., their baseline).
How does the Meaningful Change Indicator work on Creyos Health reports?
There are two specific questions the Creyos Health report will now answer as it relates to meaningful change:
- Has there been a potentially significant change relative to the patient's baseline assessment?
- Has there been a potentially significant change since the last time the patient was assessed?
On the report, you'll see meaningful change indicated in the results statement for the task.
What are the conditions for determining if a change is likely meaningful?
The vast Creyos normative database allows us to characterize how performance on every task fluctuates over time across a range of testing intervals. Using this data, the bounds of a meaningful change for every task have been quantified and implemented into Creyos Health.
Further details regarding what constitutes a meaningful change for each task are included in the Report Interpretation Guide.
Other report improvements
Further to the updates outlined above, we have implemented several other usability enhancements based on your feedback. They include the following:
- Updated raw score trends graph, which now includes dates and percentile ranks for each data point.
- New classification criteria terminology, utilizing more objective language ("Below" or "Above" average), as opposed to the subjective terms used previously (i.e., "Poor" or "Superior").
- Updated task descriptions, which now include how the task outcome measures may relate to common everyday activities (e.g., example activities related to Number Ladder include planning the day's errands, then carrying out those errands in the correct order by memory).
Share your feedback
These enhancements are a direct result of the feedback from, and discussions with, the Creyos Health community. As always, if you have any other comments or questions, we are more than happy to assist and look forward to hearing from you.
Written by Mike Battista, Director of Science & Research at Creyos
Mike Battista specializes in brain health, cognition, and neuropsychological testing. He received his PhD in personality and measurement psychology at Western University in 2010 and has been doing fun and useful stuff in the intersection between science and technology ever since.