Research Article
Poor Concordance Between Remote Neurocognitive Testing and Face-to-Face Testing of Verbal Memory Processes in Young Healthy Volunteers
Abstract
Purpose: Recently published studies indicate increased interest in remote health services that have expanded greatly with digitalization and whether they can be implemented into neuropsychological testing. Remote neurocognitive testing has become a frequently used approach today. The current study examines the concordance between the face-to-face and remote Oktem Verbal Memory Processes Test results of healthy young participants.
Method: 45 healthy young participants with a mean age of 23.41 were invited to participate in the study. Participants were 42.2% of male and 57.8% female. The Öktem Verbal Memory Processes Test was administered to participants. There was at least one week between face-to-face and remote testing. The results were compared with paired t-tests and Lin's Concordance Correlation analysis. Reliability was analyzed by examining the Intraclass Correlation Coefficient.
Results: For immediate memory score, ICC was 0.286 (95% CI: 0.016-0.524).. ICC of total learning score was calculated as 0.665 (95% CI: 0.465 -0.801). The highest learning score had an ICC of 0.868 (95% CI: 0.773 -0.926). ICC of the total inconsistency score was 0.639 (95% CI: 0.414 -0.79). Finally, the score for reaching the criterion displayed an ICC of 0.253 (95% CI: 0.079 -0.535).
Conclusions: The findings demonstrate that there is poor concordance between the results obtained with remote and face-to-face application of the Öktem Verbal Memory Process Test. As a result, using it remotely is not recommended at this stage.
Supporting Institution
None
Project Number
None
Thanks
Thanks to all participants.
References
- Germine L, Reinecke K, Chaytor NS. Digital neuropsychology: Challenges and opportunities at the intersection of science and software. Clin Neuropsychol 2019;33:271–86.
- Monaghesh E, Hajizadeh A. The role of telehealth during COVID-19 outbreak: a systematic review based on current evidence. BMC Public Health 2020;20:1–9.
- Wosik J, Fudim M, Cameron B, Gellad ZF, Cho A, Phinney D, et al. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Informatics Assoc 2020;27:957–62.
- Klein SB. What memory is. Wiley Interdiscip Rev Cogn Sci 2015;6:1–38. https://doi.org/10.1002/WCS.1333.
- Tatsumi IF, Watanabe M. Verbal Memory. Encycl Neurosci 2009:4176–8. https://doi.org/10.1007/978-3-540-29678-2_6266.
- Öktem Ö. Öktem Verbal Memory Processes Test (ÖKTEM-VMPT) Manuel book. Ankara: Turkish Psychologist Society Publish; 2011.
- Pietkiewicz IJ, Helka AM, Tomalski R. Validity and reliability of the Polish online and pen-and-paper versions of the Somatoform Dissociation Questionnaires (SDQ-20 and PSDQ-5). Eur J Trauma Dissociation 2019;3:23–31.
- de Brito Silva B, de Brito A, Monteiro EP, Mondelo GP, Remor E. Evidence of validity for the online version of the Assessment of Adherence to Antiretroviral Therapy Questionnaire. SAGE Open 2019;9:2158244019877201.
- Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15:155–63.
- Steichen TJ, Cox NJ. A note on the concordance correlation coefficient. Stata J 2002;2:183–9.
- McBride GB. A proposal for strength-of-agreement criteria for Lin’s concordance correlation coefficient. NIWA Client Rep HAM2005-062 2005;45:307–10.
- Şair A, Şair YB, Ayar S. Differentiating Mild Forms of Cognitive Impairment and Dementia: Where Other Tests Fail, Verbal Memory Assessment May Prove Critical. Meandros Med Dent J 2021;22:187.
- Moore RC, Swendsen J, Depp CA. Applications for self‐administered mobile cognitive assessments in clinical research: A systematic review. Int J Methods Psychiatr Res 2017;26:e1562.
- Resnick HE, Lathan CE. From battlefield to home: a mobile platform for assessing brain health. Mhealth 2016;2.
- Lajoie AC, Crane J, Robinson AR, Lafontaine AL, Benedetti A, Kimoff RJ, et al. Feasibility of remote neurocognitive assessment: pandemic adaptations for a clinical trial, the Cognition and Obstructive Sleep Apnea in Parkinson’s Disease, Effect of Positive Airway Pressure Therapy (COPE-PAP) study. Trials 2021;22:1–9. https://doi.org/10.1186/S13063-021-05879-1/TABLES/3.
- DeYoung N, Shenal B V. The reliability of the Montreal Cognitive Assessment using telehealth in a rural setting with veterans. J Telemed Telecare 2019;25:197–203. https://doi.org/10.1177/1357633X17752030.
- Mahon S, Webb J, Snell D, Theadom A. Feasibility of administering the WAIS-IV using a home-based telehealth videoconferencing model. Https://DoiOrg/101080/1385404620211985172 2021;36:558–70. https://doi.org/10.1080/13854046.2021.1985172.
- Sumpter R, Camsey E, Meldrum S, Alford M, Campbell I, Bois C, et al. Remote neuropsychological assessment: Acceptability and feasibility of direct-to-home teleneuropsychology methodology during the COVID-19 pandemic. Https://DoiOrg/101080/1385404620222056922 2022:1–16. https://doi.org/10.1080/13854046.2022.2056922.
- Zeghari R, Guerchouche R, Tran-Duc M, Bremond F, Langel K, Ramakers I, et al. Feasibility Study of an Internet-Based Platform for Tele-Neuropsychological Assessment of Elderly in Remote Areas. Diagnostics 2022, Vol 12, Page 925 2022;12:925. https://doi.org/10.3390/DIAGNOSTICS12040925.
- Cullum CM, Hynan LS, Grosch M, Parikh M, Weiner MF. Teleneuropsychology: Evidence for video teleconference-based neuropsychological assessment. J Int Neuropsychol Soc 2014;20:1028–33.
- Ye S, Sun K, Phi H, Hosseini Ghomi R, Ghomi RH. Validation of Remote Testing using BrainCheck, a Computerized Neurocognitive Test. MedRxiv 2020:2020.06.01.20119289. https://doi.org/10.1101/2020.06.01.20119289.
Year 2023,
Volume: 14 Issue: 3, 339 - 344, 10.07.2023
Abstract
Project Number
None
References
- Germine L, Reinecke K, Chaytor NS. Digital neuropsychology: Challenges and opportunities at the intersection of science and software. Clin Neuropsychol 2019;33:271–86.
- Monaghesh E, Hajizadeh A. The role of telehealth during COVID-19 outbreak: a systematic review based on current evidence. BMC Public Health 2020;20:1–9.
- Wosik J, Fudim M, Cameron B, Gellad ZF, Cho A, Phinney D, et al. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Informatics Assoc 2020;27:957–62.
- Klein SB. What memory is. Wiley Interdiscip Rev Cogn Sci 2015;6:1–38. https://doi.org/10.1002/WCS.1333.
- Tatsumi IF, Watanabe M. Verbal Memory. Encycl Neurosci 2009:4176–8. https://doi.org/10.1007/978-3-540-29678-2_6266.
- Öktem Ö. Öktem Verbal Memory Processes Test (ÖKTEM-VMPT) Manuel book. Ankara: Turkish Psychologist Society Publish; 2011.
- Pietkiewicz IJ, Helka AM, Tomalski R. Validity and reliability of the Polish online and pen-and-paper versions of the Somatoform Dissociation Questionnaires (SDQ-20 and PSDQ-5). Eur J Trauma Dissociation 2019;3:23–31.
- de Brito Silva B, de Brito A, Monteiro EP, Mondelo GP, Remor E. Evidence of validity for the online version of the Assessment of Adherence to Antiretroviral Therapy Questionnaire. SAGE Open 2019;9:2158244019877201.
- Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15:155–63.
- Steichen TJ, Cox NJ. A note on the concordance correlation coefficient. Stata J 2002;2:183–9.
- McBride GB. A proposal for strength-of-agreement criteria for Lin’s concordance correlation coefficient. NIWA Client Rep HAM2005-062 2005;45:307–10.
- Şair A, Şair YB, Ayar S. Differentiating Mild Forms of Cognitive Impairment and Dementia: Where Other Tests Fail, Verbal Memory Assessment May Prove Critical. Meandros Med Dent J 2021;22:187.
- Moore RC, Swendsen J, Depp CA. Applications for self‐administered mobile cognitive assessments in clinical research: A systematic review. Int J Methods Psychiatr Res 2017;26:e1562.
- Resnick HE, Lathan CE. From battlefield to home: a mobile platform for assessing brain health. Mhealth 2016;2.
- Lajoie AC, Crane J, Robinson AR, Lafontaine AL, Benedetti A, Kimoff RJ, et al. Feasibility of remote neurocognitive assessment: pandemic adaptations for a clinical trial, the Cognition and Obstructive Sleep Apnea in Parkinson’s Disease, Effect of Positive Airway Pressure Therapy (COPE-PAP) study. Trials 2021;22:1–9. https://doi.org/10.1186/S13063-021-05879-1/TABLES/3.
- DeYoung N, Shenal B V. The reliability of the Montreal Cognitive Assessment using telehealth in a rural setting with veterans. J Telemed Telecare 2019;25:197–203. https://doi.org/10.1177/1357633X17752030.
- Mahon S, Webb J, Snell D, Theadom A. Feasibility of administering the WAIS-IV using a home-based telehealth videoconferencing model. Https://DoiOrg/101080/1385404620211985172 2021;36:558–70. https://doi.org/10.1080/13854046.2021.1985172.
- Sumpter R, Camsey E, Meldrum S, Alford M, Campbell I, Bois C, et al. Remote neuropsychological assessment: Acceptability and feasibility of direct-to-home teleneuropsychology methodology during the COVID-19 pandemic. Https://DoiOrg/101080/1385404620222056922 2022:1–16. https://doi.org/10.1080/13854046.2022.2056922.
- Zeghari R, Guerchouche R, Tran-Duc M, Bremond F, Langel K, Ramakers I, et al. Feasibility Study of an Internet-Based Platform for Tele-Neuropsychological Assessment of Elderly in Remote Areas. Diagnostics 2022, Vol 12, Page 925 2022;12:925. https://doi.org/10.3390/DIAGNOSTICS12040925.
- Cullum CM, Hynan LS, Grosch M, Parikh M, Weiner MF. Teleneuropsychology: Evidence for video teleconference-based neuropsychological assessment. J Int Neuropsychol Soc 2014;20:1028–33.
- Ye S, Sun K, Phi H, Hosseini Ghomi R, Ghomi RH. Validation of Remote Testing using BrainCheck, a Computerized Neurocognitive Test. MedRxiv 2020:2020.06.01.20119289. https://doi.org/10.1101/2020.06.01.20119289.
There are 21 citations in total.
Details
| Primary Language | English |
|---|---|
| Subjects | Health Care Administration |
| Journal Section | Research Article |
| Authors | |
| Project Number | None |
| Early Pub Date | June 20, 2023 |
| Publication Date | July 10, 2023 |
| Submission Date | November 22, 2022 |
| Published in Issue | Year 2023Volume: 14 Issue: 3 |
