Hi there,

Has anyone in this community used a form of inter-rater reliability measure for multiple people editing for one project? I have some PIs who are interested in doing this for a project where several labs will be editing Freesurfer for a project. I am a bit stumped as I have always just used visual review to ensure reliability/validity between editors. Also, would a inter-rater reliability measure be more effective than visual reviewing?

Thank you for any insight you can provide!

Lisa