Improving your conference peer-review process

Article
Published on 12 November 2019

Acting as a reviewer for your event is a great opportunity for members. But make sure you walk a mile in their shoes when designing your conference peer-review process.

Peer review has been a foundation of the scientific method since the 1600s. And it’s still the best option out there for validating the scientific and technical research presented at your event. Without it, your conference proceedings would be filled with sub-standard submissions instead of the latest and greatest research in your field. And members who submit their work to your event wouldn’t get the peer feedback they need to improve.

In short, you can’t have a quality research conference without a willing army of reviewers.

Acting as a reviewer is a rite of passage for the members within your community. It can help upskill early-career members, it validates the quality of published research, and it gives your members the opportunity to engage with emerging trends within your field.

But conference peer review is also a delicate balancing act. It doesn’t take a whole lot to make the whole thing come tumbling down...

Reviewing for a conference is a big time commitment for members. The majority of conference reviewers assess multiple submissions over a few intense weeks. So if your review process is marred by shortcomings, things tend to go south, fast. And when a conference review process breaks down, reviewers are likely to withdraw their offers of help or rush their reviews to meet your deadline.

So don’t underestimate the importance of a well-designed conference peer-review process. Use the following tips and walk a mile in your reviewers’ shoes before you build yours.

Invite enough reviewers

Overloading reviewers is one sure way to annoy them. And unhappy reviewers tend to withdraw their offer to review or go AWOL altogether. When they do, you’ll be left scrambling to replace them or off-loading their submissions onto other reviewers. Calculate how many reviewers you need, then add a comfortable margin of error. And if you haven’t had enough reviewers in previous years, look beyond your traditional contacts and diversify your reviewer pool.

Set realistic deadlines

Your members are busy people. One great way to get on their bad side is to set up review deadlines that they can’t possibly meet. It’s nigh impossible to precisely plan every conference deadline. But this doesn’t justify putting them under pressure with an unworkable review timeline. So whether your next immovable date is sending your book of proceedings to the printers or building your conference’s mobile app, budget some buffer time for completing reviews.

Craft a straightforward reviewer experience

A clear and straightforward revieiwing experience is the basis of any successful peer-review process. Without one, you’ll be asking reviewers to spend unnecessary amounts of time trying to interpret your instructions. So make sure you’re following online form best practice. For example, if you have lots of optional sections for reviewers to complete, ask yourself whether these sections are truly needed. If they’re not, get rid of them.

Allocate fairly

A good way to infuriate reviewers is to allocate them submissions under topics they’re unfamiliar with. This forces them to give superficial reviews and it means authors don’t get a fair assessment, which is incredibly frustrating for both sides. Using an old clunky abstract management system can make allocating submissions to the right reviewers a difficult task. So use software that’s intelligent enough to match submissions to reviewers using shared topics. It should enable single or double-blind reviews and also allow you to set the number of reviews needed per submission.

Intervene where necessary

Find software that helps you track the progress of reviews. This will allow you to spot reviewers who are falling behind, and means you can send reminders to a select few, instead of spamming everyone. And, once you’ve identified who’s struggling to complete their reviews, you can step in and re-assign their submissions well before your deadline.

The path to a smoother peer review process

When the peer-review process goes awry, it can be a frustrating experience for everyone involved. So put some thought and preparation into building a pain-free peer-review process.

Author

Brian Campbell
Brian Campbell

Community Manager, Ex Ordo
www.exordo.com/