Top tips for JOSS editors

Aim for reviewer redundancy

If you have 3 people agree to review, take them up on their offer(s), that way if one person drops out, you’ll have a backup and won’t have to look for more reviewers. Also, when sending invites, try pinging a number of people at the same time rather than doing it one-by-one.

Check development history early

When reviewing submissions, check the repository’s development history early in the process. Look for sustained development over time, evidence of open development practices, and collaborative effort. If you have concerns about whether the software meets the development practice criteria (timeline, open development, collaboration), raise these with the authors or EiC early rather than late in the review.

Email is a good backup

Email is often the most reliable way of contacting people. Whether it’s inviting reviewers, following up with reviewers or authors etc., if you’ve not heard back from someone on GitHub, try emailing them (their email is often available on their GitHub profile page).

Default to over-communicating

When you take an action (even if it isn’t on GitHub), share on the review thready what you’re up to. For example, if you’re looking for reviewers and are sending emails – leave a note on the review thread saying as much.

Use the JOSS Slack

There’s lots of historical knowledge in our Slack, and it’s a great way to get questions answered.

Ask reviewers to complete their review in 4-6 weeks

We aim for a total submission … publication time of ~3 months. This means we ideally want reviewers to complete their review in 4-6 weeks (max).

Use saved replies on GitHub

Saved replies on GitHub can be a huge productivity boost. Try making some using the example messages listed above.

Ping reviewers if they’ve not started after 2 weeks

If a reviewer hasn’t started within 1-2 weeks, you should probably give them a nudge. People often agree to review, and then forget about the review.

Learn how to nudge gently, and often

One of your jobs as the editor is to ensure the review keeps moving at a reasonable pace. If nothing has happened for a week or so, consider nudging the author or reviewers (depending upon who you’re waiting for). A friendly “👋 reviewer, how are you getting along here” can often be sufficient to get things moving again.

Manage author responsiveness

We generally ask authors to respond to reviewer feedback within 2 weeks and complete requested changes within 4-6 weeks, matching the timeliness we expect from reviewers, unless otherwise negotiated with you (the editor). Use your judgment about when to follow up with authors; for example, if the changes requested are particularly substantial, you may want to allow more time. If you haven’t heard from them in a while (often around 2 weeks), give them a friendly nudge. If there’s no response after a reasonable period (typically 4 weeks) without clear communication about delays, warn the author that their paper may be rejected due to lack of engagement and escalate to your Track Editor-in-Chief. This ensures reviewers’ time is respected and prevents reviews from stalling indefinitely.

Check in twice a week

Try to check in on your JOSS submissions twice per week, even if only for 5 minutes. Use your dashboard to stay on top of the current status of your submissions (i.e., who was the last person to comment on the thread).

Use editorialbot’s reminders

Make frequent use of the editorialbot remind function to remind yourself to check on the status of a review after a week or two, to check on if an author responds to a message, to make sure you’re staying on top of looking for reviewers, etc.