Editor Onboarding

Onboarding a new JOSS editor

All new editors at JOSS have an onboarding call with an Editor-in-Chief. You can use the structure below to make sure you highlight the most important aspects of being an editor.

Thing to check before the call

Have they reviewed or published in JOSS before? If not, you’ll need to spend significantly more time explaining how the review process works.
Check on their research background (e.g., what tracks they might edit most actively in).
Make sure to send them the editorial guide to read before the call.

The onboarding call

Preamble/introductions

Welcome! Thank them for their application to join the team.
Point out that this isn’t an interview. Rather, this is an informational call designed to give the candidate the information they need to make an informed decision about editing at JOSS.
90-day trial period/try out. Editor or JOSS editorial board can decide to part ways after that period.
No strict term limits. Some editors have been with us for 7+ years, others do 1-2 years. Most important thing is to be proactive with your editing responsibilities and communicating any issues with the EiCs.
Confirm with them their level of familiarity with JOSS/our review process.
Point out that they do not need to make a decision on the call today. They are welcome to have a think about joining and get back to us.

Share your screen

Visit JOSS (https://joss.theoj.org)
Pick a recently-published paper (you might want to identify this before the call one that shows off the review process well).
Show the paper on the JOSS site, and then go to the linked review issue.
Explain that there are two issues per submission – the pre-review issue and the main review issue.

The pre-review issue

The ‘meeting room for the paper’. Where author meets editor, and reviewers are identified.
Note that the EiC may have initiated a scope review. The editor should not start editing until this has completed. Also, editors are able to query the scope (as are reviewers) if they think the EiC should have (but didn’t).
Walk them through what is happening in the pre-review issue…
Editor is invited (likely with GitHub mention but also via email invite (@editorialbot invite @editor as editor))
Once editor accepts they start looking for reviewers.

Finding reviewers

Explain that this is one of the more time-intensive aspects of editing.
Explain where you can look for editors (your own professional network, asking the authors for recommendations, the reviewers application, similar papers identified by Editorialbot, )
Point out that we have a minimum of two reviewers, but if more than that accept (e.g., 3/4 then take them all – this gives you redundancy if one drops out).
Don’t invite only one reviewer at a time! If you do this, it may take many weeks to find two reviewers who accept. Try 3/4/5 invites simultaneously.
The sample messages section of the documentation has some example language you can use.

The review

Once at least two reviewers are assigned, time to get going!
Encourage reviewers to complete their review in 4-6 weeks.
Make sure to check in on the review – if reviewers haven’t started after ~1-2 weeks, time to remind them.
Your role as editor is not to do the review yourself, rather, your job is to ensure that both reviewers give a good review and to facilitate discussion and consensus on what the author needs to do.
Walk the editor through the various review artifacts: The checklist, comments/questions/discussion between reviewers and author, issues opened on the upstream repository (and cross-linked into the review thread).
Point editors to the ‘top tips’ section of our docs. Much of what makes an editor successful is regular check-ins on the review, and nudging people if nothing is happening.
Do not let a review go multiple weeks without checking in.

Wrapping up the review

Once the review is wrapping up, show the candidate the checks that an editor should be doing (reading the paper, suggested edits, asking for an archive etc.)
Show the recommend-accept step which is the formal hand-off between editor and editor-in-chief.
The sample messages section of the documentation has a checklist for the last steps of the review (for both authors and editors).

Show them the dashboard on the JOSS site

Point out that this means you do not need to stay on top of all of your notifications (the dashboard has the latest information).
Highlight here that we ask editors to handle 8-12 submissions per year on average.
…and that means 3-4 submissions on their desk at any one time (once they have completed their initial probationary period).
Show them the backlog for a track, and how they are welcome to pick papers from it (ideally oldest first).
Show them their profile page, and how they can list their tracks there, and also what their availability is.

Other important things to highlight

Don’t invite other editors as reviewers. We’re all busy editing our own papers…
Please be willing to edit outside of your specialisms. This helps JOSS run smoothly – often we don’t have the ‘ideal’ editor for a submission and someone has to take it.
Highlight that editors will have a buddy to work with for the first few months, and that it’s very common for editors to ask questions in Slack (and people generally respond quickly).
Scope reviews only work if editors vote! Please respond and vote on the weekly scope review email if you can. The process is private (authors don’t know what editors are saying). Detailed comments are really helpful for the EiCs.

Wrapping up

Make sure you’ve highlighted everything in the ‘top tips’ section of our docs.
Reinforce that this is a commitment, and thay regular attention to their submissions is absolutely critical (i.e., check in a couple of times per week).
Ask if they would like to move forward or would like time to consider the opportunity.
If they want to move forward, highlight they will receive a small number of invites: One to the JOSS editors GitHub team, a Slack invite, a Google Group invite, and an invite to the JOSS website to fill out their profile.
If they are joining the team, make sure they mark themselves as unavailable in the JOSS reviewers database (so they don’t get asked to review any longer).
Thank them again, and welcome them to the team.

Communicate outcome to EiC

Let the EiC know what the outcome was, and ask them to send out the invites to our various systems.
Work with EiC to identify onboarding buddy.
Decide who is going to identify the first couple of papers for the editor to work on.

Editorial buddy

New editors are assigned an editorial ‘buddy’ from the existing editorial team. The buddy is there to help the new editor onboard successfully and to provide a dedicated resource for any questions they might have but don’t feel comfortable posting to the editor mailing list.

Buddy assignments don’t have a fixed term but generally require a commitment for 1-2 months.

Some things you might need to do as a buddy for a new editor:

Respond to questions via email or on GitHub review issues.
Check in with the new editor every couple of weeks if there hasn’t been any other communication.
(Optionally) keep an eye on the new editor’s submissions.

Managing notifications

Being on the JOSS editorial team means that there can be a lot of notifications from GitHub if you don’t take some proactive steps to minimize noise from the reviews repository.

Things you should do when joining the editorial team

Curate your GitHub notifications experience

GitHub has extensive documentation on managing notifications which explains when and why different notifications are sent from a repository.

Set up email filters

Email filters can be very useful for managing incoming email notifications, here are some recommended resources:

A GitHub blog post describing how to set up email filters.

If you use Gmail:

Use a dedicated tool

For papers that you are already assigned to edit, the dedicated JOSS dashboard aggregates notifications associated with each paper. The dashboard is available at: https://joss.theoj.org/dashboard/<yourgithubusername>

Another tool you might want to try out is Octobox.