What I look for in a Software paper.

I read, review, edit (and write) a lot of papers describing software for ecological research. So much so that I ended up developing a taste for it, and I thought it may be relevant to share it. This may also be the last blog post until September, so enjoy!

Disclaimer: nothing in this post represents the opinion of any journals in which I act as a reviewer, member of the editorial board, author, etc…

Software papers are usually shorter than regular papers, and this is an important piece of information. You do not have a lot of space to use, so going to the essential is key. Here are my three Dos, and my three Dont’ses.

Tell me why I should care.

This is usually the basic message of every scientific communication course, but it also translates to software. Tell me what I can do. Show me examples of cool science that you have made possible. Give me a nice figure in the end.

Tell me how I can use it.

By which I mean, it is important to see the license, where I can download the code, and any specific information. A link to the documentation helps, too.

Tell me I can trust it.

Unit tests, continuous integration, and a way to report bugs? I know that you have at least an idea of how to maintain quality. This seems trivial, but doing it shows that you have done your homework.

Don’t tell me too much.

The least code, the better. How to write things should be in the documentation, and in the user manual, so why should it be in the paper? Unless you can fit an entire analysis in 25 lines of code, I will not read the code anyways.

Don’t give me too much to use.

This is more of a software issue than a paper issue, but I am a firm believer in software doing one thing, and doing it well (with a few exceptions). Listing a table with 30 functions, when 90% of users will just need three of them, makes me wonder if there is not a way to simplify the design.

Don’t explain what the software does.

By which I mean, do not re-write the methods that the software implements. Give me a one or two-paragraph summary, and then references. The authors of the original paper had more room than you to explain how it works, so why should I want to read a second-hand abridged version? The shorter, the better.