In a discussion on #moose-dev today, ether made the following distinction:
author tests are expected to pass on every commit; release tests only need to pass just before release
I think this is a good distinction. It also means that almost every single “xt” type test you might think of should probably be an author test. The only one we came up with in #moose-dev that was obviously a release test was a test to check that Changes has content for the release.
I’m sending PRs to various dzil plugins to move them to author tests, with the goal of being able to safely not run release tests under Travis.