If a service fails to provision, suspend, unsuspend, or cancel, it may be attempted as often as every 5 minutes, and would send an email to staff each time regarding the failure (This doesn't happen, there is no such email to staff).

So we propose to create a setting that sets a service action backoff on failure, and sets a time interval for which not to attempt the action again. (Question: Where should this setting go?)

Allow time intervals for waiting to attempt the action again:
15 min, 30 min, 1 hour, ... 24 hours

Edit: 2019/01/04

In addition to this backoff feature, we should add:

• A new email subscription notice under My Info for staff (including ACL to enable the option) that will email staff on a failure
• A new page under Tools that lists services that have failed to provision, suspend, unsuspend, cancel, or module renew at least once and information about the service, when it will next be attempted, with an option to clear any delay so the next cron will re-attempt. (Or maybe a new tab under Tools > Logs called "Service Action")
• A setting for the maximum number of retries before it's no longer attempted.