fnware fnware logo

Popular Python decorators ported to R with the adorn framework.

Decorator What it does Python equivalent
retry() Calls a function again when it fails tenacity
async_retry() Like retry(), but waits without blocking and returns a promise tenacity (async)
timeout() Stops a single call that runs too long wrapt-timeout-decorator
async_timeout() Like timeout(), but for functions that return a promise asyncio.timeout
lru_cache() Caches results, evicting the least recently used functools.lru_cache
cache Caches results, never evicting functools.cache
disk_cache() Caches results as files, so they outlive the session joblib.Memory.cache
async_lru_cache(), async_cache, async_disk_cache() Like the caches above, for functions that return a promise
validate_args(), validate_return() Checks arguments and return values with any validator function pydantic validate_call
async_validate_return() Like validate_return(), checking the value a promise resolves to
circuit_breaker() Stops calling a function after repeated failures, then retries after a cooldown pybreaker
async_circuit_breaker() Like circuit_breaker(), for functions that return a promise pybreaker (async)
rate_limit() Limits how often a function can be called in a time window ratelimit
async_rate_limit() Like rate_limit(), but queues calls instead of blocking ratelimit (async)
fallback() Returns a default value, or a handler's result, when a function fails tenacity retry_error_callback
async_fallback() Like fallback(), for functions that return a promise
once Runs a function the first time and reuses the result
async_once Like once, for functions that return a promise
log_call() Logs when a call starts, finishes or fails, with its arguments. Writes with message() or any logging package via a logger_fn() method
async_log_call() Like log_call(), logging the finish when the promise settles
contextmanager Writes setup and teardown once, as local_, with_ and decorator forms contextlib.contextmanager

Installation

# install.packages("pak")
# pak::pak("DyfanJones/fnware")
install.packages("fnware", repos = c('https://dyfanjones.r-universe.dev', 'https://cloud.r-project.org'))

Usage

retry() and lru_cache() build a decorator, which you apply to a function with %@% or by calling it on the function.

Retry

library(fnware)

n <- 0
flaky <- function() {
  n <<- n + 1
  if (n < 3) stop("not yet")
  n
}

retry(stop = stop_after_attempt(5))(flaky)()
#> [1] 3

Choose when to give up, how long to wait, and what to retry:

fetch := retry(
  stop = stop_after_attempt(4),
  wait = wait_exp(multiplier = 0.1) + wait_rand(0, 0.1),
  retry = retry_if_error_message(match = "timeout"),
  before_sleep = before_sleep_log()
) %@% function(url) {
  # ...
}

When stop fires, a fnware_retry_error condition is signalled. Set reraise = TRUE to re-throw the last error instead, or give retry_error_callback a function to return a fallback value:

poll := retry(
  stop = stop_after_attempt(3),
  retry = retry_if_result(is.null),
  retry_error_callback = function(state) "gave up"
) %@% function() NULL

poll()
#> [1] "gave up"
Strategy Functions
Stop (stop) stop_never(), stop_after_attempt(), stop_after_delay(), stop_before_delay(), stop_any(), stop_all()
Wait (wait) wait_none(), wait_fixed(), wait_rand(), wait_exp(), wait_exp_jitter(), wait_chain(), wait_combine() or a + b
Retry (retry) retry_all(), retry_never(), retry_if_error(), retry_if_error_message(), retry_if_result()
Hooks before_sleep_none(), before_sleep_log(), after_sleep_none(), after_sleep_log()

LRU cache

slow_square := lru_cache() %@% function(x) {
  Sys.sleep(0.2)
  x^2
}

slow_square(4) # slow
slow_square(4) # cached

cache is the same without a size limit. It takes no parameters, so apply it directly:

slow_cube := cache %@% function(x) {
  Sys.sleep(0.2)
  x^3
}

disk_cache() keeps results in files under dir, so they survive the session and can be shared between sessions. dir has no default, because a temporary directory would defeat the point:

slow_cube := disk_cache("~/.cache/my-analysis") %@% function(x) {
  Sys.sleep(0.2)
  x^3
}

Set format = "qs2" to use the qs2 package instead of saveRDS(). On a 1,000,000-row data frame it was about 7x faster to write and 3.5x faster to read back, and slightly smaller on disk (one machine, so measure your own). qs2 is optional and must be installed.

Editing a function's body invalidates its entries, but changing something it calls does not. Results are saved with saveRDS(), so environments and external pointers do not round-trip.

Context manager

R has no yield, so a context manager calls teardown() right after acquiring each resource and returns the resource. Unlike retry() and lru_cache(), contextmanager takes no parameters, so it is applied without calling it first.

local_tempdir := contextmanager %@% function(pattern = "tmp") {
  path <- tempfile(pattern)
  dir.create(path)
  teardown(unlink(path, recursive = TRUE))
  path
}

That one definition gives three forms. Called inside a function, cleanup runs when that function exits, like withr::local_*():

f <- function() {
  d <- local_tempdir()
  writeLines("hi", file.path(d, "x.txt"))
  d
}

d <- f()
dir.exists(d)
#> [1] FALSE

with_context() scopes it to a block, passing the value to a function:

with_context(local_tempdir(), function(d) list.files(d))
#> character(0)

as_decorator() runs a function inside the context:

g := as_decorator(local_tempdir) %@% function() "ran inside context"
g()
#> [1] "ran inside context"

Cleanups run last in, first out, and the ones registered before a failing setup step still run. A context manager called from another's setup is cleaned up with the outer one.

Validation

validate_args() takes one validator per argument and validate_return() takes one for the result. A validator is any function (or formula) that returns TRUE to pass. Returning a string fails with that string as the message, which is how checkmate's check_*() functions behave, so they work without fnware depending on them:

scale_by := validate_args(x = is.numeric, n = ~ .x > 0) %@% function(x, n = 1) {
  x * n
}

scale_by(2, 3)
#> [1] 6

scale_by("a", -1)
#> Error: Invalid arguments.
#> ✖ `x`: failed validation.
#> ✖ `n`: failed validation.

All failures are reported together as a fnware_validation_error. Defaults are validated too.

Stacking decorators

Decorators stack, and in a %@% b %@% f, a is outermost. From outside to inside, the order that works for the common cases is validate_args(), fallback(), circuit_breaker(), a cache, retry(), then rate_limit() and timeout(), so each attempt is throttled and capped. vignette("decorators") has a table of which decorator fits which problem, and the reasons for the order.

fetch := validate_args(url = is.character) %@%
  fallback(value = NULL) %@%
  circuit_breaker(failures = 3) %@%
  retry(stop = stop_after_attempt(3)) %@%
  rate_limit(calls = 5, period = 1) %@%
  timeout(10) %@%
  function(url) {
    # ...
  }

Learn more

Vignettes: vignette("decorators"), vignette("retry"), vignette("circuitbreaker"), vignette("fallback"), vignette("caching"), vignette("logging"), vignette("validation"), vignette("contextmanager") and vignette("async"). The help pages are ?retry, ?lru_cache (which also covers cache and disk_cache), ?contextmanager and ?validate_args. See the adorn documentation for how decorators are applied, stacked and removed.