Caching results

library(garnish)

A cache remembers what a function returned for a given set of arguments, so the second call is a lookup instead of the work. garnish has three caching decorators, and the difference between them is what happens to old entries:

Decorator Keeps results When full
lru_cache() in memory drops the least recently used entry
cache in memory never drops anything
disk_cache() in files, so they outlive the session drops the least recently used file

lru_cache() and cache are ports of Python’s functools.lru_cache and functools.cache. disk_cache() is closer to joblib.Memory.cache.

The examples use a function that reports when it actually does the work:

slow_square <- function(x) {
  cat("  computing", x, "\n")
  x^2
}

In memory

square <- lru_cache() %@% slow_square
square(4)

#>   computing 4 
#> [1] 16

square(4)

#> [1] 16

square(5)

#>   computing 5 
#> [1] 25

The second call printed nothing, because the function did not run. cache is the same but has no size limit, and takes no parameters, so apply it directly:

square <- cache %@% slow_square
square(4)

#>   computing 4 
#> [1] 16

square(4)

#> [1] 16

Use lru_cache() when the number of distinct inputs is unbounded and memory matters. Use cache when you know it is small.

What counts as the same call

The key is the matched arguments including defaults, so spelling a call differently does not miss the cache:

area <- lru_cache() %@% function(w, h = 2) {
  cat("  computing\n")
  w * h
}
area(3)

#>   computing
#> [1] 6

area(w = 3)

#> [1] 6

area(3, 2)

#> [1] 6

area(h = 2, w = 3)

#> [1] 6

Those four calls ran the function once. Changing an argument that matters does run it again:

area(3, 5)

#>   computing
#> [1] 15

What is and is not cached

attempts <- 0
f <- lru_cache() %@% function() {
  attempts <<- attempts + 1
  if (attempts < 2) stop("first call fails")
  "ok"
}
try(f(), silent = TRUE)
f()

#> [1] "ok"

f()

#> [1] "ok"

attempts

#> [1] 2

Limiting size

For lru_cache(), maxsize is a size in bytes (default 1 GiB), not a number of entries as in Python. When the cache is over that, the entry used least recently goes first.

g <- lru_cache(maxsize = 7000) %@% function(i) {
  cat("  computing", i, "\n")
  numeric(200) + i # roughly 2 KB per entry, so three fit
}
invisible(g(1)); invisible(g(2)); invisible(g(3))

#>   computing 1 
#>   computing 2 
#>   computing 3 

invisible(g(1)) # a hit, so 2 is now the least recently used
invisible(g(4)) # over the limit, so 2 is evicted

#>   computing 4 

invisible(g(2)) # recomputed

#>   computing 2

On disk

disk_cache() stores each result as a file in dir. Close R, start again, and the results are still there. There is no default for dir: a temporary directory would disappear with the session and defeat the point.

dir <- tempfile("garnish-cache")
square <- disk_cache(dir) %@% slow_square
square(7)

#>   computing 7 
#> [1] 49

square(7)

#> [1] 49

length(list.files(dir))

#> [1] 1

A second decoration of the same function, such as in a new session, reuses the files:

square_again <- disk_cache(dir) %@% slow_square
square_again(7)

#> [1] 49

Changing the function

The key includes the function’s formals and body, so editing it starts a fresh cache for that function. Entries from the old version stay in dir until they are evicted:

v1 <- disk_cache(dir) %@% function(x) x + 1
v2 <- disk_cache(dir) %@% function(x) x + 2
c(v1(1), v2(1))

#> [1] 2 3

What it does not notice is a change to something the function calls. If f calls helper() and you change helper(), the cached results of f are stale. Point dir somewhere new, or delete the files, when that happens.

Choosing a file format

By default results are saved with saveRDS(). With the qs2 package installed, format = "qs2" is faster and a little smaller. On a data frame of one million rows, qs2 was about seven times faster to write and three and a half times faster to read back on one machine, so measure your own data.

fast <- disk_cache(tempfile("garnish-cache"), format = "qs2") %@% slow_square
fast(3)

#>   computing 3 
#> [1] 9

fast(3)

#> [1] 9

The two formats do not share files. Changing format for an existing dir starts with an empty cache.

With promises

async_lru_cache(), async_cache and async_disk_cache() are the versions for functions that return a promise. They take the same arguments and always return a promise. Three things differ:

requests <- 0
lookup <- async_lru_cache() %@% function(id) {
  requests <<- requests + 1
  promises::promise(function(resolve, reject) {
    later::later(function() resolve(id * 2), 0.05)
  })
}

# Three identical calls at once make one request
for (i in 1:3) promises::then(lookup(21), function(value) cat(value, "\n"))
while (!later::loop_empty()) later::run_now(0.05)

#> 42 
#> 42 
#> 42 

requests

#> [1] 1

When not to cache

Caching assumes that the same arguments give the same answer, and that the function has no effects you need to happen every time.

Choosing