---
title: "Caching results"
output: litedown::html_format
vignette: >
  %\VignetteIndexEntry{Caching results}
  %\VignetteEngine{litedown::vignette}
  %\VignetteEncoding{UTF-8}
---

```{r setup}
library(garnish)
```

A cache remembers what a function returned for a given set of arguments, so the
second call is a lookup instead of the work. garnish has three caching
decorators, and the difference between them is what happens to old entries:

| Decorator | Keeps results | When full |
|---|---|---|
| `lru_cache()` | in memory | drops the least recently used entry |
| `cache` | in memory | never drops anything |
| `disk_cache()` | in files, so they outlive the session | drops the least recently used file |

`lru_cache()` and `cache` are ports of Python's `functools.lru_cache` and
`functools.cache`. `disk_cache()` is closer to `joblib.Memory.cache`.

The examples use a function that reports when it actually does the work:

```{r slow}
slow_square <- function(x) {
  cat("  computing", x, "\n")
  x^2
}
```

## In memory

```{r lru}
square <- lru_cache() %@% slow_square
square(4)
square(4)
square(5)
```

The second call printed nothing, because the function did not run. `cache` is
the same but has no size limit, and takes no parameters, so apply it directly:

```{r cache}
square <- cache %@% slow_square
square(4)
square(4)
```

Use `lru_cache()` when the number of distinct inputs is unbounded and memory
matters. Use `cache` when you know it is small.

### What counts as the same call

The key is the *matched* arguments including defaults, so spelling a call
differently does not miss the cache:

```{r same-key}
area <- lru_cache() %@% function(w, h = 2) {
  cat("  computing\n")
  w * h
}
area(3)
area(w = 3)
area(3, 2)
area(h = 2, w = 3)
```

Those four calls ran the function once. Changing an argument that matters does
run it again:

```{r different-key}
area(3, 5)
```

### What is and is not cached

- A **result** is cached, including `NULL` and including `invisible()` results,
  which stay invisible on a hit.
- An **error is not cached**, so a call that failed is tried again next time.

```{r errors}
attempts <- 0
f <- lru_cache() %@% function() {
  attempts <<- attempts + 1
  if (attempts < 2) stop("first call fails")
  "ok"
}
try(f(), silent = TRUE)
f()
f()
attempts
```

### Limiting size

For `lru_cache()`, `maxsize` is a size in **bytes** (default 1 GiB), not a
number of entries as in Python. When the cache is over that, the entry used
least recently goes first.

```{r maxsize}
g <- lru_cache(maxsize = 7000) %@% function(i) {
  cat("  computing", i, "\n")
  numeric(200) + i # roughly 2 KB per entry, so three fit
}
invisible(g(1)); invisible(g(2)); invisible(g(3))
invisible(g(1)) # a hit, so 2 is now the least recently used
invisible(g(4)) # over the limit, so 2 is evicted
invisible(g(2)) # recomputed
```

## On disk

`disk_cache()` stores each result as a file in `dir`. Close R, start again, and
the results are still there. There is no default for `dir`: a temporary
directory would disappear with the session and defeat the point.

```{r disk}
dir <- tempfile("garnish-cache")
square <- disk_cache(dir) %@% slow_square
square(7)
square(7)
length(list.files(dir))
```

A second decoration of the same function, such as in a new session, reuses the
files:

```{r disk-again}
square_again <- disk_cache(dir) %@% slow_square
square_again(7)
```

### Changing the function

The key includes the function's formals and body, so editing it starts a fresh
cache for that function. Entries from the old version stay in `dir` until they
are evicted:

```{r invalidate}
v1 <- disk_cache(dir) %@% function(x) x + 1
v2 <- disk_cache(dir) %@% function(x) x + 2
c(v1(1), v2(1))
```

What it does **not** notice is a change to something the function *calls*. If
`f` calls `helper()` and you change `helper()`, the cached results of `f` are
stale. Point `dir` somewhere new, or delete the files, when that happens.

### Choosing a file format

By default results are saved with `saveRDS()`. With the
[qs2](https://github.com/qsbase/qs2) package installed, `format = "qs2"` is
faster and a little smaller. On a data frame of one million rows, qs2 was about
seven times faster to write and three and a half times faster to read back on
one machine, so measure your own data.

```{r qs2, eval = requireNamespace("qs2", quietly = TRUE)}
fast <- disk_cache(tempfile("garnish-cache"), format = "qs2") %@% slow_square
fast(3)
fast(3)
```

The two formats do not share files. Changing `format` for an existing `dir`
starts with an empty cache.

## With promises

`async_lru_cache()`, `async_cache` and `async_disk_cache()` are the versions for
functions that return a promise. They take the same arguments and always return
a promise. Three things differ:

- The value a promise resolves with is cached, not the promise, so a call that
  rejects is not cached and the next call tries again.
- Identical calls made while one is still running share its promise, so a burst
  of the same request makes one call.
- Because only values are stored, `async_disk_cache()` works, and it reads and
  writes the same files as `disk_cache()`.

```{r async, eval = requireNamespace("promises", quietly = TRUE) && requireNamespace("later", quietly = TRUE)}
requests <- 0
lookup <- async_lru_cache() %@% function(id) {
  requests <<- requests + 1
  promises::promise(function(resolve, reject) {
    later::later(function() resolve(id * 2), 0.05)
  })
}

# Three identical calls at once make one request
for (i in 1:3) promises::then(lookup(21), function(value) cat(value, "\n"))
while (!later::loop_empty()) later::run_now(0.05)
requests
```

## When not to cache

Caching assumes that the same arguments give the same answer, and that the
function has no effects you need to happen every time.

- **Not pure.** A function that reads the clock, a random number generator, a
  file, or a database can return a different answer next time. The cache will
  keep returning the first one.
- **Side effects.** A cached call skips them. `cat("computing")` above ran once.
  If the function also writes a file, a hit will not write it again.
- **Large or unstable arguments.** Each call hashes its arguments, so hashing a
  huge data frame can cost more than the function it saves.
- **Objects that do not serialise.** Environments, external pointers and R6
  objects do not round-trip through `disk_cache()`.

## Choosing

- Same session, small set of inputs: `cache`.
- Same session, many inputs, memory matters: `lru_cache()`.
- You want results to survive a restart, or share between sessions:
  `disk_cache()`.
- The function returns a promise: the `async_` version of whichever of these
  fits.