Introduction to Cache

Eliot J. B. McIntire

December 21 2022

1 Reproducible workflow

As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.

1.1 Differences with other approaches

Cache users DBI as a backend, with key functions, dbReadTable, dbRemoveTable, dbSendQuery, dbSendStatement, dbCreateTable and dbAppendTable. These can all be accessed via Cache, showCache, clearCache, and keepCache. It is optimized for speed of transactions, using fastdigest::fastdigest on R memory objects and digest::digest on files. The main function is superficially similar to archivist::cache, which uses digest::digest in all cases to determine whether the arguments are identical in subsequent iterations. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.

1.2 Function-level caching

Any function can be cached by wrapping Cache around the function call, or by using base pipe |> or using: Cache(FUN = functionName, ...)

This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).

This is particularly useful for expensive operations.

library(raster)
## Loading required package: sp
library(reproducible)
library(data.table)
## 
## Attaching package: 'data.table'
## The following object is masked from 'package:raster':
## 
##     shift
setDTthreads(1) # user can omit this with little effect

tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
dir.create(tmpDir, recursive = TRUE)

ras <- raster(extent(0, 1000, 0, 1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +datum=WGS84"

newCRS <- "+init=epsg:4326" # A longlat crs

# No Cache
system.time(suppressWarnings(map1 <- projectRaster(ras, crs = newCRS))) # Warnings due to new PROJ
##    user  system elapsed 
##   1.879   0.198   2.079
# Try with memoise for this example -- for many simple cases, memoising will not be faster
opts <- options("reproducible.useMemoise" = TRUE)
# With Cache -- a little slower the first time because saving to disk
system.time({
  suppressWarnings({
    map1 <- Cache(projectRaster, ras, crs = newCRS, cachePath = tmpDir, notOlderThan = Sys.time())
  })
})
##    user  system elapsed 
##   1.677   0.153   1.835
# faster the second time
system.time({
  map2 <- Cache(projectRaster, ras, crs = newCRS, cachePath = tmpDir)
})
##   ...(Object to retrieve (b96ade7aaca8821a.rds) is large: 6.5 Mb)
##      loaded memoised result from previous projectRaster call
##    user  system elapsed 
##   0.359   0.000   0.361
options(opts)

all.equal(map1, map2) # TRUE
## [1] TRUE

1.3 Caching examples

1.3.1 Basic use

library(raster)
library(magrittr)
## 
## Attaching package: 'magrittr'
## The following object is masked from 'package:raster':
## 
##     extract
try(clearCache(tmpDir, ask = FALSE), silent = TRUE) # just to make sure it is clear

ranNumsA <- Cache(rnorm, 10, 16, cachePath = tmpDir)

# All same
ranNumsB <- Cache(rnorm, 10, 16, cachePath = tmpDir) # recovers cached copy
##   ...(Object to retrieve (4fca280cda001fc9.rds))
##      loaded cached result from previous rnorm call
ranNumsD1 <- Cache(quote(rnorm(n = 10, 16)), cachePath = tmpDir) # recovers cached copy
##   ...(Object to retrieve (4fca280cda001fc9.rds))
##      loaded cached result from previous quote call
ranNumsD2 <- Cache(rnorm(n = 10, 16), cachePath = tmpDir) # recovers cached copy
##   ...(Object to retrieve (4fca280cda001fc9.rds))
##      loaded cached result from previous rnorm call
# Any minor change makes it different
ranNumsE <- Cache(rnorm, 10, 6, cachePath = tmpDir) # different

1.3.1.1 Example 1: Basic cache use with tags

ranNumsA <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif(4), cachePath = tmpDir, userTags = "objectName:b")

showCache(tmpDir, userTags = c("objectName"))
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: ad0ea27476c50b66          objectName                     a
##  2: ad0ea27476c50b66            function                 rnorm
##  3: ad0ea27476c50b66               class               numeric
##  4: ad0ea27476c50b66         object.size                  1008
##  5: ad0ea27476c50b66            accessed   2022-12-21 22:54:22
##  6: ad0ea27476c50b66             inCloud                 FALSE
##  7: ad0ea27476c50b66          resultHash                      
##  8: ad0ea27476c50b66   elapsedTimeDigest      0.001453161 secs
##  9: ad0ea27476c50b66 elapsedTimeFirstRun     1.573563e-05 secs
## 10: ad0ea27476c50b66      otherFunctions      vweave_rmarkdown
## 11: ad0ea27476c50b66      otherFunctions          process_file
## 12: ad0ea27476c50b66      otherFunctions         process_group
## 13: ad0ea27476c50b66      otherFunctions   process_group.block
## 14: ad0ea27476c50b66      otherFunctions            call_block
## 15: ad0ea27476c50b66      otherFunctions            block_exec
## 16: ad0ea27476c50b66      otherFunctions                 eng_r
## 17: ad0ea27476c50b66      otherFunctions          in_input_dir
## 18: ad0ea27476c50b66      otherFunctions                in_dir
## 19: ad0ea27476c50b66      otherFunctions             timing_fn
## 20: ad0ea27476c50b66      otherFunctions                handle
## 21: ad0ea27476c50b66           preDigest    n:7eef4eae85fd9229
## 22: ad0ea27476c50b66           preDigest mean:c40c00762a0dac94
## 23: ad0ea27476c50b66           preDigest   sd:853b1797f54b229c
## 24: ad0ea27476c50b66           preDigest .FUN:4f604aa46882b368
## 25: ad0ea27476c50b66           file.size                   175
## 26: deaa37372f85861b          objectName                     b
## 27: deaa37372f85861b            function                 runif
## 28: deaa37372f85861b               class               numeric
## 29: deaa37372f85861b         object.size                  1008
## 30: deaa37372f85861b            accessed   2022-12-21 22:54:22
## 31: deaa37372f85861b             inCloud                 FALSE
## 32: deaa37372f85861b          resultHash                      
## 33: deaa37372f85861b   elapsedTimeDigest      0.001456499 secs
## 34: deaa37372f85861b elapsedTimeFirstRun     1.454353e-05 secs
## 35: deaa37372f85861b      otherFunctions      vweave_rmarkdown
## 36: deaa37372f85861b      otherFunctions          process_file
## 37: deaa37372f85861b      otherFunctions         process_group
## 38: deaa37372f85861b      otherFunctions   process_group.block
## 39: deaa37372f85861b      otherFunctions            call_block
## 40: deaa37372f85861b      otherFunctions            block_exec
## 41: deaa37372f85861b      otherFunctions                 eng_r
## 42: deaa37372f85861b      otherFunctions          in_input_dir
## 43: deaa37372f85861b      otherFunctions                in_dir
## 44: deaa37372f85861b      otherFunctions             timing_fn
## 45: deaa37372f85861b      otherFunctions                handle
## 46: deaa37372f85861b           preDigest    n:7eef4eae85fd9229
## 47: deaa37372f85861b           preDigest  min:c40c00762a0dac94
## 48: deaa37372f85861b           preDigest  max:853b1797f54b229c
## 49: deaa37372f85861b           preDigest .FUN:881ec847b7161f3c
## 50: deaa37372f85861b           file.size                   170
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:22
##  2: 2022-12-21 22:54:22
##  3: 2022-12-21 22:54:22
##  4: 2022-12-21 22:54:22
##  5: 2022-12-21 22:54:22
##  6: 2022-12-21 22:54:22
##  7: 2022-12-21 22:54:22
##  8: 2022-12-21 22:54:22
##  9: 2022-12-21 22:54:22
## 10: 2022-12-21 22:54:22
## 11: 2022-12-21 22:54:22
## 12: 2022-12-21 22:54:22
## 13: 2022-12-21 22:54:22
## 14: 2022-12-21 22:54:22
## 15: 2022-12-21 22:54:22
## 16: 2022-12-21 22:54:22
## 17: 2022-12-21 22:54:22
## 18: 2022-12-21 22:54:22
## 19: 2022-12-21 22:54:22
## 20: 2022-12-21 22:54:22
## 21: 2022-12-21 22:54:22
## 22: 2022-12-21 22:54:22
## 23: 2022-12-21 22:54:22
## 24: 2022-12-21 22:54:22
## 25: 2022-12-21 22:54:22
## 26: 2022-12-21 22:54:22
## 27: 2022-12-21 22:54:22
## 28: 2022-12-21 22:54:22
## 29: 2022-12-21 22:54:22
## 30: 2022-12-21 22:54:22
## 31: 2022-12-21 22:54:22
## 32: 2022-12-21 22:54:22
## 33: 2022-12-21 22:54:22
## 34: 2022-12-21 22:54:22
## 35: 2022-12-21 22:54:22
## 36: 2022-12-21 22:54:22
## 37: 2022-12-21 22:54:22
## 38: 2022-12-21 22:54:22
## 39: 2022-12-21 22:54:22
## 40: 2022-12-21 22:54:22
## 41: 2022-12-21 22:54:22
## 42: 2022-12-21 22:54:22
## 43: 2022-12-21 22:54:22
## 44: 2022-12-21 22:54:22
## 45: 2022-12-21 22:54:22
## 46: 2022-12-21 22:54:22
## 47: 2022-12-21 22:54:22
## 48: 2022-12-21 22:54:22
## 49: 2022-12-21 22:54:22
## 50: 2022-12-21 22:54:22
##             createdDate
showCache(tmpDir, userTags = c("^a$")) # regular expression ... "a" exactly
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##              cacheId              tagKey              tagValue
##  1: ad0ea27476c50b66          objectName                     a
##  2: ad0ea27476c50b66            function                 rnorm
##  3: ad0ea27476c50b66               class               numeric
##  4: ad0ea27476c50b66         object.size                  1008
##  5: ad0ea27476c50b66            accessed   2022-12-21 22:54:22
##  6: ad0ea27476c50b66             inCloud                 FALSE
##  7: ad0ea27476c50b66          resultHash                      
##  8: ad0ea27476c50b66   elapsedTimeDigest      0.001453161 secs
##  9: ad0ea27476c50b66 elapsedTimeFirstRun     1.573563e-05 secs
## 10: ad0ea27476c50b66      otherFunctions      vweave_rmarkdown
## 11: ad0ea27476c50b66      otherFunctions          process_file
## 12: ad0ea27476c50b66      otherFunctions         process_group
## 13: ad0ea27476c50b66      otherFunctions   process_group.block
## 14: ad0ea27476c50b66      otherFunctions            call_block
## 15: ad0ea27476c50b66      otherFunctions            block_exec
## 16: ad0ea27476c50b66      otherFunctions                 eng_r
## 17: ad0ea27476c50b66      otherFunctions          in_input_dir
## 18: ad0ea27476c50b66      otherFunctions                in_dir
## 19: ad0ea27476c50b66      otherFunctions             timing_fn
## 20: ad0ea27476c50b66      otherFunctions                handle
## 21: ad0ea27476c50b66           preDigest    n:7eef4eae85fd9229
## 22: ad0ea27476c50b66           preDigest mean:c40c00762a0dac94
## 23: ad0ea27476c50b66           preDigest   sd:853b1797f54b229c
## 24: ad0ea27476c50b66           preDigest .FUN:4f604aa46882b368
## 25: ad0ea27476c50b66           file.size                   175
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:22
##  2: 2022-12-21 22:54:22
##  3: 2022-12-21 22:54:22
##  4: 2022-12-21 22:54:22
##  5: 2022-12-21 22:54:22
##  6: 2022-12-21 22:54:22
##  7: 2022-12-21 22:54:22
##  8: 2022-12-21 22:54:22
##  9: 2022-12-21 22:54:22
## 10: 2022-12-21 22:54:22
## 11: 2022-12-21 22:54:22
## 12: 2022-12-21 22:54:22
## 13: 2022-12-21 22:54:22
## 14: 2022-12-21 22:54:22
## 15: 2022-12-21 22:54:22
## 16: 2022-12-21 22:54:22
## 17: 2022-12-21 22:54:22
## 18: 2022-12-21 22:54:22
## 19: 2022-12-21 22:54:22
## 20: 2022-12-21 22:54:22
## 21: 2022-12-21 22:54:22
## 22: 2022-12-21 22:54:22
## 23: 2022-12-21 22:54:22
## 24: 2022-12-21 22:54:22
## 25: 2022-12-21 22:54:22
##             createdDate
showCache(tmpDir, userTags = c("runif")) # show only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##              cacheId              tagKey              tagValue
##  1: deaa37372f85861b          objectName                     b
##  2: deaa37372f85861b            function                 runif
##  3: deaa37372f85861b               class               numeric
##  4: deaa37372f85861b         object.size                  1008
##  5: deaa37372f85861b            accessed   2022-12-21 22:54:22
##  6: deaa37372f85861b             inCloud                 FALSE
##  7: deaa37372f85861b          resultHash                      
##  8: deaa37372f85861b   elapsedTimeDigest      0.001456499 secs
##  9: deaa37372f85861b elapsedTimeFirstRun     1.454353e-05 secs
## 10: deaa37372f85861b      otherFunctions      vweave_rmarkdown
## 11: deaa37372f85861b      otherFunctions          process_file
## 12: deaa37372f85861b      otherFunctions         process_group
## 13: deaa37372f85861b      otherFunctions   process_group.block
## 14: deaa37372f85861b      otherFunctions            call_block
## 15: deaa37372f85861b      otherFunctions            block_exec
## 16: deaa37372f85861b      otherFunctions                 eng_r
## 17: deaa37372f85861b      otherFunctions          in_input_dir
## 18: deaa37372f85861b      otherFunctions                in_dir
## 19: deaa37372f85861b      otherFunctions             timing_fn
## 20: deaa37372f85861b      otherFunctions                handle
## 21: deaa37372f85861b           preDigest    n:7eef4eae85fd9229
## 22: deaa37372f85861b           preDigest  min:c40c00762a0dac94
## 23: deaa37372f85861b           preDigest  max:853b1797f54b229c
## 24: deaa37372f85861b           preDigest .FUN:881ec847b7161f3c
## 25: deaa37372f85861b           file.size                   170
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:22
##  2: 2022-12-21 22:54:22
##  3: 2022-12-21 22:54:22
##  4: 2022-12-21 22:54:22
##  5: 2022-12-21 22:54:22
##  6: 2022-12-21 22:54:22
##  7: 2022-12-21 22:54:22
##  8: 2022-12-21 22:54:22
##  9: 2022-12-21 22:54:22
## 10: 2022-12-21 22:54:22
## 11: 2022-12-21 22:54:22
## 12: 2022-12-21 22:54:22
## 13: 2022-12-21 22:54:22
## 14: 2022-12-21 22:54:22
## 15: 2022-12-21 22:54:22
## 16: 2022-12-21 22:54:22
## 17: 2022-12-21 22:54:22
## 18: 2022-12-21 22:54:22
## 19: 2022-12-21 22:54:22
## 20: 2022-12-21 22:54:22
## 21: 2022-12-21 22:54:22
## 22: 2022-12-21 22:54:22
## 23: 2022-12-21 22:54:22
## 24: 2022-12-21 22:54:22
## 25: 2022-12-21 22:54:22
##             createdDate
clearCache(tmpDir, userTags = c("runif"), ask = FALSE) # remove only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # only those made during rnorm call
## Cache size:
##   Total (including Rasters): 804 bytes
##   Selected objects (not including Rasters): 804 bytes
##              cacheId              tagKey              tagValue
##  1: 4fca280cda001fc9            function                 rnorm
##  2: 4fca280cda001fc9               class               numeric
##  3: 4fca280cda001fc9         object.size                  1104
##  4: 4fca280cda001fc9            accessed   2022-12-21 22:54:22
##  5: 4fca280cda001fc9             inCloud                 FALSE
##  6: 4fca280cda001fc9          resultHash                      
##  7: 4fca280cda001fc9   elapsedTimeDigest      0.001773596 secs
##  8: 4fca280cda001fc9 elapsedTimeFirstRun     2.479553e-05 secs
##  9: 4fca280cda001fc9      otherFunctions      vweave_rmarkdown
## 10: 4fca280cda001fc9      otherFunctions          process_file
## 11: 4fca280cda001fc9      otherFunctions         process_group
## 12: 4fca280cda001fc9      otherFunctions   process_group.block
## 13: 4fca280cda001fc9      otherFunctions            call_block
## 14: 4fca280cda001fc9      otherFunctions            block_exec
## 15: 4fca280cda001fc9      otherFunctions                 eng_r
## 16: 4fca280cda001fc9      otherFunctions          in_input_dir
## 17: 4fca280cda001fc9      otherFunctions                in_dir
## 18: 4fca280cda001fc9      otherFunctions             timing_fn
## 19: 4fca280cda001fc9      otherFunctions                handle
## 20: 4fca280cda001fc9           preDigest    n:c5775c3b366fb719
## 21: 4fca280cda001fc9           preDigest mean:15620f138033a66c
## 22: 4fca280cda001fc9           preDigest   sd:853b1797f54b229c
## 23: 4fca280cda001fc9           preDigest .FUN:4f604aa46882b368
## 24: 4fca280cda001fc9           file.size                   224
## 25: 4fca280cda001fc9            accessed   2022-12-21 22:54:22
## 26: 4fca280cda001fc9     elapsedTimeLoad      0.001853228 secs
## 27: 4fca280cda001fc9            accessed   2022-12-21 22:54:22
## 28: 4fca280cda001fc9            accessed   2022-12-21 22:54:22
## 29: 5efb386c43c29ce1            function                 rnorm
## 30: 5efb386c43c29ce1               class               numeric
## 31: 5efb386c43c29ce1         object.size                  1104
## 32: 5efb386c43c29ce1            accessed   2022-12-21 22:54:22
## 33: 5efb386c43c29ce1             inCloud                 FALSE
## 34: 5efb386c43c29ce1          resultHash                      
## 35: 5efb386c43c29ce1   elapsedTimeDigest      0.001333237 secs
## 36: 5efb386c43c29ce1 elapsedTimeFirstRun     1.597404e-05 secs
## 37: 5efb386c43c29ce1      otherFunctions      vweave_rmarkdown
## 38: 5efb386c43c29ce1      otherFunctions          process_file
## 39: 5efb386c43c29ce1      otherFunctions         process_group
## 40: 5efb386c43c29ce1      otherFunctions   process_group.block
## 41: 5efb386c43c29ce1      otherFunctions            call_block
## 42: 5efb386c43c29ce1      otherFunctions            block_exec
## 43: 5efb386c43c29ce1      otherFunctions                 eng_r
## 44: 5efb386c43c29ce1      otherFunctions          in_input_dir
## 45: 5efb386c43c29ce1      otherFunctions                in_dir
## 46: 5efb386c43c29ce1      otherFunctions             timing_fn
## 47: 5efb386c43c29ce1      otherFunctions                handle
## 48: 5efb386c43c29ce1           preDigest    n:c5775c3b366fb719
## 49: 5efb386c43c29ce1           preDigest mean:152602b8ff81e5bb
## 50: 5efb386c43c29ce1           preDigest   sd:853b1797f54b229c
## 51: 5efb386c43c29ce1           preDigest .FUN:4f604aa46882b368
## 52: 5efb386c43c29ce1           file.size                   222
## 53: ad0ea27476c50b66          objectName                     a
## 54: ad0ea27476c50b66            function                 rnorm
## 55: ad0ea27476c50b66               class               numeric
## 56: ad0ea27476c50b66         object.size                  1008
## 57: ad0ea27476c50b66            accessed   2022-12-21 22:54:22
## 58: ad0ea27476c50b66             inCloud                 FALSE
## 59: ad0ea27476c50b66          resultHash                      
## 60: ad0ea27476c50b66   elapsedTimeDigest      0.001453161 secs
## 61: ad0ea27476c50b66 elapsedTimeFirstRun     1.573563e-05 secs
## 62: ad0ea27476c50b66      otherFunctions      vweave_rmarkdown
## 63: ad0ea27476c50b66      otherFunctions          process_file
## 64: ad0ea27476c50b66      otherFunctions         process_group
## 65: ad0ea27476c50b66      otherFunctions   process_group.block
## 66: ad0ea27476c50b66      otherFunctions            call_block
## 67: ad0ea27476c50b66      otherFunctions            block_exec
## 68: ad0ea27476c50b66      otherFunctions                 eng_r
## 69: ad0ea27476c50b66      otherFunctions          in_input_dir
## 70: ad0ea27476c50b66      otherFunctions                in_dir
## 71: ad0ea27476c50b66      otherFunctions             timing_fn
## 72: ad0ea27476c50b66      otherFunctions                handle
## 73: ad0ea27476c50b66           preDigest    n:7eef4eae85fd9229
## 74: ad0ea27476c50b66           preDigest mean:c40c00762a0dac94
## 75: ad0ea27476c50b66           preDigest   sd:853b1797f54b229c
## 76: ad0ea27476c50b66           preDigest .FUN:4f604aa46882b368
## 77: ad0ea27476c50b66           file.size                   175
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:22
##  2: 2022-12-21 22:54:22
##  3: 2022-12-21 22:54:22
##  4: 2022-12-21 22:54:22
##  5: 2022-12-21 22:54:22
##  6: 2022-12-21 22:54:22
##  7: 2022-12-21 22:54:22
##  8: 2022-12-21 22:54:22
##  9: 2022-12-21 22:54:22
## 10: 2022-12-21 22:54:22
## 11: 2022-12-21 22:54:22
## 12: 2022-12-21 22:54:22
## 13: 2022-12-21 22:54:22
## 14: 2022-12-21 22:54:22
## 15: 2022-12-21 22:54:22
## 16: 2022-12-21 22:54:22
## 17: 2022-12-21 22:54:22
## 18: 2022-12-21 22:54:22
## 19: 2022-12-21 22:54:22
## 20: 2022-12-21 22:54:22
## 21: 2022-12-21 22:54:22
## 22: 2022-12-21 22:54:22
## 23: 2022-12-21 22:54:22
## 24: 2022-12-21 22:54:22
## 25: 2022-12-21 22:54:22
## 26: 2022-12-21 22:54:22
## 27: 2022-12-21 22:54:22
## 28: 2022-12-21 22:54:22
## 29: 2022-12-21 22:54:22
## 30: 2022-12-21 22:54:22
## 31: 2022-12-21 22:54:22
## 32: 2022-12-21 22:54:22
## 33: 2022-12-21 22:54:22
## 34: 2022-12-21 22:54:22
## 35: 2022-12-21 22:54:22
## 36: 2022-12-21 22:54:22
## 37: 2022-12-21 22:54:22
## 38: 2022-12-21 22:54:22
## 39: 2022-12-21 22:54:22
## 40: 2022-12-21 22:54:22
## 41: 2022-12-21 22:54:22
## 42: 2022-12-21 22:54:22
## 43: 2022-12-21 22:54:22
## 44: 2022-12-21 22:54:22
## 45: 2022-12-21 22:54:22
## 46: 2022-12-21 22:54:22
## 47: 2022-12-21 22:54:22
## 48: 2022-12-21 22:54:22
## 49: 2022-12-21 22:54:22
## 50: 2022-12-21 22:54:22
## 51: 2022-12-21 22:54:22
## 52: 2022-12-21 22:54:22
## 53: 2022-12-21 22:54:22
## 54: 2022-12-21 22:54:22
## 55: 2022-12-21 22:54:22
## 56: 2022-12-21 22:54:22
## 57: 2022-12-21 22:54:22
## 58: 2022-12-21 22:54:22
## 59: 2022-12-21 22:54:22
## 60: 2022-12-21 22:54:22
## 61: 2022-12-21 22:54:22
## 62: 2022-12-21 22:54:22
## 63: 2022-12-21 22:54:22
## 64: 2022-12-21 22:54:22
## 65: 2022-12-21 22:54:22
## 66: 2022-12-21 22:54:22
## 67: 2022-12-21 22:54:22
## 68: 2022-12-21 22:54:22
## 69: 2022-12-21 22:54:22
## 70: 2022-12-21 22:54:22
## 71: 2022-12-21 22:54:22
## 72: 2022-12-21 22:54:22
## 73: 2022-12-21 22:54:22
## 74: 2022-12-21 22:54:22
## 75: 2022-12-21 22:54:22
## 76: 2022-12-21 22:54:22
## 77: 2022-12-21 22:54:22
##             createdDate
clearCache(tmpDir, ask = FALSE)

1.3.1.2 Example 2: using the “accessed” tag

ranNumsA <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cachePath = tmpDir, userTags = "objectName:b")

# access it again, from Cache
Sys.sleep(1)
ranNumsA <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:a")
##   ...(Object to retrieve (ad0ea27476c50b66.rds))
##      loaded cached result from previous rnorm call
wholeCache <- showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
#   Note: the two different ways to access -- old way with "artifact" will be deprecated
toRemove <- unique(wholeCache[!onlyRecentlyAccessed, on = "cacheId"], by = "cacheId")$cacheId
clearCache(tmpDir, toRemove, ask = FALSE) # remove ones not recently accessed
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##              cacheId              tagKey              tagValue
##  1: ad0ea27476c50b66          objectName                     a
##  2: ad0ea27476c50b66            function                 rnorm
##  3: ad0ea27476c50b66               class               numeric
##  4: ad0ea27476c50b66         object.size                  1008
##  5: ad0ea27476c50b66            accessed   2022-12-21 22:54:22
##  6: ad0ea27476c50b66             inCloud                 FALSE
##  7: ad0ea27476c50b66          resultHash                      
##  8: ad0ea27476c50b66   elapsedTimeDigest      0.001512051 secs
##  9: ad0ea27476c50b66 elapsedTimeFirstRun     1.573563e-05 secs
## 10: ad0ea27476c50b66      otherFunctions      vweave_rmarkdown
## 11: ad0ea27476c50b66      otherFunctions          process_file
## 12: ad0ea27476c50b66      otherFunctions         process_group
## 13: ad0ea27476c50b66      otherFunctions   process_group.block
## 14: ad0ea27476c50b66      otherFunctions            call_block
## 15: ad0ea27476c50b66      otherFunctions            block_exec
## 16: ad0ea27476c50b66      otherFunctions                 eng_r
## 17: ad0ea27476c50b66      otherFunctions          in_input_dir
## 18: ad0ea27476c50b66      otherFunctions                in_dir
## 19: ad0ea27476c50b66      otherFunctions             timing_fn
## 20: ad0ea27476c50b66      otherFunctions                handle
## 21: ad0ea27476c50b66           preDigest    n:7eef4eae85fd9229
## 22: ad0ea27476c50b66           preDigest mean:c40c00762a0dac94
## 23: ad0ea27476c50b66           preDigest   sd:853b1797f54b229c
## 24: ad0ea27476c50b66           preDigest .FUN:4f604aa46882b368
## 25: ad0ea27476c50b66           file.size                   175
## 26: ad0ea27476c50b66            accessed   2022-12-21 22:54:23
## 27: ad0ea27476c50b66     elapsedTimeLoad      0.002373457 secs
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:22
##  2: 2022-12-21 22:54:22
##  3: 2022-12-21 22:54:22
##  4: 2022-12-21 22:54:22
##  5: 2022-12-21 22:54:22
##  6: 2022-12-21 22:54:22
##  7: 2022-12-21 22:54:22
##  8: 2022-12-21 22:54:22
##  9: 2022-12-21 22:54:22
## 10: 2022-12-21 22:54:22
## 11: 2022-12-21 22:54:22
## 12: 2022-12-21 22:54:22
## 13: 2022-12-21 22:54:22
## 14: 2022-12-21 22:54:22
## 15: 2022-12-21 22:54:22
## 16: 2022-12-21 22:54:22
## 17: 2022-12-21 22:54:22
## 18: 2022-12-21 22:54:22
## 19: 2022-12-21 22:54:22
## 20: 2022-12-21 22:54:22
## 21: 2022-12-21 22:54:22
## 22: 2022-12-21 22:54:22
## 23: 2022-12-21 22:54:22
## 24: 2022-12-21 22:54:22
## 25: 2022-12-21 22:54:22
## 26: 2022-12-21 22:54:23
## 27: 2022-12-21 22:54:23
##             createdDate

1.3.1.3 Example 3: using keepCache

ranNumsA <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:a")
##   ...(Object to retrieve (ad0ea27476c50b66.rds))
##      loaded cached result from previous rnorm call
ranNumsB <- Cache(runif(4), cachePath = tmpDir, userTags = "objectName:b")

# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay, ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: ad0ea27476c50b66          objectName                     a
##  2: ad0ea27476c50b66            function                 rnorm
##  3: ad0ea27476c50b66               class               numeric
##  4: ad0ea27476c50b66         object.size                  1008
##  5: ad0ea27476c50b66            accessed   2022-12-21 22:54:22
##  6: ad0ea27476c50b66             inCloud                 FALSE
##  7: ad0ea27476c50b66          resultHash                      
##  8: ad0ea27476c50b66   elapsedTimeDigest      0.001512051 secs
##  9: ad0ea27476c50b66 elapsedTimeFirstRun     1.573563e-05 secs
## 10: ad0ea27476c50b66      otherFunctions      vweave_rmarkdown
## 11: ad0ea27476c50b66      otherFunctions          process_file
## 12: ad0ea27476c50b66      otherFunctions         process_group
## 13: ad0ea27476c50b66      otherFunctions   process_group.block
## 14: ad0ea27476c50b66      otherFunctions            call_block
## 15: ad0ea27476c50b66      otherFunctions            block_exec
## 16: ad0ea27476c50b66      otherFunctions                 eng_r
## 17: ad0ea27476c50b66      otherFunctions          in_input_dir
## 18: ad0ea27476c50b66      otherFunctions                in_dir
## 19: ad0ea27476c50b66      otherFunctions             timing_fn
## 20: ad0ea27476c50b66      otherFunctions                handle
## 21: ad0ea27476c50b66           preDigest    n:7eef4eae85fd9229
## 22: ad0ea27476c50b66           preDigest mean:c40c00762a0dac94
## 23: ad0ea27476c50b66           preDigest   sd:853b1797f54b229c
## 24: ad0ea27476c50b66           preDigest .FUN:4f604aa46882b368
## 25: ad0ea27476c50b66           file.size                   175
## 26: ad0ea27476c50b66            accessed   2022-12-21 22:54:23
## 27: ad0ea27476c50b66     elapsedTimeLoad       0.00207448 secs
## 28: ad0ea27476c50b66            accessed   2022-12-21 22:54:23
## 29: deaa37372f85861b          objectName                     b
## 30: deaa37372f85861b            function                 runif
## 31: deaa37372f85861b               class               numeric
## 32: deaa37372f85861b         object.size                  1008
## 33: deaa37372f85861b            accessed   2022-12-21 22:54:23
## 34: deaa37372f85861b             inCloud                 FALSE
## 35: deaa37372f85861b          resultHash                      
## 36: deaa37372f85861b   elapsedTimeDigest      0.001382589 secs
## 37: deaa37372f85861b elapsedTimeFirstRun     1.549721e-05 secs
## 38: deaa37372f85861b      otherFunctions      vweave_rmarkdown
## 39: deaa37372f85861b      otherFunctions          process_file
## 40: deaa37372f85861b      otherFunctions         process_group
## 41: deaa37372f85861b      otherFunctions   process_group.block
## 42: deaa37372f85861b      otherFunctions            call_block
## 43: deaa37372f85861b      otherFunctions            block_exec
## 44: deaa37372f85861b      otherFunctions                 eng_r
## 45: deaa37372f85861b      otherFunctions          in_input_dir
## 46: deaa37372f85861b      otherFunctions                in_dir
## 47: deaa37372f85861b      otherFunctions             timing_fn
## 48: deaa37372f85861b      otherFunctions                handle
## 49: deaa37372f85861b           preDigest    n:7eef4eae85fd9229
## 50: deaa37372f85861b           preDigest  min:c40c00762a0dac94
## 51: deaa37372f85861b           preDigest  max:853b1797f54b229c
## 52: deaa37372f85861b           preDigest .FUN:881ec847b7161f3c
## 53: deaa37372f85861b           file.size                   170
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:22
##  2: 2022-12-21 22:54:22
##  3: 2022-12-21 22:54:22
##  4: 2022-12-21 22:54:22
##  5: 2022-12-21 22:54:22
##  6: 2022-12-21 22:54:22
##  7: 2022-12-21 22:54:22
##  8: 2022-12-21 22:54:22
##  9: 2022-12-21 22:54:22
## 10: 2022-12-21 22:54:22
## 11: 2022-12-21 22:54:22
## 12: 2022-12-21 22:54:22
## 13: 2022-12-21 22:54:22
## 14: 2022-12-21 22:54:22
## 15: 2022-12-21 22:54:22
## 16: 2022-12-21 22:54:22
## 17: 2022-12-21 22:54:22
## 18: 2022-12-21 22:54:22
## 19: 2022-12-21 22:54:22
## 20: 2022-12-21 22:54:22
## 21: 2022-12-21 22:54:22
## 22: 2022-12-21 22:54:22
## 23: 2022-12-21 22:54:22
## 24: 2022-12-21 22:54:22
## 25: 2022-12-21 22:54:22
## 26: 2022-12-21 22:54:23
## 27: 2022-12-21 22:54:23
## 28: 2022-12-21 22:54:23
## 29: 2022-12-21 22:54:23
## 30: 2022-12-21 22:54:23
## 31: 2022-12-21 22:54:23
## 32: 2022-12-21 22:54:23
## 33: 2022-12-21 22:54:23
## 34: 2022-12-21 22:54:23
## 35: 2022-12-21 22:54:23
## 36: 2022-12-21 22:54:23
## 37: 2022-12-21 22:54:23
## 38: 2022-12-21 22:54:23
## 39: 2022-12-21 22:54:23
## 40: 2022-12-21 22:54:23
## 41: 2022-12-21 22:54:23
## 42: 2022-12-21 22:54:23
## 43: 2022-12-21 22:54:23
## 44: 2022-12-21 22:54:23
## 45: 2022-12-21 22:54:23
## 46: 2022-12-21 22:54:23
## 47: 2022-12-21 22:54:23
## 48: 2022-12-21 22:54:23
## 49: 2022-12-21 22:54:23
## 50: 2022-12-21 22:54:23
## 51: 2022-12-21 22:54:23
## 52: 2022-12-21 22:54:23
## 53: 2022-12-21 22:54:23
##             createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##              cacheId              tagKey              tagValue
##  1: ad0ea27476c50b66          objectName                     a
##  2: ad0ea27476c50b66            function                 rnorm
##  3: ad0ea27476c50b66               class               numeric
##  4: ad0ea27476c50b66         object.size                  1008
##  5: ad0ea27476c50b66            accessed   2022-12-21 22:54:22
##  6: ad0ea27476c50b66             inCloud                 FALSE
##  7: ad0ea27476c50b66          resultHash                      
##  8: ad0ea27476c50b66   elapsedTimeDigest      0.001512051 secs
##  9: ad0ea27476c50b66 elapsedTimeFirstRun     1.573563e-05 secs
## 10: ad0ea27476c50b66      otherFunctions      vweave_rmarkdown
## 11: ad0ea27476c50b66      otherFunctions          process_file
## 12: ad0ea27476c50b66      otherFunctions         process_group
## 13: ad0ea27476c50b66      otherFunctions   process_group.block
## 14: ad0ea27476c50b66      otherFunctions            call_block
## 15: ad0ea27476c50b66      otherFunctions            block_exec
## 16: ad0ea27476c50b66      otherFunctions                 eng_r
## 17: ad0ea27476c50b66      otherFunctions          in_input_dir
## 18: ad0ea27476c50b66      otherFunctions                in_dir
## 19: ad0ea27476c50b66      otherFunctions             timing_fn
## 20: ad0ea27476c50b66      otherFunctions                handle
## 21: ad0ea27476c50b66           preDigest    n:7eef4eae85fd9229
## 22: ad0ea27476c50b66           preDigest mean:c40c00762a0dac94
## 23: ad0ea27476c50b66           preDigest   sd:853b1797f54b229c
## 24: ad0ea27476c50b66           preDigest .FUN:4f604aa46882b368
## 25: ad0ea27476c50b66           file.size                   175
## 26: ad0ea27476c50b66            accessed   2022-12-21 22:54:23
## 27: ad0ea27476c50b66     elapsedTimeLoad       0.00207448 secs
## 28: ad0ea27476c50b66            accessed   2022-12-21 22:54:23
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:22
##  2: 2022-12-21 22:54:22
##  3: 2022-12-21 22:54:22
##  4: 2022-12-21 22:54:22
##  5: 2022-12-21 22:54:22
##  6: 2022-12-21 22:54:22
##  7: 2022-12-21 22:54:22
##  8: 2022-12-21 22:54:22
##  9: 2022-12-21 22:54:22
## 10: 2022-12-21 22:54:22
## 11: 2022-12-21 22:54:22
## 12: 2022-12-21 22:54:22
## 13: 2022-12-21 22:54:22
## 14: 2022-12-21 22:54:22
## 15: 2022-12-21 22:54:22
## 16: 2022-12-21 22:54:22
## 17: 2022-12-21 22:54:22
## 18: 2022-12-21 22:54:22
## 19: 2022-12-21 22:54:22
## 20: 2022-12-21 22:54:22
## 21: 2022-12-21 22:54:22
## 22: 2022-12-21 22:54:22
## 23: 2022-12-21 22:54:22
## 24: 2022-12-21 22:54:22
## 25: 2022-12-21 22:54:22
## 26: 2022-12-21 22:54:23
## 27: 2022-12-21 22:54:23
## 28: 2022-12-21 22:54:23
##             createdDate
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows and 4 cols): cacheId,tagKey,tagValue,createdDate
# Also, can set a time before caching happens and remove based on this
#  --> a useful, simple way to control Cache
ranNumsA <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:a")
startTime <- Sys.time()
Sys.sleep(1)
ranNumsB <- Cache(rnorm, 5, cachePath = tmpDir, userTags = "objectName:b")
keepCache(tmpDir, after = startTime, ask = FALSE) # keep only those newer than startTime
## Cache size:
##   Total (including Rasters): 256 bytes
##   Selected objects (not including Rasters): 256 bytes
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##              cacheId              tagKey              tagValue
##  1: ccacbf62081a42b4          objectName                     b
##  2: ccacbf62081a42b4            function                 rnorm
##  3: ccacbf62081a42b4               class               numeric
##  4: ccacbf62081a42b4         object.size                  1024
##  5: ccacbf62081a42b4            accessed   2022-12-21 22:54:24
##  6: ccacbf62081a42b4             inCloud                 FALSE
##  7: ccacbf62081a42b4          resultHash                      
##  8: ccacbf62081a42b4   elapsedTimeDigest      0.002631187 secs
##  9: ccacbf62081a42b4 elapsedTimeFirstRun     1.430511e-05 secs
## 10: ccacbf62081a42b4      otherFunctions      vweave_rmarkdown
## 11: ccacbf62081a42b4      otherFunctions          process_file
## 12: ccacbf62081a42b4      otherFunctions         process_group
## 13: ccacbf62081a42b4      otherFunctions   process_group.block
## 14: ccacbf62081a42b4      otherFunctions            call_block
## 15: ccacbf62081a42b4      otherFunctions            block_exec
## 16: ccacbf62081a42b4      otherFunctions                 eng_r
## 17: ccacbf62081a42b4      otherFunctions          in_input_dir
## 18: ccacbf62081a42b4      otherFunctions                in_dir
## 19: ccacbf62081a42b4      otherFunctions             timing_fn
## 20: ccacbf62081a42b4      otherFunctions                handle
## 21: ccacbf62081a42b4           preDigest    n:a4f076b3db622faf
## 22: ccacbf62081a42b4           preDigest mean:c40c00762a0dac94
## 23: ccacbf62081a42b4           preDigest   sd:853b1797f54b229c
## 24: ccacbf62081a42b4           preDigest .FUN:4f604aa46882b368
## 25: ccacbf62081a42b4           file.size                   181
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:24
##  2: 2022-12-21 22:54:24
##  3: 2022-12-21 22:54:24
##  4: 2022-12-21 22:54:24
##  5: 2022-12-21 22:54:24
##  6: 2022-12-21 22:54:24
##  7: 2022-12-21 22:54:24
##  8: 2022-12-21 22:54:24
##  9: 2022-12-21 22:54:24
## 10: 2022-12-21 22:54:24
## 11: 2022-12-21 22:54:24
## 12: 2022-12-21 22:54:24
## 13: 2022-12-21 22:54:24
## 14: 2022-12-21 22:54:24
## 15: 2022-12-21 22:54:24
## 16: 2022-12-21 22:54:24
## 17: 2022-12-21 22:54:24
## 18: 2022-12-21 22:54:24
## 19: 2022-12-21 22:54:24
## 20: 2022-12-21 22:54:24
## 21: 2022-12-21 22:54:24
## 22: 2022-12-21 22:54:24
## 23: 2022-12-21 22:54:24
## 24: 2022-12-21 22:54:24
## 25: 2022-12-21 22:54:24
##             createdDate
clearCache(tmpDir, ask = FALSE)

1.3.1.4 Example 4: searching for multiple objects in the cache

# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cachePath = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:b")

# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: ad0ea27476c50b66          objectName                     b
##  2: ad0ea27476c50b66            function                 rnorm
##  3: ad0ea27476c50b66               class               numeric
##  4: ad0ea27476c50b66         object.size                  1008
##  5: ad0ea27476c50b66            accessed   2022-12-21 22:54:24
##  6: ad0ea27476c50b66             inCloud                 FALSE
##  7: ad0ea27476c50b66          resultHash                      
##  8: ad0ea27476c50b66   elapsedTimeDigest      0.001421928 secs
##  9: ad0ea27476c50b66 elapsedTimeFirstRun     1.597404e-05 secs
## 10: ad0ea27476c50b66      otherFunctions      vweave_rmarkdown
## 11: ad0ea27476c50b66      otherFunctions          process_file
## 12: ad0ea27476c50b66      otherFunctions         process_group
## 13: ad0ea27476c50b66      otherFunctions   process_group.block
## 14: ad0ea27476c50b66      otherFunctions            call_block
## 15: ad0ea27476c50b66      otherFunctions            block_exec
## 16: ad0ea27476c50b66      otherFunctions                 eng_r
## 17: ad0ea27476c50b66      otherFunctions          in_input_dir
## 18: ad0ea27476c50b66      otherFunctions                in_dir
## 19: ad0ea27476c50b66      otherFunctions             timing_fn
## 20: ad0ea27476c50b66      otherFunctions                handle
## 21: ad0ea27476c50b66           preDigest    n:7eef4eae85fd9229
## 22: ad0ea27476c50b66           preDigest mean:c40c00762a0dac94
## 23: ad0ea27476c50b66           preDigest   sd:853b1797f54b229c
## 24: ad0ea27476c50b66           preDigest .FUN:4f604aa46882b368
## 25: ad0ea27476c50b66           file.size                   173
## 26: deaa37372f85861b          objectName                     a
## 27: deaa37372f85861b            function                 runif
## 28: deaa37372f85861b               class               numeric
## 29: deaa37372f85861b         object.size                  1008
## 30: deaa37372f85861b            accessed   2022-12-21 22:54:24
## 31: deaa37372f85861b             inCloud                 FALSE
## 32: deaa37372f85861b          resultHash                      
## 33: deaa37372f85861b   elapsedTimeDigest      0.001481533 secs
## 34: deaa37372f85861b elapsedTimeFirstRun     1.645088e-05 secs
## 35: deaa37372f85861b      otherFunctions      vweave_rmarkdown
## 36: deaa37372f85861b      otherFunctions          process_file
## 37: deaa37372f85861b      otherFunctions         process_group
## 38: deaa37372f85861b      otherFunctions   process_group.block
## 39: deaa37372f85861b      otherFunctions            call_block
## 40: deaa37372f85861b      otherFunctions            block_exec
## 41: deaa37372f85861b      otherFunctions                 eng_r
## 42: deaa37372f85861b      otherFunctions          in_input_dir
## 43: deaa37372f85861b      otherFunctions                in_dir
## 44: deaa37372f85861b      otherFunctions             timing_fn
## 45: deaa37372f85861b      otherFunctions                handle
## 46: deaa37372f85861b           preDigest    n:7eef4eae85fd9229
## 47: deaa37372f85861b           preDigest  min:c40c00762a0dac94
## 48: deaa37372f85861b           preDigest  max:853b1797f54b229c
## 49: deaa37372f85861b           preDigest .FUN:881ec847b7161f3c
## 50: deaa37372f85861b           file.size                   170
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:24
##  2: 2022-12-21 22:54:24
##  3: 2022-12-21 22:54:24
##  4: 2022-12-21 22:54:24
##  5: 2022-12-21 22:54:24
##  6: 2022-12-21 22:54:24
##  7: 2022-12-21 22:54:24
##  8: 2022-12-21 22:54:24
##  9: 2022-12-21 22:54:24
## 10: 2022-12-21 22:54:24
## 11: 2022-12-21 22:54:24
## 12: 2022-12-21 22:54:24
## 13: 2022-12-21 22:54:24
## 14: 2022-12-21 22:54:24
## 15: 2022-12-21 22:54:24
## 16: 2022-12-21 22:54:24
## 17: 2022-12-21 22:54:24
## 18: 2022-12-21 22:54:24
## 19: 2022-12-21 22:54:24
## 20: 2022-12-21 22:54:24
## 21: 2022-12-21 22:54:24
## 22: 2022-12-21 22:54:24
## 23: 2022-12-21 22:54:24
## 24: 2022-12-21 22:54:24
## 25: 2022-12-21 22:54:24
## 26: 2022-12-21 22:54:24
## 27: 2022-12-21 22:54:24
## 28: 2022-12-21 22:54:24
## 29: 2022-12-21 22:54:24
## 30: 2022-12-21 22:54:24
## 31: 2022-12-21 22:54:24
## 32: 2022-12-21 22:54:24
## 33: 2022-12-21 22:54:24
## 34: 2022-12-21 22:54:24
## 35: 2022-12-21 22:54:24
## 36: 2022-12-21 22:54:24
## 37: 2022-12-21 22:54:24
## 38: 2022-12-21 22:54:24
## 39: 2022-12-21 22:54:24
## 40: 2022-12-21 22:54:24
## 41: 2022-12-21 22:54:24
## 42: 2022-12-21 22:54:24
## 43: 2022-12-21 22:54:24
## 44: 2022-12-21 22:54:24
## 45: 2022-12-21 22:54:24
## 46: 2022-12-21 22:54:24
## 47: 2022-12-21 22:54:24
## 48: 2022-12-21 22:54:24
## 49: 2022-12-21 22:54:24
## 50: 2022-12-21 22:54:24
##             createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows and 4 cols): cacheId,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: ad0ea27476c50b66          objectName                     b
##  2: ad0ea27476c50b66            function                 rnorm
##  3: ad0ea27476c50b66               class               numeric
##  4: ad0ea27476c50b66         object.size                  1008
##  5: ad0ea27476c50b66            accessed   2022-12-21 22:54:24
##  6: ad0ea27476c50b66             inCloud                 FALSE
##  7: ad0ea27476c50b66          resultHash                      
##  8: ad0ea27476c50b66   elapsedTimeDigest      0.001421928 secs
##  9: ad0ea27476c50b66 elapsedTimeFirstRun     1.597404e-05 secs
## 10: ad0ea27476c50b66      otherFunctions      vweave_rmarkdown
## 11: ad0ea27476c50b66      otherFunctions          process_file
## 12: ad0ea27476c50b66      otherFunctions         process_group
## 13: ad0ea27476c50b66      otherFunctions   process_group.block
## 14: ad0ea27476c50b66      otherFunctions            call_block
## 15: ad0ea27476c50b66      otherFunctions            block_exec
## 16: ad0ea27476c50b66      otherFunctions                 eng_r
## 17: ad0ea27476c50b66      otherFunctions          in_input_dir
## 18: ad0ea27476c50b66      otherFunctions                in_dir
## 19: ad0ea27476c50b66      otherFunctions             timing_fn
## 20: ad0ea27476c50b66      otherFunctions                handle
## 21: ad0ea27476c50b66           preDigest    n:7eef4eae85fd9229
## 22: ad0ea27476c50b66           preDigest mean:c40c00762a0dac94
## 23: ad0ea27476c50b66           preDigest   sd:853b1797f54b229c
## 24: ad0ea27476c50b66           preDigest .FUN:4f604aa46882b368
## 25: ad0ea27476c50b66           file.size                   173
## 26: deaa37372f85861b          objectName                     a
## 27: deaa37372f85861b            function                 runif
## 28: deaa37372f85861b               class               numeric
## 29: deaa37372f85861b         object.size                  1008
## 30: deaa37372f85861b            accessed   2022-12-21 22:54:24
## 31: deaa37372f85861b             inCloud                 FALSE
## 32: deaa37372f85861b          resultHash                      
## 33: deaa37372f85861b   elapsedTimeDigest      0.001481533 secs
## 34: deaa37372f85861b elapsedTimeFirstRun     1.645088e-05 secs
## 35: deaa37372f85861b      otherFunctions      vweave_rmarkdown
## 36: deaa37372f85861b      otherFunctions          process_file
## 37: deaa37372f85861b      otherFunctions         process_group
## 38: deaa37372f85861b      otherFunctions   process_group.block
## 39: deaa37372f85861b      otherFunctions            call_block
## 40: deaa37372f85861b      otherFunctions            block_exec
## 41: deaa37372f85861b      otherFunctions                 eng_r
## 42: deaa37372f85861b      otherFunctions          in_input_dir
## 43: deaa37372f85861b      otherFunctions                in_dir
## 44: deaa37372f85861b      otherFunctions             timing_fn
## 45: deaa37372f85861b      otherFunctions                handle
## 46: deaa37372f85861b           preDigest    n:7eef4eae85fd9229
## 47: deaa37372f85861b           preDigest  min:c40c00762a0dac94
## 48: deaa37372f85861b           preDigest  max:853b1797f54b229c
## 49: deaa37372f85861b           preDigest .FUN:881ec847b7161f3c
## 50: deaa37372f85861b           file.size                   170
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:24
##  2: 2022-12-21 22:54:24
##  3: 2022-12-21 22:54:24
##  4: 2022-12-21 22:54:24
##  5: 2022-12-21 22:54:24
##  6: 2022-12-21 22:54:24
##  7: 2022-12-21 22:54:24
##  8: 2022-12-21 22:54:24
##  9: 2022-12-21 22:54:24
## 10: 2022-12-21 22:54:24
## 11: 2022-12-21 22:54:24
## 12: 2022-12-21 22:54:24
## 13: 2022-12-21 22:54:24
## 14: 2022-12-21 22:54:24
## 15: 2022-12-21 22:54:24
## 16: 2022-12-21 22:54:24
## 17: 2022-12-21 22:54:24
## 18: 2022-12-21 22:54:24
## 19: 2022-12-21 22:54:24
## 20: 2022-12-21 22:54:24
## 21: 2022-12-21 22:54:24
## 22: 2022-12-21 22:54:24
## 23: 2022-12-21 22:54:24
## 24: 2022-12-21 22:54:24
## 25: 2022-12-21 22:54:24
## 26: 2022-12-21 22:54:24
## 27: 2022-12-21 22:54:24
## 28: 2022-12-21 22:54:24
## 29: 2022-12-21 22:54:24
## 30: 2022-12-21 22:54:24
## 31: 2022-12-21 22:54:24
## 32: 2022-12-21 22:54:24
## 33: 2022-12-21 22:54:24
## 34: 2022-12-21 22:54:24
## 35: 2022-12-21 22:54:24
## 36: 2022-12-21 22:54:24
## 37: 2022-12-21 22:54:24
## 38: 2022-12-21 22:54:24
## 39: 2022-12-21 22:54:24
## 40: 2022-12-21 22:54:24
## 41: 2022-12-21 22:54:24
## 42: 2022-12-21 22:54:24
## 43: 2022-12-21 22:54:24
## 44: 2022-12-21 22:54:24
## 45: 2022-12-21 22:54:24
## 46: 2022-12-21 22:54:24
## 47: 2022-12-21 22:54:24
## 48: 2022-12-21 22:54:24
## 49: 2022-12-21 22:54:24
## 50: 2022-12-21 22:54:24
##             createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: ad0ea27476c50b66          objectName                     b
##  2: ad0ea27476c50b66            function                 rnorm
##  3: ad0ea27476c50b66               class               numeric
##  4: ad0ea27476c50b66         object.size                  1008
##  5: ad0ea27476c50b66            accessed   2022-12-21 22:54:24
##  6: ad0ea27476c50b66             inCloud                 FALSE
##  7: ad0ea27476c50b66          resultHash                      
##  8: ad0ea27476c50b66   elapsedTimeDigest      0.001421928 secs
##  9: ad0ea27476c50b66 elapsedTimeFirstRun     1.597404e-05 secs
## 10: ad0ea27476c50b66      otherFunctions      vweave_rmarkdown
## 11: ad0ea27476c50b66      otherFunctions          process_file
## 12: ad0ea27476c50b66      otherFunctions         process_group
## 13: ad0ea27476c50b66      otherFunctions   process_group.block
## 14: ad0ea27476c50b66      otherFunctions            call_block
## 15: ad0ea27476c50b66      otherFunctions            block_exec
## 16: ad0ea27476c50b66      otherFunctions                 eng_r
## 17: ad0ea27476c50b66      otherFunctions          in_input_dir
## 18: ad0ea27476c50b66      otherFunctions                in_dir
## 19: ad0ea27476c50b66      otherFunctions             timing_fn
## 20: ad0ea27476c50b66      otherFunctions                handle
## 21: ad0ea27476c50b66           preDigest    n:7eef4eae85fd9229
## 22: ad0ea27476c50b66           preDigest mean:c40c00762a0dac94
## 23: ad0ea27476c50b66           preDigest   sd:853b1797f54b229c
## 24: ad0ea27476c50b66           preDigest .FUN:4f604aa46882b368
## 25: ad0ea27476c50b66           file.size                   173
## 26: deaa37372f85861b          objectName                     a
## 27: deaa37372f85861b            function                 runif
## 28: deaa37372f85861b               class               numeric
## 29: deaa37372f85861b         object.size                  1008
## 30: deaa37372f85861b            accessed   2022-12-21 22:54:24
## 31: deaa37372f85861b             inCloud                 FALSE
## 32: deaa37372f85861b          resultHash                      
## 33: deaa37372f85861b   elapsedTimeDigest      0.001481533 secs
## 34: deaa37372f85861b elapsedTimeFirstRun     1.645088e-05 secs
## 35: deaa37372f85861b      otherFunctions      vweave_rmarkdown
## 36: deaa37372f85861b      otherFunctions          process_file
## 37: deaa37372f85861b      otherFunctions         process_group
## 38: deaa37372f85861b      otherFunctions   process_group.block
## 39: deaa37372f85861b      otherFunctions            call_block
## 40: deaa37372f85861b      otherFunctions            block_exec
## 41: deaa37372f85861b      otherFunctions                 eng_r
## 42: deaa37372f85861b      otherFunctions          in_input_dir
## 43: deaa37372f85861b      otherFunctions                in_dir
## 44: deaa37372f85861b      otherFunctions             timing_fn
## 45: deaa37372f85861b      otherFunctions                handle
## 46: deaa37372f85861b           preDigest    n:7eef4eae85fd9229
## 47: deaa37372f85861b           preDigest  min:c40c00762a0dac94
## 48: deaa37372f85861b           preDigest  max:853b1797f54b229c
## 49: deaa37372f85861b           preDigest .FUN:881ec847b7161f3c
## 50: deaa37372f85861b           file.size                   170
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:24
##  2: 2022-12-21 22:54:24
##  3: 2022-12-21 22:54:24
##  4: 2022-12-21 22:54:24
##  5: 2022-12-21 22:54:24
##  6: 2022-12-21 22:54:24
##  7: 2022-12-21 22:54:24
##  8: 2022-12-21 22:54:24
##  9: 2022-12-21 22:54:24
## 10: 2022-12-21 22:54:24
## 11: 2022-12-21 22:54:24
## 12: 2022-12-21 22:54:24
## 13: 2022-12-21 22:54:24
## 14: 2022-12-21 22:54:24
## 15: 2022-12-21 22:54:24
## 16: 2022-12-21 22:54:24
## 17: 2022-12-21 22:54:24
## 18: 2022-12-21 22:54:24
## 19: 2022-12-21 22:54:24
## 20: 2022-12-21 22:54:24
## 21: 2022-12-21 22:54:24
## 22: 2022-12-21 22:54:24
## 23: 2022-12-21 22:54:24
## 24: 2022-12-21 22:54:24
## 25: 2022-12-21 22:54:24
## 26: 2022-12-21 22:54:24
## 27: 2022-12-21 22:54:24
## 28: 2022-12-21 22:54:24
## 29: 2022-12-21 22:54:24
## 30: 2022-12-21 22:54:24
## 31: 2022-12-21 22:54:24
## 32: 2022-12-21 22:54:24
## 33: 2022-12-21 22:54:24
## 34: 2022-12-21 22:54:24
## 35: 2022-12-21 22:54:24
## 36: 2022-12-21 22:54:24
## 37: 2022-12-21 22:54:24
## 38: 2022-12-21 22:54:24
## 39: 2022-12-21 22:54:24
## 40: 2022-12-21 22:54:24
## 41: 2022-12-21 22:54:24
## 42: 2022-12-21 22:54:24
## 43: 2022-12-21 22:54:24
## 44: 2022-12-21 22:54:24
## 45: 2022-12-21 22:54:24
## 46: 2022-12-21 22:54:24
## 47: 2022-12-21 22:54:24
## 48: 2022-12-21 22:54:24
## 49: 2022-12-21 22:54:24
## 50: 2022-12-21 22:54:24
##             createdDate
clearCache(tmpDir, ask = FALSE)

1.3.1.5 Example 5: using caching to speed up rerunning expensive computations

ras <- raster(extent(0, 5, 0, 5), res = 1,
              vals = sample(1:5, replace = TRUE, size = 25),
              crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")

# A slow operation, like GIS operation
notCached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  projectRaster(ras, crs = crs(ras), res = 5, cachePath = tmpDir)
)

cached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  # using quote works also
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cachePath = tmpDir)
)

# second time is much faster
reRun <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cachePath = tmpDir)
)
##   ...(Object to retrieve (0c54a7a26e836024.rds))
##      loaded cached result from previous projectRaster call
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE
## [1] TRUE

1.3.2 Nested Caching

Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less than a few seconds. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.

##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
  Cache(inner, 0.1, cachePath = tmpdir2)
}

# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")

# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
#   but cachePath is tmpdir1 in top level Cache and all nested
#   Cache calls, unless individually overridden ... here inner
#   uses tmpdir2 repository
Cache(outer, n = 2, cachePath = tmpdir1, notOlderThan = Sys.time())
## [1] 1.0861434 0.4903813 1.2225930
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:dffc8e3bf8c23b6e"
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: dffc8e3bf8c23b6e            function                 outer
##  2: dffc8e3bf8c23b6e               class               numeric
##  3: dffc8e3bf8c23b6e         object.size                  1008
##  4: dffc8e3bf8c23b6e            accessed   2022-12-21 22:54:24
##  5: dffc8e3bf8c23b6e             inCloud                 FALSE
##  6: dffc8e3bf8c23b6e          resultHash                      
##  7: dffc8e3bf8c23b6e   elapsedTimeDigest      0.001378536 secs
##  8: dffc8e3bf8c23b6e elapsedTimeFirstRun        0.0258007 secs
##  9: dffc8e3bf8c23b6e      otherFunctions      vweave_rmarkdown
## 10: dffc8e3bf8c23b6e      otherFunctions          process_file
## 11: dffc8e3bf8c23b6e      otherFunctions         process_group
## 12: dffc8e3bf8c23b6e      otherFunctions   process_group.block
## 13: dffc8e3bf8c23b6e      otherFunctions            call_block
## 14: dffc8e3bf8c23b6e      otherFunctions            block_exec
## 15: dffc8e3bf8c23b6e      otherFunctions                 eng_r
## 16: dffc8e3bf8c23b6e      otherFunctions          in_input_dir
## 17: dffc8e3bf8c23b6e      otherFunctions                in_dir
## 18: dffc8e3bf8c23b6e      otherFunctions             timing_fn
## 19: dffc8e3bf8c23b6e      otherFunctions                handle
## 20: dffc8e3bf8c23b6e           preDigest    n:82dc709f2b91918a
## 21: dffc8e3bf8c23b6e           preDigest .FUN:b89dd186387954a0
## 22: dffc8e3bf8c23b6e           file.size                   165
## 23: efa1ccee79a31d4c            function                 rnorm
## 24: efa1ccee79a31d4c               class               numeric
## 25: efa1ccee79a31d4c         object.size                  1008
## 26: efa1ccee79a31d4c            accessed   2022-12-21 22:54:24
## 27: efa1ccee79a31d4c             inCloud                 FALSE
## 28: efa1ccee79a31d4c          resultHash                      
## 29: efa1ccee79a31d4c   elapsedTimeDigest      0.001420498 secs
## 30: efa1ccee79a31d4c elapsedTimeFirstRun     1.478195e-05 secs
## 31: efa1ccee79a31d4c      otherFunctions      vweave_rmarkdown
## 32: efa1ccee79a31d4c      otherFunctions          process_file
## 33: efa1ccee79a31d4c      otherFunctions         process_group
## 34: efa1ccee79a31d4c      otherFunctions   process_group.block
## 35: efa1ccee79a31d4c      otherFunctions            call_block
## 36: efa1ccee79a31d4c      otherFunctions            block_exec
## 37: efa1ccee79a31d4c      otherFunctions                 eng_r
## 38: efa1ccee79a31d4c      otherFunctions          in_input_dir
## 39: efa1ccee79a31d4c      otherFunctions                in_dir
## 40: efa1ccee79a31d4c      otherFunctions             timing_fn
## 41: efa1ccee79a31d4c      otherFunctions                handle
## 42: efa1ccee79a31d4c      otherFunctions                 outer
## 43: efa1ccee79a31d4c      otherFunctions                 inner
## 44: efa1ccee79a31d4c           preDigest    n:7f12988bd88a0fb8
## 45: efa1ccee79a31d4c           preDigest mean:22413394efd9f6a3
## 46: efa1ccee79a31d4c           preDigest   sd:853b1797f54b229c
## 47: efa1ccee79a31d4c           preDigest .FUN:4f604aa46882b368
## 48: efa1ccee79a31d4c           file.size                   165
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:24
##  2: 2022-12-21 22:54:24
##  3: 2022-12-21 22:54:24
##  4: 2022-12-21 22:54:24
##  5: 2022-12-21 22:54:24
##  6: 2022-12-21 22:54:24
##  7: 2022-12-21 22:54:24
##  8: 2022-12-21 22:54:24
##  9: 2022-12-21 22:54:24
## 10: 2022-12-21 22:54:24
## 11: 2022-12-21 22:54:24
## 12: 2022-12-21 22:54:24
## 13: 2022-12-21 22:54:24
## 14: 2022-12-21 22:54:24
## 15: 2022-12-21 22:54:24
## 16: 2022-12-21 22:54:24
## 17: 2022-12-21 22:54:24
## 18: 2022-12-21 22:54:24
## 19: 2022-12-21 22:54:24
## 20: 2022-12-21 22:54:24
## 21: 2022-12-21 22:54:24
## 22: 2022-12-21 22:54:24
## 23: 2022-12-21 22:54:24
## 24: 2022-12-21 22:54:24
## 25: 2022-12-21 22:54:24
## 26: 2022-12-21 22:54:24
## 27: 2022-12-21 22:54:24
## 28: 2022-12-21 22:54:24
## 29: 2022-12-21 22:54:24
## 30: 2022-12-21 22:54:24
## 31: 2022-12-21 22:54:24
## 32: 2022-12-21 22:54:24
## 33: 2022-12-21 22:54:24
## 34: 2022-12-21 22:54:24
## 35: 2022-12-21 22:54:24
## 36: 2022-12-21 22:54:24
## 37: 2022-12-21 22:54:24
## 38: 2022-12-21 22:54:24
## 39: 2022-12-21 22:54:24
## 40: 2022-12-21 22:54:24
## 41: 2022-12-21 22:54:24
## 42: 2022-12-21 22:54:24
## 43: 2022-12-21 22:54:24
## 44: 2022-12-21 22:54:24
## 45: 2022-12-21 22:54:24
## 46: 2022-12-21 22:54:24
## 47: 2022-12-21 22:54:24
## 48: 2022-12-21 22:54:24
##             createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##              cacheId              tagKey              tagValue
##  1: 33ceb4fb525fd08f            function                 inner
##  2: 33ceb4fb525fd08f               class               numeric
##  3: 33ceb4fb525fd08f         object.size                  1008
##  4: 33ceb4fb525fd08f            accessed   2022-12-21 22:54:24
##  5: 33ceb4fb525fd08f             inCloud                 FALSE
##  6: 33ceb4fb525fd08f          resultHash                      
##  7: 33ceb4fb525fd08f   elapsedTimeDigest      0.001311779 secs
##  8: 33ceb4fb525fd08f elapsedTimeFirstRun      0.009453058 secs
##  9: 33ceb4fb525fd08f      otherFunctions      vweave_rmarkdown
## 10: 33ceb4fb525fd08f      otherFunctions          process_file
## 11: 33ceb4fb525fd08f      otherFunctions         process_group
## 12: 33ceb4fb525fd08f      otherFunctions   process_group.block
## 13: 33ceb4fb525fd08f      otherFunctions            call_block
## 14: 33ceb4fb525fd08f      otherFunctions            block_exec
## 15: 33ceb4fb525fd08f      otherFunctions                 eng_r
## 16: 33ceb4fb525fd08f      otherFunctions          in_input_dir
## 17: 33ceb4fb525fd08f      otherFunctions                in_dir
## 18: 33ceb4fb525fd08f      otherFunctions             timing_fn
## 19: 33ceb4fb525fd08f      otherFunctions                handle
## 20: 33ceb4fb525fd08f      otherFunctions                 outer
## 21: 33ceb4fb525fd08f           preDigest mean:22413394efd9f6a3
## 22: 33ceb4fb525fd08f           preDigest .FUN:87e2c30917a34d25
## 23: 33ceb4fb525fd08f           file.size                   165
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:24
##  2: 2022-12-21 22:54:24
##  3: 2022-12-21 22:54:24
##  4: 2022-12-21 22:54:24
##  5: 2022-12-21 22:54:24
##  6: 2022-12-21 22:54:24
##  7: 2022-12-21 22:54:24
##  8: 2022-12-21 22:54:24
##  9: 2022-12-21 22:54:24
## 10: 2022-12-21 22:54:24
## 11: 2022-12-21 22:54:24
## 12: 2022-12-21 22:54:24
## 13: 2022-12-21 22:54:24
## 14: 2022-12-21 22:54:24
## 15: 2022-12-21 22:54:24
## 16: 2022-12-21 22:54:24
## 17: 2022-12-21 22:54:24
## 18: 2022-12-21 22:54:24
## 19: 2022-12-21 22:54:24
## 20: 2022-12-21 22:54:24
## 21: 2022-12-21 22:54:24
## 22: 2022-12-21 22:54:24
## 23: 2022-12-21 22:54:24
##             createdDate
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1, ask = FALSE)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
  Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cachePath = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
##   Total (including Rasters): 756 bytes
##   Selected objects (not including Rasters): 756 bytes
##              cacheId              tagKey              tagValue
##  1: 88a34e1d033329e5            outerTag              outerTag
##  2: 88a34e1d033329e5            function                 outer
##  3: 88a34e1d033329e5               class               numeric
##  4: 88a34e1d033329e5         object.size                  1008
##  5: 88a34e1d033329e5            accessed   2022-12-21 22:54:24
##  6: 88a34e1d033329e5             inCloud                 FALSE
##  7: 88a34e1d033329e5          resultHash                      
##  8: 88a34e1d033329e5   elapsedTimeDigest      0.001269102 secs
##  9: 88a34e1d033329e5 elapsedTimeFirstRun       0.02336645 secs
## 10: 88a34e1d033329e5      otherFunctions      vweave_rmarkdown
## 11: 88a34e1d033329e5      otherFunctions          process_file
## 12: 88a34e1d033329e5      otherFunctions         process_group
## 13: 88a34e1d033329e5      otherFunctions   process_group.block
## 14: 88a34e1d033329e5      otherFunctions            call_block
## 15: 88a34e1d033329e5      otherFunctions            block_exec
## 16: 88a34e1d033329e5      otherFunctions                 eng_r
## 17: 88a34e1d033329e5      otherFunctions          in_input_dir
## 18: 88a34e1d033329e5      otherFunctions                in_dir
## 19: 88a34e1d033329e5      otherFunctions             timing_fn
## 20: 88a34e1d033329e5      otherFunctions                handle
## 21: 88a34e1d033329e5           preDigest    n:82dc709f2b91918a
## 22: 88a34e1d033329e5           preDigest .FUN:5f06fb5fbffe9e3b
## 23: 88a34e1d033329e5           file.size                   165
## 24: b06af03d5a73dc7d            outerTag              outerTag
## 25: b06af03d5a73dc7d            function                 inner
## 26: b06af03d5a73dc7d               class               numeric
## 27: b06af03d5a73dc7d         object.size                  1008
## 28: b06af03d5a73dc7d            accessed   2022-12-21 22:54:24
## 29: b06af03d5a73dc7d             inCloud                 FALSE
## 30: b06af03d5a73dc7d          resultHash                      
## 31: b06af03d5a73dc7d   elapsedTimeDigest      0.001157522 secs
## 32: b06af03d5a73dc7d elapsedTimeFirstRun       0.01464105 secs
## 33: b06af03d5a73dc7d      otherFunctions      vweave_rmarkdown
## 34: b06af03d5a73dc7d      otherFunctions          process_file
## 35: b06af03d5a73dc7d      otherFunctions         process_group
## 36: b06af03d5a73dc7d      otherFunctions   process_group.block
## 37: b06af03d5a73dc7d      otherFunctions            call_block
## 38: b06af03d5a73dc7d      otherFunctions            block_exec
## 39: b06af03d5a73dc7d      otherFunctions                 eng_r
## 40: b06af03d5a73dc7d      otherFunctions          in_input_dir
## 41: b06af03d5a73dc7d      otherFunctions                in_dir
## 42: b06af03d5a73dc7d      otherFunctions             timing_fn
## 43: b06af03d5a73dc7d      otherFunctions                handle
## 44: b06af03d5a73dc7d      otherFunctions                 outer
## 45: b06af03d5a73dc7d           preDigest mean:22413394efd9f6a3
## 46: b06af03d5a73dc7d           preDigest .FUN:7ad10bc1ae528d8c
## 47: b06af03d5a73dc7d           file.size                   165
## 48: efa1ccee79a31d4c            innerTag              innerTag
## 49: efa1ccee79a31d4c            outerTag              outerTag
## 50: efa1ccee79a31d4c            function                 rnorm
## 51: efa1ccee79a31d4c               class               numeric
## 52: efa1ccee79a31d4c         object.size                  1008
## 53: efa1ccee79a31d4c            accessed   2022-12-21 22:54:24
## 54: efa1ccee79a31d4c             inCloud                 FALSE
## 55: efa1ccee79a31d4c          resultHash                      
## 56: efa1ccee79a31d4c   elapsedTimeDigest      0.001362801 secs
## 57: efa1ccee79a31d4c elapsedTimeFirstRun     2.026558e-05 secs
## 58: efa1ccee79a31d4c      otherFunctions      vweave_rmarkdown
## 59: efa1ccee79a31d4c      otherFunctions          process_file
## 60: efa1ccee79a31d4c      otherFunctions         process_group
## 61: efa1ccee79a31d4c      otherFunctions   process_group.block
## 62: efa1ccee79a31d4c      otherFunctions            call_block
## 63: efa1ccee79a31d4c      otherFunctions            block_exec
## 64: efa1ccee79a31d4c      otherFunctions                 eng_r
## 65: efa1ccee79a31d4c      otherFunctions          in_input_dir
## 66: efa1ccee79a31d4c      otherFunctions                in_dir
## 67: efa1ccee79a31d4c      otherFunctions             timing_fn
## 68: efa1ccee79a31d4c      otherFunctions                handle
## 69: efa1ccee79a31d4c      otherFunctions                 outer
## 70: efa1ccee79a31d4c      otherFunctions                 inner
## 71: efa1ccee79a31d4c           preDigest    n:7f12988bd88a0fb8
## 72: efa1ccee79a31d4c           preDigest mean:22413394efd9f6a3
## 73: efa1ccee79a31d4c           preDigest   sd:853b1797f54b229c
## 74: efa1ccee79a31d4c           preDigest .FUN:4f604aa46882b368
## 75: efa1ccee79a31d4c           file.size                   165
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2022-12-21 22:54:24
##  2: 2022-12-21 22:54:24
##  3: 2022-12-21 22:54:24
##  4: 2022-12-21 22:54:24
##  5: 2022-12-21 22:54:24
##  6: 2022-12-21 22:54:24
##  7: 2022-12-21 22:54:24
##  8: 2022-12-21 22:54:24
##  9: 2022-12-21 22:54:24
## 10: 2022-12-21 22:54:24
## 11: 2022-12-21 22:54:24
## 12: 2022-12-21 22:54:24
## 13: 2022-12-21 22:54:24
## 14: 2022-12-21 22:54:24
## 15: 2022-12-21 22:54:24
## 16: 2022-12-21 22:54:24
## 17: 2022-12-21 22:54:24
## 18: 2022-12-21 22:54:24
## 19: 2022-12-21 22:54:24
## 20: 2022-12-21 22:54:24
## 21: 2022-12-21 22:54:24
## 22: 2022-12-21 22:54:24
## 23: 2022-12-21 22:54:24
## 24: 2022-12-21 22:54:24
## 25: 2022-12-21 22:54:24
## 26: 2022-12-21 22:54:24
## 27: 2022-12-21 22:54:24
## 28: 2022-12-21 22:54:24
## 29: 2022-12-21 22:54:24
## 30: 2022-12-21 22:54:24
## 31: 2022-12-21 22:54:24
## 32: 2022-12-21 22:54:24
## 33: 2022-12-21 22:54:24
## 34: 2022-12-21 22:54:24
## 35: 2022-12-21 22:54:24
## 36: 2022-12-21 22:54:24
## 37: 2022-12-21 22:54:24
## 38: 2022-12-21 22:54:24
## 39: 2022-12-21 22:54:24
## 40: 2022-12-21 22:54:24
## 41: 2022-12-21 22:54:24
## 42: 2022-12-21 22:54:24
## 43: 2022-12-21 22:54:24
## 44: 2022-12-21 22:54:24
## 45: 2022-12-21 22:54:24
## 46: 2022-12-21 22:54:24
## 47: 2022-12-21 22:54:24
## 48: 2022-12-21 22:54:24
## 49: 2022-12-21 22:54:24
## 50: 2022-12-21 22:54:24
## 51: 2022-12-21 22:54:24
## 52: 2022-12-21 22:54:24
## 53: 2022-12-21 22:54:24
## 54: 2022-12-21 22:54:24
## 55: 2022-12-21 22:54:24
## 56: 2022-12-21 22:54:24
## 57: 2022-12-21 22:54:24
## 58: 2022-12-21 22:54:24
## 59: 2022-12-21 22:54:24
## 60: 2022-12-21 22:54:24
## 61: 2022-12-21 22:54:24
## 62: 2022-12-21 22:54:24
## 63: 2022-12-21 22:54:24
## 64: 2022-12-21 22:54:24
## 65: 2022-12-21 22:54:24
## 66: 2022-12-21 22:54:24
## 67: 2022-12-21 22:54:24
## 68: 2022-12-21 22:54:24
## 69: 2022-12-21 22:54:24
## 70: 2022-12-21 22:54:24
## 71: 2022-12-21 22:54:24
## 72: 2022-12-21 22:54:24
## 73: 2022-12-21 22:54:24
## 74: 2022-12-21 22:54:24
## 75: 2022-12-21 22:54:24
##             createdDate

1.3.3 cacheId

Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".

### cacheId
set.seed(1)
Cache(rnorm, 1, cachePath = tmpdir1)
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:422bae4ed2f770cc"
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: 7072c305d8c69df0
Cache(rnorm, 1, cachePath = tmpdir1, cacheId = "7072c305d8c69df0") # same value
## cacheId is not same as calculated hash. Manually searching for cacheId:7072c305d8c69df0
##   ...(Object to retrieve (422bae4ed2f770cc.rds))
##      loaded cached result from previous rnorm call
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
## 
## attr(,"tags")
## [1] "cacheId:422bae4ed2f770cc"
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cachePath = tmpdir1, cacheId = "7072c305d8c69df0")
## cacheId is not same as calculated hash. Manually searching for cacheId:7072c305d8c69df0
## [1] 0.7383247 0.5757814
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""

1.4 Working with the Cache manually

Since the cache is simply a DBI data table (of an SQLite database by default). In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).

# As of reproducible version 1.0, there is a new backend directly using DBI
mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$cacheId)
## Cache size:
##   Total (including Rasters): 3.4 Kb
##   Selected objects (not including Rasters): 3.4 Kb
map <- loadFromCache(mapHash[1], cachePath = tmpDir)
plot(map)

## cleanup
unlink(dirname(tmpDir), recursive = TRUE)

2 Alternative database backends

By default, caching relies on a sqlite database for it’s backend. While this works in many situations, there are some important limitations of using sqlite for caching, including 1) speed; 2) concurrent transactions; 3) sharing database across machines or projects. Fortunately, Cache makes use of DBI package and thus supports several database backends, including mysql and postgresql.

See https://github.com/PredictiveEcology/SpaDES/wiki/Using-alternate-database-backends-for-Cache for further information on configuring these additional backends.

3 Reproducible workflow

In general, we feel that a liberal use of Cache will make a re-usable and reproducible work flow. shiny apps can be made, taking advantage of Cache. Indeed, much of the difficulty in managing data sets and saving them for future use, can be accommodated by caching.