watcher - Record Function State During Evaluation

Overview

A mechanism to record function state during evaluation for later
analysis. Variable values and meta data are recorded for each
‘top-level’ call. Top-level calls are, roughly speaking, distinct R
expressions that are visible directly in the source of the R function.

This is package is experimental, rough around the edges, and not
currently intended for CRAN publication. It has no tests, and was built
with expediency rather than robustness as the guiding principle. The API
may change completely in future iterations if there are any. More likely
there will be no future updates to the project, except in the unlikely
case it garners substantial interest from others.

See the issues page for
known limitations.

Usage

watcher works by “instrumenting” a target function. Consider a simple
function that adds the integers up to n:

seq_sum <- function(n) {
  x <- 0
  for(i in seq_len(n)) {
    x <- x + i
  }
  x
}
res0 <- seq_sum(10)
res0

## [1] 55

We can make an instrumented version of it, which will behave mostly as
the original except that the variable values at each top-level
evaluation step are attached to the result as the “watch.data”
attribute:

seq_sum_w <- watcher::watch(seq_sum)
res1 <- seq_sum_w(10)
all.equal(res0, res1, check.attributes=FALSE)

## [1] TRUE

watch.dat0 <- attr(res1, 'watch.data')
str(watch.dat0[1:2])

## List of 2
##  $ :List of 2
##   ..$ x: num 0
##   ..$ n: num 10
##   ..- attr(*, "line")= int 2
##  $ :List of 3
##   ..$ i: int 1
##   ..$ x: num 1
##   ..$ n: num 10
##   ..- attr(*, "line")= int 4

Each step of the function evaluation is recorded as a list element. This
is a bit awkward to deal with so we can use simplify_data to make the
data more accessible. For example, scalar variables are turned into
vectors and returned as members of the “.scalar” data frame in the
simplified list.

watch.dat1 <- watcher::simplify_data(watch.dat0)[['.scalar']]
head(watch.dat1, 2)

##   .id .line x  n  i
## 1   1     2 0 10 NA
## 2   2     4 1 10  1

One possible visualization is to plot variable values vs. evaluation
step (.id):

watch.melt <- reshape2::melt(watch.dat1[c('.id', 'x', 'i')], id.vars='.id')
suppressPackageStartupMessages(library(ggplot2))
ggplot(watch.melt, aes(x=.id, y=value, color=variable)) +
  geom_point()

Code And Data

With a little work we can combine the line information with the function
data as we do here with an animation of the insertion sort algorithm:

For more details on how this is done see the vignette
(vignette('watcher', package='watcher')).

Possible Improvements

• Handle nested/recursive calls. Conceptually this isn’t too
  difficult: we just need to add a call that checks .res for the
  watch.* attributes after each top level statement. Simplification
  / visualization will be another matter.
• Protect the symbols used in tracking (i.e. .res, .line, .id)
  to avoid conflicts with existing variables of the same name (or at
  least check for conflict).
• Automate more of what can be automated, particularly the code / plot
  juxtaposition.
• Optimize code; currently we’ve made no effort to make the code
  efficient.

Acknowledgements

• Jim Hester for the instrumentation
  concept which I borrowed from
  covr; if you are interested in
  writing your own instrumented code please base it off of his and not
  mine as I just threw something together quickly with little thought.
• Thomas Lin Pedersen for
  gganimate which I used
  for the insertion sort animation.
• Hadley Wickham for
  ggplot.