TensorFlow Integration

Overview

While {reservr} is capable of fitting distributions to censored and truncated observations, it does not directly allow modelling the influence of exogenous variables observed alongside the primary outcome. This is where the integration with TensorFlow comes in.

The TensorFlow integration allows to fit a neural network simultaneously to all parameters of a distribution while taking exogenous variables into account.

{reservr} accepts all partial tensorflow networks which yield a single arbitrary-dimension rank 2 tensor (e.g. any dense layer) as output and can connect suitable layers to this intermediate output such that the complete network predicts the parameters of any pre-specified distribution family.

It also dynamically compiles a suitable conditional likelihood based loss, depending on the type of problem (censoring, truncation), which can be optimized using the keras::fit implementation out-of-the box. This means there is full flexibility with respect to callbacks, optimizers, mini-batching, etc.

library(reservr)
library(tensorflow)
library(keras)
library(tibble)
library(ggplot2)

A simple linear model

The following example will show the code necessary to fit a simple model with the same assumptions as OLS to data. As a true relationship we use \(y = 2 x + \epsilon\) with \(\epsilon \sim \mathcal{N}(0, 1)\). We will not use censoring or truncation.

if (keras::is_keras_available()) {
  set.seed(1431L)
  tensorflow::set_random_seed(1432L)

  dataset <- tibble(
    x = runif(100, min = 10, max = 20),
    y = 2 * x + rnorm(100)
  )

  ggplot2::qplot(x, y, data = dataset)

  # Specify distributional assumption of OLS:
  dist <- dist_normal(sd = 1.0) # OLS assumption: homoskedasticity

  # Optional: Compute a global fit
  global_fit <- fit(dist, dataset$y)

  # Define a neural network
  nnet_input <- layer_input(shape = 1L, name = "x_input")
  # in practice, this would be deeper
  nnet_output <- nnet_input

  optimizer <- if (packageVersion("keras") >= "2.6.0") {
    optimizer_adam(learning_rate = 0.1)
  } else {
    optimizer_adam(lr = 0.1)
  }

  nnet <- tf_compile_model(
    inputs = list(nnet_input),
    intermediate_output = nnet_output,
    dist = dist,
    optimizer = optimizer,
    censoring = FALSE, # Turn off unnecessary features for this problem
    truncation = FALSE
  )

  nnet_fit <- fit(nnet, x = dataset$x, y = dataset$y, epochs = 100L, batch_size = 100L, shuffle = FALSE)

  plot(nnet_fit)

  pred_params <- predict(nnet, data = list(k_constant(dataset$x)))

  lm_fit <- lm(y ~ x, data = dataset)

  dataset$y_pred <- pred_params$mean
  dataset$y_lm <- predict(lm_fit, newdata = dataset, type = "response")

  ggplot(dataset, aes(x = x, y = y)) %+%
    geom_point() %+%
    geom_line(aes(y = y_pred)) %+%
    geom_line(aes(y = y_lm), linetype = 2L)

  coef_nnet <- rev(as.numeric(nnet$model$get_weights()))
  coef_lm <- coef(lm_fit)

  print(coef_nnet)
  print(coef_lm)
}
#> Loaded Tensorflow version 2.9.2
#> Warning: `qplot()` was deprecated in ggplot2 3.4.0.
#> [1] 0.8800764 1.9325488
#> (Intercept)           x 
#>   0.5645856   1.9574191