Package 'compareMCMCs'

Description

Create transformed parameters from original parameters in MCMC output

Usage

applyConversions(samples, conversions)

Arguments

Details

A conversion specification is a named list. For each element:

• its name will be the name of a new column appended to a samples matrix.

• its value should be a character string that can be parsed as code to calculate elements of the new column. It can use existing column names in samples. Calculations will be done row-wise. Column names are often something like "beta[2]". To have this used as a name, enclose it in backticks, e.g. "`beta[2]`". For example, an entry could be ⁠log_beta2 = "log(`beta\[2\]`)"⁠. A list value of NULL will remove the named column.

The conversion specification list will be processed in order. This allows creating new columns and removing old ones in a sensible order.

If both conversions and samples are named lists, they will be matched: the conversions element (itself a list of conversion specifications) used on a samples element will have the same name. If there is no conversions element for a given samples element, that samples element will be included in the returned list without any conversions.

Value

An object of the same type as samples after application of conversions.

Description

This is useful for seeing results from multiple MCMC engines compactly.

Usage

combineMetrics(
  results,
  include_times = FALSE,
  params = NULL,
  paramFilter = NULL,
  MCMCs = NULL,
  MCMCFilter = NULL
)

Arguments

Value

A list with elements byParameter, byMCMC and, if include_times=TRUE, times. Each element combines the corresponding elements for each MCMCresult object in the results argument.

See Also

modifyMetrics

Description

Run one or more MCMC engines for one model specification, with timing and performance metrics calculated. See details for special case of precompiled nimble MCMCs.

Usage

compareMCMCs(
  modelInfo = list(),
  MCMCcontrol = list(niter = 10000, thin = 1, burnin = 2000),
  MCMCs = names(nimbleMCMCdefs),
  monitors = character(),
  nimbleMCMCdefs = list(),
  externalMCMCinfo = list(),
  metrics = c("mean", "median", "sd", "CI95_low", "CI95_upp", "efficiency_coda"),
  metricOptions = list(),
  conversions = list(),
  seed = NULL,
  needRmodel,
  verbose = TRUE,
  sessionInfo = TRUE
)

Arguments

Details

The special cases provided for the MCMCs argument include:

• "nimble_noConj": use adaptive random-walk Metropolis-Hastings (ARWMH) samplers in place of Gibbs (conjugate) samplers.

• "nimble_RW": use all adaptive random-walk Metropolis-Hastings samplers.

• "nimble_slice": use all slice samplers.

If you have already used compileNimble to compile both a nimble model and one or more nimble MCMCs, provide the compiled model as modelInfo$model and provide the compiled MCMCs as elements of a named list for nimbleMCMCdefs. In that case, the monitors will already be set in the MCMCs and can't be changed. However, you can still use the monitors argument to subset and/or re-order the monitored nodes (parameters).

See package vignette for more details and examples.

Value

A list of MCMCresult objects.

Description

Create html output with comparisons of MCMC results

Usage

make_MCMC_comparison_pages(
  results,
  dir = tempdir(),
  pageComponents,
  modelName = "model",
  control,
  params = NULL,
  paramFilter = NULL,
  MCMCs = NULL,
  MCMCFilter = NULL,
  plot = TRUE
)

Arguments

Details

See package vignette for information about page components, including about default page components and how to write and register new page components.

To see built-in page components and their options, use as.list(getPageComponents()).

The arguments params, paramFilter, MCMCs, and MCMCFilter are passed to combineMetrics. Both paramFilter and MCMCFilter are passed as expressions. One can call combineMetrics directly (with results as the first argument and any of these four arguments) to see the results tables that will be used to create figures.

Value

A list of objects returned from each page component plugin. For figures, these contain a plottable object such as a ggplot object. For text, these contain information for text output such as an xtable object.

Description

These functions are normally called from compareMCMCs, which passes its arguments or elements extracted from its arguments to these functions.

Usage

MCMCdef_dummy(MCMCinfo, MCMCcontrol, monitorInfo, modelInfo)

MCMCdef_jags(MCMCinfo, MCMCcontrol, monitorInfo, modelInfo)

MCMCdef_stan(MCMCinfo, MCMCcontrol, monitorInfo, modelInfo)

Arguments

Details

These functions are called internally from compareMCMCs. Each one runs an MCMC engine. Functions to interface to other MCMC engines can be registered via registerMCMCengine.

MCMCs in nimble are run from runNIMBLE. This uses a different system because there may be multiple nimble MCMC configurations for one model.

MCMCdef_dummy does not run a real MCMC. It provides a quick way to generate MCMC-formatted output for testing other parts of this package.

MCMCdef_jags runs JAGS via package rjags. It uses model information from modelInfo. It does not use MCMCinfo.

MCMCdef_stan runs Stan via package rstan. It does not use modelInfo. It accepts the following elements of the MCMCinfo list:

• file: file argument to stan_model function in rstan. This can alternatively be provided via stan_model_args$file.

• data: data argument to sampling function in rstan. This can alternatively be provided via sampling_args$data.

• inits: inits argument to sampling function in rstan. This can alternatively be provided via sampling_args$inits.

• stan_model_args: list of arguments to stan_model. Note that this can provide the stan model in the model_code element (as a character string) or in the file element (an alternative way to provide the file name).

• sampling_args: list of arguments to sampling.

The elements file, data, and inits take precendence over corresponding entries in stan_model_args or sampling_args.

If elements warmup, iter, and/or thin are provided in sampling_args, those take precedence over corresponding values in the MCMCcontrol argument to compareMCMCs. Otherwise iter is set to MCMCcontrol$niter and warmup is set to MCMCcontrol$niter/2. Only one chain will be run.

Total sampling time for Stan is recorded via system.call(sampling(...)). This is similar to how time is recorded for other MCMCs. The warmup time (called "burnin" in compareMCMCs for consistency across different MCMCs) is obtained from rstan function get_elapsed_time. The post-burnin time is the total sampling time minus the burnin time.

Description

R6 class to hold MCMC samples, timing results, and metrics

R6 class to hold MCMC samples, timing results, and metrics

Public fields

Methods

Public methods

• MCMCresult$new()

• MCMCresult$setSamples()

• MCMCresult$rename()

• MCMCresult$initializeMetrics()

• MCMCresult$clearMetrics()

• MCMCresult$addMetricResult()

• MCMCresult$clone()

Method new()

Create a new MCMCresult object.

Usage

MCMCresult$new(...)

Arguments

Method setSamples()

Populate the samples and initialize the metrics

Usage

MCMCresult$setSamples(samples)

Arguments

Returns

NULL

Method rename()

Change the MCMC method name from oldName to newName

Usage

MCMCresult$rename(newName, oldName)

Arguments

Details

This change the MCMC field and the corresponding columns of metrics$byParameter and metrics$byMCMC.

If oldName is not the MCMC method name, this function does nothing.

Returns

NULL

Method initializeMetrics()

Initialize metrics if necessary

Usage

MCMCresult$initializeMetrics(silent = FALSE)

Arguments

Details

This function does nothing if metrics are already initialized. It does not clear metrics. See clearMetrics for information on how metrics are initialized.

Returns

logical indicating whether metrics is well-formed or not.

Method clearMetrics()

Clear (reset) byParameter and/or byMCMC metrics

Usage

MCMCresult$clearMetrics(byParameter = TRUE, byMCMC = TRUE)

Arguments

Details

byParameter metrics are initialized to a data.frame with columns for MCMC (all the same entry, the MCMC field) and Parameter (taken from column names of the samples).

byMCMC metrics are initialized to a data.frame with a column for MCMC.

Method addMetricResult()

Add one set of metric results

Usage

MCMCresult$addMetricResult(metricResult)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

MCMCresult$clone(deep = FALSE)

Arguments

See Also

renameMCMC to change the name of an MCMC method throughout the structure of a list of MCMCresult objects.

Description

These functions are normally called via compareMCMCs or addMetric.

Usage

MCMCmetric_mean(result, ...)

MCMCmetric_median(result, ...)

MCMCmetric_sd(result, ...)

MCMCmetric_CI95(result, ...)

MCMCmetric_CI95low(result, ...)

MCMCmetric_CI95upp(result, ...)

MCMCmetric_ESS(result, options = NULL)

MCMCmetric_efficiency(result, options = NULL)

Arguments

Details

A metric is a summary of MCMC output. The summary may include results for each parameter, for each MCMC sample (across all parameters), and/or by arbitrary list. The last option is not used by any built-in metrics.

The built-in metrics include:

• mean : mean for each parameter

• median : median for each parameter

• sd : standard deviation for each parameter

• CI95 : both ends of 95% credible interval, a combination of CI95low and CI95upp

• CI95low : lower end of 95% credible interval

• CI95upp : upper end of 95% credible interval

• ESS : effective sample size (ESS). Control options include ESSfun (a function to estimate ESS, with default = coda::effectiveSize), and suffix (a character string to be appended to "ESS" to form a label, with default = "").

• efficiency or (synonomously) efficiency_coda : effective sample size (ESS) and efficiency (ESS / computation time). If ESS was already calculated, it will not be re-calculated. Control options include ESSfun (passed to ESS), suffix (a character string to be appended to "efficiency" to form a label, with default = ""), and time (a character string to be used as an expression to calculate the computation time from elements of the times element of the result object, with default = "sampling" for burning+postburnin times).

Value

A list that may contain elements named:

• byParameter: A named list of vectors. In each vector, the elements correspond to parameters. The list names will become parameter names in the byParameter element of metrics elements in MCMCresult objects.

• byMCMC: A named list of numbers.

It is also valid to return a list of such lists.

In normal use, metrics are called by addMetrics (possibly from compareMCMCs) and the results are collated in the metrics field of MCMCresult objects.

Description

Clear metrics or add metrics to MCMC results.

Usage

clearMetrics(results, byParameter = TRUE, byMCMC = TRUE)

addMetrics(
  results,
  metrics = c("mean", "median", "sd", "CI95_low", "CI95_upp", "ESS", "efficiency"),
  options = list()
)

Arguments

Details

These functions provide ways to manipulate the collection of metrics inside one or more MCMCresult objects.

The MCMCresult class is fairly simple. One can also modify contents of an MCMCresult object using class methods or direct manipulation of contents.

Metrics are organized as "byParameter", when there is one result for each parameter (column) of MCMC output, and "byMCMC", when there is one result for an entire MCMC sample (across all parameters).

clearMetrics clears all metrics by parameter, by MCMC, or both.

addMetrics populates a set of metrics. See package vignette for more information.

See Also

combineMetrics

Description

Register, unregister and access page components used by make_MCMC_comparison_pages

Usage

registerPageComponents(pageComponents)

unregisterPageComponents(name)

getPageComponents()

Arguments

Details

A page component is an element that can be included in an MCMC comparison page by naming it in the pageComponents argument to make_MCMC_comparison_pages. See package vignette for explanation page components.

See Also

make_MCMC_comparison_pages

Description

Register an MCMC function for use by compareMCMCs

Usage

registerMCMCengine(name, fun)

Arguments

Details

See package vignette for information about the arguments that will be passed to fun from compareMCMCs and the MCMCresult object that should be returned by fun.

For more information, see builtin_MCMCs.

MCMCs from nimble are run in a different way, since there can be multiple MCMCs for the same nimble model. These are run by runNIMBLE, which is not exported.

Description

Register, unregister, or access registered MCMC metric functions for use by compareMCMCs or addMetrics

Usage

registerMetrics(metrics)

unregisterMetric(name)

getMetrics()

Arguments

Details

These functions are called for their "side effects" of modifying the list metric functions for MCMC results that will be recognized by name from the compareMCMCs or addMetrics functions. Those functions take a metrics argument that can be a character vector or a list. Names in the character vector will be looked up from the registered metric functions.

registerMetrics takes a named list and adds its elements to the list of recognized metrics with the corresponding names.

unregisterMetric removes one metric from the list at a time.

getMetrics returns the list of registered metrics.

Value

registerMetrics and getMetrics return the environment of registered metrics.

unregisterMetric returns the result (which should be NULL) of a call to rm that attempts to remove a metric.

Description

This is useful because an MCMC method name appears in multiple places

Usage

renameMCMC(MCMCresult, newName, oldName)

Arguments

Details

This replaces the MCMC label oldName with newName anywhere they appear in the MCMCresult list. This includes various places in the metrics elements of the MCMCresult objects.

If oldName is omitted, MCMCresult must be a single MCMCresult object, in which the existing MCMC method name will be replaced by newName. Hence oldName is only necessary if MCMCresult is a list of MCMCresult objects.