Brief Research Report: Bayesian Versus REML Estimations With Noninformative Priors in Multilevel Single-Case Data

Eunkyeng Baek, S Natasha Beretvas, Wim Van den Noortgate, John M. Ferron

Research output: Contribution to journalArticle

Abstract

Recently, researchers have used multilevel models for estimating intervention effects in single-case experiments that include replications across participants (e.g., multiple baseline designs) or for combining results across multiple single-case studies. Researchers estimating these multilevel models have primarily relied on restricted maximum likelihood (REML) techniques, but Bayesian approaches have also been suggested. The purpose of this Monte Carlo simulation study was to examine the impact of estimation method (REML versus Bayesian with noninformative priors) on the estimation of treatment effects (relative bias, root mean square error) and on the inferences about those effects (interval coverage) for autocorrelated multiple-baseline data. Simulated conditions varied with regard to the number of participants, series length, and distribution of the variance within and across participants. REML and Bayesian estimation led to estimates of the fixed effects that showed little to no bias but that differentially impacted the inferences about the fixed effects and the estimates of the variances. Implications for applied researchers and methodologists are discussed.

Original languageEnglish (US)
JournalJournal of Experimental Education
DOIs
StatePublished - Jan 1 2019

Fingerprint

Research Personnel
Bayes Theorem
trend
coverage
simulation
experiment

Keywords

  • Bayesian estimation
  • estimation method
  • multilevel modeling
  • noninformative priors
  • restricted maximum likelihood
  • single-case research

ASJC Scopus subject areas

  • Education
  • Developmental and Educational Psychology

Cite this

Brief Research Report : Bayesian Versus REML Estimations With Noninformative Priors in Multilevel Single-Case Data. / Baek, Eunkyeng; Beretvas, S Natasha; Van den Noortgate, Wim; Ferron, John M.

In: Journal of Experimental Education, 01.01.2019.

Research output: Contribution to journalArticle

@article{c8d0ac299f264d8db35ef3ef58d95eae,
title = "Brief Research Report: Bayesian Versus REML Estimations With Noninformative Priors in Multilevel Single-Case Data",
abstract = "Recently, researchers have used multilevel models for estimating intervention effects in single-case experiments that include replications across participants (e.g., multiple baseline designs) or for combining results across multiple single-case studies. Researchers estimating these multilevel models have primarily relied on restricted maximum likelihood (REML) techniques, but Bayesian approaches have also been suggested. The purpose of this Monte Carlo simulation study was to examine the impact of estimation method (REML versus Bayesian with noninformative priors) on the estimation of treatment effects (relative bias, root mean square error) and on the inferences about those effects (interval coverage) for autocorrelated multiple-baseline data. Simulated conditions varied with regard to the number of participants, series length, and distribution of the variance within and across participants. REML and Bayesian estimation led to estimates of the fixed effects that showed little to no bias but that differentially impacted the inferences about the fixed effects and the estimates of the variances. Implications for applied researchers and methodologists are discussed.",
keywords = "Bayesian estimation, estimation method, multilevel modeling, noninformative priors, restricted maximum likelihood, single-case research",
author = "Eunkyeng Baek and Beretvas, {S Natasha} and {Van den Noortgate}, Wim and Ferron, {John M.}",
year = "2019",
month = "1",
day = "1",
doi = "10.1080/00220973.2018.1527280",
language = "English (US)",
journal = "Journal of Experimental Education",
issn = "0022-0973",
publisher = "Routledge",

}

TY - JOUR

T1 - Brief Research Report

T2 - Bayesian Versus REML Estimations With Noninformative Priors in Multilevel Single-Case Data

AU - Baek, Eunkyeng

AU - Beretvas, S Natasha

AU - Van den Noortgate, Wim

AU - Ferron, John M.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Recently, researchers have used multilevel models for estimating intervention effects in single-case experiments that include replications across participants (e.g., multiple baseline designs) or for combining results across multiple single-case studies. Researchers estimating these multilevel models have primarily relied on restricted maximum likelihood (REML) techniques, but Bayesian approaches have also been suggested. The purpose of this Monte Carlo simulation study was to examine the impact of estimation method (REML versus Bayesian with noninformative priors) on the estimation of treatment effects (relative bias, root mean square error) and on the inferences about those effects (interval coverage) for autocorrelated multiple-baseline data. Simulated conditions varied with regard to the number of participants, series length, and distribution of the variance within and across participants. REML and Bayesian estimation led to estimates of the fixed effects that showed little to no bias but that differentially impacted the inferences about the fixed effects and the estimates of the variances. Implications for applied researchers and methodologists are discussed.

AB - Recently, researchers have used multilevel models for estimating intervention effects in single-case experiments that include replications across participants (e.g., multiple baseline designs) or for combining results across multiple single-case studies. Researchers estimating these multilevel models have primarily relied on restricted maximum likelihood (REML) techniques, but Bayesian approaches have also been suggested. The purpose of this Monte Carlo simulation study was to examine the impact of estimation method (REML versus Bayesian with noninformative priors) on the estimation of treatment effects (relative bias, root mean square error) and on the inferences about those effects (interval coverage) for autocorrelated multiple-baseline data. Simulated conditions varied with regard to the number of participants, series length, and distribution of the variance within and across participants. REML and Bayesian estimation led to estimates of the fixed effects that showed little to no bias but that differentially impacted the inferences about the fixed effects and the estimates of the variances. Implications for applied researchers and methodologists are discussed.

KW - Bayesian estimation

KW - estimation method

KW - multilevel modeling

KW - noninformative priors

KW - restricted maximum likelihood

KW - single-case research

UR - http://www.scopus.com/inward/record.url?scp=85062478942&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062478942&partnerID=8YFLogxK

U2 - 10.1080/00220973.2018.1527280

DO - 10.1080/00220973.2018.1527280

M3 - Article

AN - SCOPUS:85062478942

JO - Journal of Experimental Education

JF - Journal of Experimental Education

SN - 0022-0973

ER -