# measr Diagnostic classification models (DCMs) are a class of psychometric models that estimate respondent abilities as a profile of proficiency on a pre-defined set of skills, or attributes. Despite the utility of DCMs for providing fine-grained and actionable feedback with shorter assessments, they have are not widely used in applied settings, in part due to a lack of user-friendly software. Using [R](https://www.r-project.org/) and [Stan](https://mc-stan.org/), measr (said: “measure”) simplifies the process of estimating and evaluating DCMs. Users can specify different DCM subtypes, define prior distributions, and estimate the model using the [rstan](https://mc-stan.org/rstan/) or [cmdstanr](https://mc-stan.org/cmdstanr/) interface to Stan. You can then easily examine model parameters, calculate model fit metrics, compare competing models, and evaluate the reliability of the attributes. ## Installation You can install the released version of measr from [CRAN](https://cran.r-project.org/) with: ``` r install.packages("measr") ``` To install the development version of measr from [GitHub](https://github.com/r-dcm/measr) use: ``` r # install.packages("remotes") remotes::install_github("r-dcm/measr") ``` Because measr is based on Stan, a C++ compiler is required. For Windows, the [Rtools program](https://cran.r-project.org/bin/windows/Rtools/) comes with a C++ compiler. On Mac, it’s recommended that you install Xcode. For additional instructions and help setting up the compilers, see the [RStan installation help page](https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started). ## Usage We can define a DCM using [`dcm_specify()`](https://dcmstan.r-dcm.org/reference/dcm_specify.html). This function requires a Q-matrix defining which attributes are measured by each item. We also identify any item identifier columns. Other arguments can be specified to customize the type of model to estimate (e.g., type of measurement or structural model; see `?dcmstan::dcm_specify()`). We can then estimate our specified DCM using [`dcm_estimate()`](https://measr.r-dcm.org/reference/dcm_estimate.md). We supply our specification and our data set, along with any respondent identifiers. As with [`dcm_specify()`](https://dcmstan.r-dcm.org/reference/dcm_specify.html), other arguments can be specified to customize the model estimation process (e.g., estimation backend and method; see `?dcm_estimate()`). To demonstrate measr’s functionality, example data sets are available in the [dcmdata](https://dcmdata.r-dcm.org) package. Here we use the Examination of Certificate of Proficiency in English (ECPE; [Templin & Hoffman, 2013](https://doi.org/10.1111/emip.12010)) data (see [`?dcmdata::ecpe`](https://dcmdata.r-dcm.org/reference/ecpe.html) for details). Note that by default, measr uses a full Markov chain Monte Carlo (MCMC) estimation with Stan, which can be time and computationally intensive. For a quicker estimation, we could use Stan’s optimizer instead of MCMC by adding `method = "optim"` to the function call. However, please note that some functionality will be lost when using the optimizer (e.g., the calculation of some model fit criteria requires the use of MCMC). ``` r library(measr) model_spec <- dcm_specify(dcmdata::roarpa_qmatrix, identifier = "item") model <- dcm_estimate( dcm_spec = model_spec, data = dcmdata::roarpa_data, identifier = "id", method = "optim", seed = 69385, refresh = 0 ) ``` Once a model has been estimated, model parameters, respondent classifications, and results of the model fit analyses can then be extracted using [`measr_extract()`](https://measr.r-dcm.org/reference/measr_extract.md). ``` r measr_extract(model, "m2") #> # A tibble: 1 × 3 #> m2 df pval #> #> 1 NA 1532 NA measr_extract(model, "classification_reliability") #> # A tibble: 3 × 3 #> attribute accuracy consistency #> #> 1 lsm 0.984 0.969 #> 2 del 0.967 0.939 #> 3 fsm 0.986 0.973 ``` ------------------------------------------------------------------------ Contributions are welcome. To ensure a smooth process, please review the [Contributing Guide](https://measr.r-dcm.org/CONTRIBUTING.html). Please note that the measr project is released with a [Contributor Code of Conduct](https://measr.r-dcm.org/CODE_OF_CONDUCT.html). By contributing to this project, you agree to abide by its terms. # Package index ## Model estimation ### Specification Specify a diagnostic model, including measurement model, structural model, and prior distributions. - [`dcm_specify()`](https://dcmstan.r-dcm.org/reference/dcm_specify.html) : Specify a diagnostic classification model (from dcmstan) - [`get_parameters()`](https://dcmstan.r-dcm.org/reference/get_parameters.html) : Identify parameters included in a diagnostic classification model (from dcmstan) - [`prior()`](https://dcmstan.r-dcm.org/reference/prior.html) [`prior_string()`](https://dcmstan.r-dcm.org/reference/prior.html) : Prior definitions for diagnostic classification models (from dcmstan) - [`default_dcm_priors()`](https://dcmstan.r-dcm.org/reference/default_dcm_priors.html) : Default priors for diagnostic classification models (from dcmstan) ### Estimation Estimate the model using Markov chain Monte Carlo or Stan’s optimizer. - [`dcm_estimate()`](https://measr.r-dcm.org/reference/dcm_estimate.md) : Fit Bayesian diagnostic classification models ## Model evaluation ### Reliability Estimate the pattern- or attribute-level classification accuracy and consistency. - [`reliability()`](https://measr.r-dcm.org/reference/reliability.md) : Estimate the reliability of a diagnostic classification model - [`cdi()`](https://measr.r-dcm.org/reference/cdi.md) : Item, attribute, and test-level discrimination indices ### Model fit Evaluate the fit of the estimated model to the observed data. - [`fit_m2(`*``*`)`](https://measr.r-dcm.org/reference/m2.md) : Estimate the M₂ fit statistic for diagnostic classification models - [`fit_ppmc()`](https://measr.r-dcm.org/reference/fit_ppmc.md) : Posterior predictive model checks for assessing model fit ### Model assumptions Evaluate assumptions of the estimated model - [`qmatrix_validation()`](https://measr.r-dcm.org/reference/qmatrix_validation.md) : Q-matrix validation - [`yens_q3()`](https://measr.r-dcm.org/reference/yens_q3.md) : Yen's Q₃ statistic for local item dependence ### Model comparisons Assess the relative fit of two competing models. - [`aic()`](https://measr.r-dcm.org/reference/aic-bic.md) [`bic()`](https://measr.r-dcm.org/reference/aic-bic.md) : Maximum likelihood based information criteria - [`loo(`*``*`)`](https://measr.r-dcm.org/reference/loo-waic.md) [`waic(`*``*`)`](https://measr.r-dcm.org/reference/loo-waic.md) [`loo_compare(`*``*`)`](https://measr.r-dcm.org/reference/loo-waic.md) : Relative fit for Bayesian models - [`loglik_array()`](https://measr.r-dcm.org/reference/loglik_array.md) : Extract the log-likelihood of an estimated model - [`log_mll()`](https://measr.r-dcm.org/reference/log_mll.md) : Log marginal likelihood calculation - [`bayes_factor()`](https://measr.r-dcm.org/reference/bayes_factor.md) : Bayes factor for model comparisons ### Add evaluations to model objects Add reliability, model fit, and model comparison information to an estimated model object. - [`add_criterion()`](https://measr.r-dcm.org/reference/model_evaluation.md) [`add_reliability()`](https://measr.r-dcm.org/reference/model_evaluation.md) [`add_fit()`](https://measr.r-dcm.org/reference/model_evaluation.md) [`add_respondent_estimates()`](https://measr.r-dcm.org/reference/model_evaluation.md) : Add model evaluation metrics model objects ## Model applications View and use an estimated model. - [`measr_extract()`](https://measr.r-dcm.org/reference/measr_extract.md) : Extract components of a `measrfit` object - [`score()`](https://measr.r-dcm.org/reference/score.md) : Posterior draws of respondent proficiency # Articles ### Get started - [measr: Bayesian psychometric measurement using Stan](https://measr.r-dcm.org/articles/paper.md): - [Estimating diagnostic classification models](https://measr.r-dcm.org/articles/model-estimation.md): - [Evaluating diagnostic classification models](https://measr.r-dcm.org/articles/model-evaluation.md):