Hyunjoo Kim

Learning Attribute Hierarchies in Cognitive Diagnosis from Data: A Graph-Theoretic Approach

Advisor: Dr. Hans-Friedrich Koehn (UIUC)
Funded by Accessible Teaching, Learning, and Assessment Systems (ATLAS), University of Kansas

• Research Description: Attribute hierarchy models (AHMs) in cognitive diagnosis assume that cognitive skills (“attributes”) are related and hierarchically structured. However, this hierarchical structure is never known a priori and must be estimated, either by content experts or directly from item responses. Extant data-driven estimation methods typically perceive attribute hierarchies as directed networks with edges to be identified using variable selection techniques via penalized EM algorithm or Bayesian approaches using the Metropolis-Hastings algorithm within Gibbs sampling. The current project explores graph-theoretic approaches to reconstruct item hierarchies directly from item responses, which are assumed to point to the hierarchical structure of the underlying attributes.

Graph-theoretical Models for Fitting Multiple Source Proximity Data

Research with Dr. Hans-Friedrich Koehn (UIUC), Dr. Justin L. Kern (UIUC)

• Conducting computational experiments using mathematical programming algorithms to fit the proximity data into a format that satisfies the Additive Tree conditions.
• Planning to apply the methodology to real-world datasets for validation and practical insights.

Identifiability Conditions in Cognitive Diagnosis: Implications for Q-matrix Estimation Algorithms

Research with Dr. Hans-Friedrich Koehn (UIUC), Dr. Chia-Yi Chiu (Teachers College-Columbia University)

• This study aimed to compare Q-matrices estimated by algorithms with and without identifiability conditions. Large-scale simulations examined the impact of factors like the number of examinees, test length, attributes, and error levels. Estimated Q-matrices were evaluated for meeting identifiability conditions and their accuracy in classifying examinees.
• Was presented at the International Meeting of the Psychometric Society 2023
• A paper is under revision

2024

Koehn, H.F., Chiu, C.-Y., Oluwalana, O., Kim, H., and Wang, J. (2024) A Two-Step Q-Matrix Estimation Algorithm
Applied Psychological Measurement https://doi.org/10.1177/01466216241284418

• Ran computational experiments analyzing and comparing the performance of MCMC algorithms, regularized EM-based algorithms, and restricted Boltzmann Machines for estimating the Q-matrices of cognitive diagnostic assessments based on the DINA and G-DINA models, which were part of the evaluation of a nonparametric algorithm for estimating the Q-matrices of cognitive diagnostic assessments
• ABSTRACT

Cognitive Diagnosis Models in educational measurement are restricted latent class models that describe ability in a knowledge domain as a composite of latent skills an examinee may have mastered or failed. Different combinations of skills define distinct latent proficiency classes to which examinees are assigned based on test performance. Items of cognitively diagnostic assessments are characterized by skill profiles specifying which skills are required for a correct item response. The item-skill profiles of a test form its Q-matrix. The validity of cognitive diagnosis depends crucially on the correct specification of the Q-matrix. Typically, Q-matrices are determined by curricular experts. However, expert judgment is fallible. Data-driven estimation methods have been developed with the promise of greater accuracy in identifying the Q-matrix of a test. Yet, many of the extant methods encounter computational feasibility issues either in the form of excessive amounts of CPU times or inadmissible estimates. In this article, a two-step algorithm for estimating the Q-matrix is proposed that can be used with any cognitive diagnosis model. Simulations showed that the new method outperformed extant estimation algorithms and was computationally more efficient. It was also applied to Tatsuoka’s famous fraction-subtraction data. The paper concludes with a discussion of theoretical and practical implications of the findings.

2023

Jin, I.H., Yun, J.H., Kim, H.J., and Jeon, M.J. (2023) Latent Space Accumulator Model for Analyzing Bipartite Networks with Its Connection Time and Its Applications to Item Response Data with Response Time
Stat https://doi.org/10.1002/sta4.632

• Identified the effect of response time on network structure in the competing risk modeling framework
• Preprocessed and analyzed data from the language learning application ’Duolingo’
• Contributed to the drafting of the first version and provided proofreading, editing, and approval for the final draft before submission
• Related-codes link
  
• ABSTRACT

Response time has attracted increased interest in educational and psychological assessment for, e.g., measuring test takers' processing speed, improving the measurement accuracy of ability, and understanding aberrant response behavior. Most models for response time analysis are based on a parametric assumption about the response time distribution. The Cox proportional hazard model has been utilized for response time analysis for the advantages of not requiring a distributional assumption of response time and enabling meaningful interpretations with respect to response processes. In this paper, we present a new version of the proportional hazard model, called a latent space accumulator model, for cognitive assessment data based on accumulators for two competing response outcomes, such as correct vs. incorrect responses. The proposed model extends a previous accumulator model by capturing dependencies between respondents and test items across accumulators in the form of distances in a two-dimensional Euclidean space. A fully Bayesian approach is developed to estimate the proposed model. The utilities of the proposed model are illustrated with two real data examples.

2022

Kim, H.J., Jeon, Y.J., Kim, H.C., Jin, I.H., and Jung, S.J. (2022) Application of latent space item response model to clustering stressful life events Beck and Depression Inventory-II: results from Korean epidemiological survey data
Epidemiology and Health https://doi.org/10.4178/epih.e2022093

• Devised the application model based on the Latent Space Item Response Model and conducted data analysis
• Contributed to the drafting of the first version and provided proofreading, editing, and approval for the final draft before submission
• Related-codes link
  
• ABSTRACT

OBJECTIVES: According to previous findings, stressful life events (SLEs) and their subtypes are associated with depressive symptoms. However, few studies have explored potential models for these events and incidental symptoms of depression.
METHODS: Participants (3,966 men; 5,709 women) were recruited from the Cardiovascular and Metabolic Diseases Etiology Research Center cohort. SLEs were measured using a 47-item Life Experiences Survey (LES) with a standardized protocol. Depressive symptoms were assessed using the Beck Depression Inventory-II (BDI-II). Joint latent space item response models were applied by gender and age group (<50 vs. ≥50 years old).
RESULTS: Among the LES items, death or illness of close relatives, legal problems, sexual difficulties, family relationships, and social relationships shared latent positions with major depressive symptoms regardless of gender or age. We also observed a gender-specific domain: occupational and family-related items.
CONCLUSIONS: By projecting LES and BDI-II data onto the same interaction map for each subgroup, we could specify the associations between specific LES items and depressive symptoms.