多模态学习诊断建模：知识结构和学习风格的双重诊断

Learning behavior consists of activities carried out by students, which reflect complex cognitive processes that are often systematically related to one another. Students with the same knowledge structure may also have differences in other learning behaviors such as learning styles. Such individual differences in learning styles cannot be diagnosed by almost all existing learning diagnosis models (LDMs) that can only analyze item response accuracy (RA) data. Nowadays, with advances in computer-based assessments, it is possible to automatically and simultaneously capture multimodal data during the problem-solving activity, such as outcome data (e.g., response accuracy), process data (e.g., response times (RTs)), and biometric data (e.g., visual fixation counts (FCs)). Utilizing the multimodal data makes it possible to dual diagnosis of knowledge structure and learning style.First, this study taken the fusion analysis of RA, RT, and FC data as an example, elaborated three multimodal data analysis methods and the corresponding models, including the separate modeling (whose model is denoted as S-MLDM), the joint-hierarchical modeling (whose model is denoted as H-MLDM) (Zhan et al., 2021), and the joint-cross-loading modeling (whose model is denoted as C-MLDM). Then, based on the joint-cross-loading modeling, three C-MLDMs with different hypotheses were proposed, including the C-MLDM-, C-MLDM-D, and C-MLDM-C, respectively. Compared with the H-MLDM, three C-MLDMs quantifies the impact of latent ability or latent attributes on RT and FC by introducing two item-level weight parameters (i.e., iand i) into the RT and FC measurement models, respectively. Model parameters were estimated via the fully Bayesian approach with the Markov Chain Monte Carlo method. A multimodal data for a real-world mathematics test was used to illustrate the application of the proposed models and to compare them with the S-MLDM and H-MLDM. Data were collected in an eye-tracking lab setting at a large university on the East Coast of the United States, in which a total ofN= 93 university students with normal or corrected vision were asked to take a test consisting ofI= 10 mathematics items.K= 4 attributes were included in the test, and the corresponding Q-matrix was presented in Figure 3. The data contain three modals of data, namely RA, RT, and FC, which were simultaneously collected. All five multimodal models were fitted to the data. Moreover, a brief simulation study was conducted to supplement the empirical study to explore whether the parameter estimates of the proposed models can converge effectively and to explore the recovery of parameter estimation under unfair test situations (the H-MLDM was used to generate data).The results of the empirical study showed that (1) the C-MLDM- has the best model-data fitting, followed by the H-MLDM and the S-MLDM. Although the DIC shown that the C-MLDM-D and C-MLDM-C also fitted the data well, the results were only for reference because some parameter estimates in these two models did not converge; that (2) the correlation coefficients between latent ability and latent processing speed and that between latent ability and latent concentration were weak, so it was difficult to give full play to the advantages of H-MLDM compared with S-MLRM in theory (Ranger, 2013). By contrast, since the C-MLDM- can directly utilize the information from RT and FC data, the standard error of the estimates of its latent ability was significantly lower than that of the previous two competing models; and that (3) the median of the estimates of iwas less than 0, which indicated that for most items, the higher the participants latent ability is, the longer the time it will take to solve the items; and the median of the estimates of iwas higher than 0, which indicated that for most items, the higher the participants latent ability is, the more number of fixation counts he/she shown in problem-solving. Furthermore, it should be noted that the estimates of iand ido not always keep the same sign for different items, indicating that the influence of latent abilities on RT and FC has different directions (i.e., facilitation or inhibition) for different items. Furthermore, the results of the simulation study indicated that the parameter estimation of the proposed three models can converge effectively and have acceptable recoverability under unfair conditions.Overall, the results of this study indicate that (1) fusion analysis is more suitable for multimodal data that providing parallel information (Jeon et al., 2021) than separate analysis; that (2) through cross-loading, the proposed models can directly use information from RT and FC data to improve the parameter estimation accuracy of latent ability or latent attributes; and that (3) the proposed models can realize the dual diagnosis of knowledge structure and learning style. "