التقارير الفنية المتخصصة تحميلA Comparison of IRT Theta Estimates and Delta Scores From the Perspective of Additive Conjoint Measurement Benjamin W. Domingue & Dimiter M. DimitrovThe National Center for Assessment (NCA) is piloting an approach to scoring and equating of tests with binary items referred to as delta-scoring method (Dimitrov, 2015). The purpose of this study was to evaluate the intervalness of the D scores, obtained under the delta-scoring method, in comparison with the intervalness of “theta” scores obtained under the three-parameter logistic model in item response theory (IRT). The question of interest was which scores (D or theta) are more consistent with the axioms of additive conjoint measurement (ACM; Luce & Tukey, 1964). This question was addressed through the approach of ConjointChecks (Domingue, 2014) with the use of real data from a large-scale assessment at the NCA. This study provides evidence that the D scores produce fewer violations of the ordering axioms of ACM than do the theta scores. ... المزيد تحميلUsing a Multiple Indicators Multiple Causes (MIMIC) model to Examine Item and Scale Performance across Differe Ioannis TsaousisTR164-2016 Abstract Item response latency has become a valuable source of information in modern psychometrics, especially after the increasing use of Computer Based Tests (CBT) in educational and psychological assessment. Previous research has shown that response latency is related to several internal (e.g. item difficulty, item discrimination, etc.) and external (e.g. time framework, item length, etc.) characteristics of the tests, but also to temperamental characteristics of the respondents (e.g., fast vs. slow respondents). The aim of this study was to investigate the extent to which speededness (i.e., how fast or slow an individual respond to an item/scale) influences overall performance in three different cognitive areas (verbal, quantitative, and advance functioning). To examine the above research question data from 8,475 individuals completing the computerized version of the Postgraduate General Aptitude Test (PAGAT) were analyzed. To determine the extent to which speededness affects scale scores, a Multiple Indicators Multiple Causes (MIMIC) model was applied, using as a covariate variable the time group membership (i.e., fast vs. slow respondents). The results sugge ... المزيد تحميلThe Effects of Repeated Testing on Person Performance on the Computerized Version of the GAT and PGAT Tests Georgios SideridisTR157-2016 Abstract The purpose of the present studies was to explore how subsequent testing influences examinee performance using the novel methodology of latent transition analysis. Participants in Study 1 were 5,091 examinees who took the computerized version of the GAT test across three time intervals. Four groups of individuals were created and empirically tested using latent class analysis with the predictors being 4 ability testlets (from low to high ability items). The four groups that emerged resembled 4 levels of performance such as those in the interquartile range. Results indicated that individuals were very stable at high ability levels with most movement observed with the above average group moving towards high ability. Interestingly, the most unstable group comprised low achievers who moved towards all possible groups, even the highest ability one. Transitioning from Time 2 to Time 3 essentially replicated the transitioning from Time 1 to Time 2. Study 2 attempted to replicate the above findings with the computerized version of the PGAT, which measures 3 general domains. Results from the latent profile analysis suggested that besides a low and a high ability ... المزيد تحميلThe Effects of Pretesting on Computer Based Testing Using the GAT Georgios SideridisTR152-2016 Abstract The purpose of the present study was to explore how subsequent testing influences examinee performance using the novel methodology of latent transition analysis. Participants were 5,091 examinees who took the CBT test more than once. Four groups were created and empirically tested using latent profile analysis using a subtests comprised of four aggregates reflecting quartile performance levels. Thus, four ability groups were created based on their performance on the quantitative component of the CBT. After testing how individuals transitioned between ability classes results indicated that examinees were mostly stable when they belonged to the high ability group at time 1 with only 4.5% of them moving to the immediately lower ability group. Individuals transitioned mostly from lower ability groups to higher ability groups with the largest numbers of transitioning being with the ability group right above the one examinees were at time 1. Thus, the hypothesis of practice effect is most likely supported. Interesting effects were observed with individuals in the above average group transitioning greatly (51% of them) to the highest ability group. Findings tha ... المزيد تحميلExamining the prevalence and impact of non- attempted items in NCA educational tests Iasonas LamprianouThe aim of this study is to examine whether NCA achievement test scores are affected by response strategy decisions. The dataset consisted of the responses of 34,500 examinees to 52 verbal and 44 quantitative items. It was found that the frequency of missing responses in the data was very small, both for the Verbal and the Quantitative tests. The examinees who produced missing responses on the one test, also tended to produce missing responses on the other test. Coding the missing responses as missing rather than as incorrect did not affect either the model-data fit of the Rasch models, or the difficulty estimates of the items. Also, coding the missing responses as missing rather than as incorrect did not affect noticeably the ability estimates of the overall sample. It was not possible to find evidence that examinees of lower ability tended to produce more missing responses. It is suggested that, although the phenomenon is not important enough to cause concerns regarding the validity and the reliability of the examination results, it should be monitored regularly. ... المزيد تحميلModeling Group Specific Differential Item Functioning in the STAPSOL Test Khurrem JehangirFit to item response theory (IRT) models in educational testing can be compromised by the presence of group-specific differential item functioning (GDIF). The current study proposes methods to detect GDIF and explores the feasibility of improving the fit of the measurement model by using group-specific item parameters to model GDIF. In this approach, it is assumed that a scale consist of both items which are free of GDIF and items with GDIF. The first set of items ensures the validity of the measure across groups. The second set of items is calibrated concurrently with the first set of items and both sets of items contribute to measurement precision. The procedure is used to model high DIF items in the STAPSOL test of Arabic proficiency. Using data of the groups that participated in this exam, concurrent maximum marginal likelihood (MML) estimates of the parameters of the Rasch model (1PLM) are obtained. Then information on observed and expected response frequencies is used to identify GDIF items. Group-specific item parameters are introduced for the items with the largest effect sizes of GDIF and new MML estimates are obtained. The impact of using group-specific item parame ... المزيد تحميلAn Investigation of Dimensionality and Psychometric Properties of the Teacher Test- Mathematics Dimiter M. Dimitrov, Abdulla AlSadaawiThe Teacher Test-Mathematics (TT-M) is administered by the National Center for Assessment in Higher Education as a large-scale assessment tool for teacher certification. This study investigates the dimensionality, reliability, differential item functioning on gender, and other psychometric features of the test in the framework of confirmatory factor analysis (CFA) and item response theory (IRT). Classical statistics of TT-M items are also provided. The data come from the responses of 1,177 teachers on 50 dichotomously scored items (1 = correct, 0 = incorrect) on a base test form of the TT-M (Form 704). The CFA results showed that the TT-M data are essentially unidimensional; that is, there is one dominant dimension (ability, trait) underlying the scores of the examinees on the test. The reliability for the TT-M data, estimated with the use of latent variable modeling, is 0.803, with 95%CI = (0.772, 0.834), which is adequate for the purpose of this study. The IRT results for TT-M data indicated that the overall fit of TT-M items to the three-parameter logistic model is adequate. It was also found that the match between the distributions of the ability of examinees and the diff ... المزيد تحميلFactorial Invariance and Latent Mean Differences of Scores on SAAT across Gender Ioannis TsaousisThis study examined the factorial structure of data on the Standard Achievement Admission Test (SAAT). SAAT has been administered by the National Center for Assessment in Higher Education (NCA) at Riyadh, Saudi Arabia. A total of 63,380 individuals participated in this study. From them, the 36,277 (57,2%) were males and the 27,041 (42,7%) were females. Sixty two participants (0.1%) did not reported their gender. We examined three models: a) a one-factor model, where all indicators loading on one general achievement factor, b) a four-factor model with four latent constructs describing the variability among the SAAT items, and b) a bifactor model, where all indicators modeled to load simultaneously onto one general achievement factor as well as onto their corresponding sub-scales. The results showed that although all models showed good fit, the bifactor model fitted the data better than the alternative models. The results also revealed that the factor loadings and the item intercepts of the SAAT were invariant across gender. Finally, the third aim of the study was to test possible gender differences between different SAAT latent constructs. Using the Latent Mean Difference proc ... المزيد تحميلNEW TEACHER TEST: Factor Structure and Reliability Dimiter M. Dimitrov & & Abdullah AlSadaawiThis study investigates the dimensionality and factor structure of data on the Teacher Test (TT) administered by the National Center for Assessment in Higher Education as an assessment tool for teacher certification. The score reliability of the TT is also estimated. The data consist of the scores of 65,496 examinees on 79 multiple-choice items of which 67 operational test items and 12 external items. The operational items are associated with four test domains, Professional Knowledge (34 items), Promoting Learning (13) items, Supporting Learning (15 items), and Professional Responsibilities (5 items). The results from the comparison of rival confirmatory factor models show that the test data are essentially unidimensional with reliability of .873. The results also suggest that the four test domains do not represent unique aspects of a general factor but, instead, they can be treated as domain-related components of a single factor that underlies the test data. In addition, the results from the item response theory (IRT) calibration of test items indicate that, with a few exceptions, the items fit the three-parameter logistic model in IRT and that there is good match between t ... المزيد تحميلThe Role of the Department of Study and Major on Teacher’s Knowledge and Attitudes Georgioes SideridisThe purpose of the present study was to evaluate differences due to specialty on teacher’s attitudes, knowledge and skills. Participants were 44,692 teachers from various specialties who completed an attitudinal and knowledge self-reported measure. The effects of age, type of major and experience were evaluated using hierarchical linear modeling techniques. Results indicated that there was a noticeable trend for science majors to have more positive attitudes compared to Art/Education majors. Furthermore, the null role of age was augmented by the fact that increases in age were associated with more positive attitudes for science majors only. The findings have implications for the incentives given to various age and gender groups in the education profession as the trajectories of change in attitudes over time are not necessarily positive across disciplines. ... المزيد 1234567 محتوى الصفحة شارك على
