An Investigation of Reliability Coefficients Estimated for Decision Studies in Generalizability Theory

Omer Kamis, C. Deha Dogan

Abstract


This research aimed to compare the G and Phi coefficients estimated in Decision studies in Generalizability theory and obtained in actual cases for the same conditions of similar facets by using crossed design. The research was conducted as pure research on 120 individuals (students), six items and 12 raters. An achievement test composed of six open ended questions and a holistic rubric developed by the researcher were used in data collection. Data analysis included obtaining the G and Phi coefficients by creating actual cases for two, four and six raters followed by D studies conducted for other actual cases with different measurement conditions to make estimates. Finally, G and Phi coefficients obtained and estimated for two, four and six raters were compared separately. Findings show that G and Phi coefficients estimated in D studies by increasing the number of raters were sometimes greater than those obtained in actual cases although they were sometimes smaller. However, it was concluded that a pattern may not always exist between the G and Phi coefficients obtained in actual cases for same number of raters and those estimated in D studies.


Full Text:

PDF


DOI: https://doi.org/10.5539/jel.v7n4p103

Copyright (c) 2018 Ömer Kamis, deha dogan

License URL: http://creativecommons.org/licenses/by/4.0

Journal of Education and Learning   ISSN 1927-5250 (Print)   ISSN 1927-5269 (Online)

Email: jel@ccsenet.org

Copyright © Canadian Center of Science and Education

To make sure that you can receive messages from us, please add the 'ccsenet.org' domain to your e-mail 'safe list'. If you do not receive e-mail in your 'inbox', check your 'bulk mail' or 'junk mail' folders.