Development of a Scale to Assess High Schoolers ’ Attitudes toward Sustainable Development

The study aims to develop a scale to assess high school students’ attitudes toward sustainable development. For this purpose, 90 Likert-type scale items pertaining to attitudes were initially developed based on existing literature. Further consultations with experts led to a reduction of the item count to 60, followed by a further reduction to 40 items. The scale was administered to 509 students enrolled in an Anatolian high school, a science high school, and an Anatolian religious vocational high school in Akçaabat. The data gathered were analyzed using the SPSS 18.0 statistics package. An item analysis, comprising the item-total correlation test, was performed on the scale, which was followed by a reliability analysis that focused on the Cronbach’s alpha internal consistency factor. Furthermore, factor analysis was performed to determine the validity of the scale. The factor analysis led to the development of a two-factor scale composed of a total of 14 items. The Cronbach’s alpha internal consistency factor of the scale is 0.811. The scale was deemed appropriate for assessing the attitudes of high school students with regards to sustainable development.


Introduction
The concept of sustainable development, which was discussed for the first time in the Brundtland Commission's report in 1987, refers to the process of meeting a generation's needs without compromising the needs of future generations (Redclift, 2005).Essentially, sustainable development, a philosophy regarding the management of global development, aims to maintain the integrity of global ecosystems while supporting economic growth and social welfare (Petrovic, Snider, Marko, Cirovic, & Milenkovic, 2012).However, it is not easy to understand sustainable development, since it involves a variety of dimensions (Likon, Asunta, Rihtarsic, & Korze, 2011).Often, sustainable development is discussed with reference to three domains: environmental, social, and economic (Brunold, 2006).From an environmental perspective, sustainable development refers to the minimization of human beings' carbon footprint over the global ecosystem, thereby ensuring that the world can maintain a healthy existence (Öztürk, 2007).From an economic perspective, sustainable development refers to the facilitation of production/consumption activities that make good use of the limited resources available in the environment without damaging it (Türer, 2010).From a social perspective, sustainable development is closely related to social equality and fairness (Dempsey, Bramley, Power, & Brown, 2009).Sustainable development can be considered an irreversible process that encompasses development and has far-reaching physical and biological repercussions worldwide (Lange, 2012).Lange (2012) reports that sustainable development, as an idea, is on the agenda of various societies and is building a strong reputation in scientific and academic circles, albeit gradually.Today, sustainable development, which is a leading global issue, is effectively a universal concept with consequences and applications in all countries (Brunold, 2006).The conservation of the planet's resources is crucial for realizing a sustainable future.In developing countries such as Turkey, population growth, deforestation, and unrestrained urbanization are only some of the problems leading to extensive environmental damage, rapid depletion of natural resources, and accumulation of solid waste.It is essential to review all human activities from the perspective of improving quality of life while enabling at least an equally high quality of life for future generations (Tekkaya, Kılıç, & Şahin, 2011).However, human beings have been inflicting irreparable damage on nature and the environment.Unless a transformational change occurs in the growth rate of the human population, the consumption of natural resources to the point of wastage, and lifestyles based on a consumption economy, the current developments in science and technology will not be able to overcome the large-scale issues of hunger and poverty that affect major parts of the world (Keleş, 2007).The rapidly growing population, resulting in increasing volumes of diverse needs, has led to an excessive level of consumption.This, in turn, has led to the disruption of the natural equilibrium, as well as the depletion and damage of natural resources.To ensure their availability for future generations, it is evident that we need to conserve natural resources from the perspective of sustainability.In this context, it is crucial that we raise future generations with an awareness of how to conserve and wisely use natural resources.Therefore, the coming generations should be equipped with knowledge about these resources and their sustainability (Tamkan, 2008).
On December 20, 2002, the 57th Session of the United Nations General Assembly declared that the Decade of Sustainable Development Education (Mulà & Tilbury, 2009) would start in 2005 (Holbrook, 2009).The primary reason for this initiative was the implementation of effective education regarding sustainable development (Contini & Pascal, 2010).This declaration was expected to raise awareness about sustainable development.To achieve a sustainable future, the awareness levels of individuals should be improved by offering them learning opportunities that are based on a sensitive perspective that enables the building of certain values, attitudes, and new lifestyles (Demirbaş, 2011).
The goals of sustainable development can only be achieved by instilling in people the attitudes and values required to leave a natural heritage to future generations; this is necessary in all stages of education, from primary to higher education (Çolak, 2012).The development of individuals' attitudes regarding sustainability will influence future changes in their behavior pertaining to sustainability.Hence, as part of sustainable development education, it is important to understand the attitudes of individuals (Kaya, 2013).By instilling sustainability attitudes in students, we can ensure the growth of individuals who will leave a habitable world behind for future generations.In order to achieve the targets of sustainable development education, it is necessary to prepare students for a sustainable life.The first step in this direction is the development of positive attitudes toward sustainable development among students.In order to effectively impart sustainable development education, it is first important to identify students' existing attitudes and awareness regarding sustainable development and sustainability.In this context, the development of an assessment tool to help determine the students' attitudes regarding sustainability is crucial.
Against the above background, this study intends to develop a scale to assess high school students' attitudes toward sustainable development and sustainability.Based on related studies, it is clear that the development of a valid and reliable scale would arguably contribute significantly to research and literature on sustainable development education at the national and international levels.

Methodology
Aiming to assess students' attitudes toward sustainable development and sustainability, this study adopted a qualitative research perspective (Creswell, 2013) and used a survey method to collect relevant data (Babbie, 1990;1997).Further, the researchers developed a five-level Likert-type Sustainable Development Attitudes Scale to determine high school students' attitudes toward sustainable development and sustainability.The draft scale, which was developed to apply reliability and validity analyses and culminated in the final version used in the study, was administered to 509 students enrolled in the ninth year of an Anatolian high school, a science high school, and an Anatolian religious vocational high school in the province of Trabzon in Turkey.The development of the scale entailed four stages: the formulation of attitude items, consultation with experts, initial administration, and the determination of structural validity and assessment of reliability.
The first stage began with a literature survey, with reference to the scale to be developed, to determine high school students' attitudes toward sustainable development and sustainability (Yavuz, 2006;Keleş, 2007;Bilgili, 2008;Çimen, 2008;İbiş, 2009;Benzer, 2010;Engin, 2010;Öztürk, 2010;Ünal, 2010;Afacan & Demirci, 2011;Güven, 2011;Elgin, 2012;Ertekin, 2012;Soysal, 2012).The items that reflected the participants' attitudes were noted, resulting in a pool of 90 items.Half of the items on attitudes were formulated in a positive manner, whereas the other half was formulated in a negative manner.Subsequently, the scale was coupled with a five-level Likert-type arrangement, whose ratings were "I definitely agree," "I agree," "I am undecided," "I don't agree," and "I definitely don't agree." In the second stage, the items included in the scale were submitted to a content validity review by experts to determine whether they were adequate for performing a qualitative and quantitative assessment of the attitudes that were intended to be assessed (Büyüköztürk, 2011).Three experts were consulted to comment on the suitability of the developed attitudes scale for sustainability education.The experts reviewed the items in the pool with reference to their ability to provide an accurate assessment of the students' attitudes toward sustainable development and sustainability.According to the experts' views and recommendations, some items that were difficult to understand or identical to other items were removed from the scale.The result of this process was a 40-item draft sustainable development education attitudes scale.
The third stage involved the finalization of the draft scale after necessary corrections were applied in accordance with the experts' feedback regarding the structure, content, and wording of the items; the existence of potentially identical questions; uncertainties in meaning; the use of words with uncertain meaning; the use of leading questions and statements; the level of difficulty of questions; overlapping items; incomplete wording; incomplete questions; the necessity of each question or item; discriminatory questions; rates of questions left unanswered; and the comprehensibility of instructions.After consulting with the experts, the researchers made the decision to apply a five-level Likert-type scale (5: I definitely agree, 4: I agree, 3: I am undecided, 2: I don't agree, and 1: I definitely don't agree) to develop the structure of the draft scale such that it accurately reflects the students' views.However, to avoid potential perception and interpretation issues in the data-gathering process of this study, prevent undue difficulties for the participants, determine the estimated time frame for the application, and observe potential shortcomings and errors, the scale was administered at an Anatolian high school in Trabzon on a pilot basis.Usually, when responding to a survey or considering the items on a scale, participants undergo various mental and cognitive processes, such as understanding the question, recalling relevant information from memory, evaluating the information through comparisons, and coming up with the answer (Bolton, 1993).However, in case they face any cognitive difficulties during the process of answering the question, the participants may submit answers that are marred by certain amounts of error (Converse & Presser, 1986).This is why pilot applications in scientific studies substantially help in the elimination of errors pertaining to responses and other issues outside of the sample (Assael & Keon, 1982).The pilot application was administered to 40 students and allowed for the correction of some ambiguities in several items.Further, it was observed that 15-20 minutes would suffice for answering the scale.
In order to gather data relevant to this study, the draft scale that was developed following the pilot application stage was administered to a total of 509 students from science, Anatolian, and religious vocational high schools.Prior to the administration of the scale, the researchers provided the students with some information on the aims of the study and asked them to respond in a manner that reflected their attitudes toward sustainable development and sustainability as accurately as possible.Similar to the pilot application, it took 15-20 minutes, on average, to respond to the scale, for all classes.
The fourth and final stage of the study was data analysis.The data gathered from the respondents of the study were first saved using Microsoft Excel and then analyzed using the SPSS 18.0 package.Subsequently, the students' responses to positive statements were scored on a scale from 5 to 1 (5: I definitely agree, 4: I agree, 3: I am undecided, 2: I don't agree, and 1: I definitely don't agree).Their responses to negative statements were scored on a scale of 1 to 5 (1: I definitely agree, 2: I agree, 3: I am undecided, 4: I don't agree, and 5: I definitely don't agree).Finally, the data were exported to the SPSS 18 package software.
In order to analyze the structural validity of the scale, exploratory factor analysis, which is a process for determining the influencing factor with reference to the relationships between variables, was performed (Büyüköztürk, 2011).Essentially, exploratory factor analysis is an analysis technique that reveals and groups items assessing a specific structure or quality among a larger set of items and explains the assessment using this smaller number of meaningful superstructures (factors) (Büyüköztürk, 2011).In this context, the Kaiser-Meyer-Olkin (KMO) value and Bartlett's test value were first calculated to check whether the scale was suitable for factor analysis.
Factor analysis involves the development of general variables, called factors, by compiling a set of variables that express a high level of correlation among themselves.The purpose of this analysis is to reduce the number of variables and categorize them (Kalaycı, 2010).Three techniques are used to assess the suitability of data for factor analysis: the development of a correlation matrix, Bartlett's test, and the KMO test (Kalaycı, 2010).The first such technique requires that the correlation factor between the variables is reviewed.The higher the correlation, the more likely the variables have a common factor.Bartlett's test is applied to check whether the data in the data set exhibit a multivariable normal distribution.Bartlett's test can identify strong correlations among variables.The KMO value should be more than 0.50, since any value lesser than 0.50 implies unsuitability for further factor analysis (Çokluk, Şekercioğlu, & Büyüköztürk, 2012).The higher the value, the more viable the data set for factor analysis (Kalaycı, 2010).
The common factor variance values of the items in the scale were checked to determine whether they had a factor loading value lower than 0.30 or had been associated with more than one factor, with a factor loading value variance of 0.10 or lower.A minimum factor loading value of 0.30 is widely considered acceptable.However, many theorists claim that a minimum factor loading value of 0.40 is necessary (Çokluk et al., 2012).In addition, this study employed principal components analysis and the varimax rotation technique to establish interpretable factors.These techniques were used because factor rotation efficiently generates denominational and interpretable factors (Kalaycı, 2010), and varimax is the most commonly used vertical rotation technique.
The reliability of the scale was evaluated using the Cronbach's alpha reliability factor, which reflects the internal consistency of the test scores in the scale; a minimum value of 0.70 is required to consider the test reliable (Büyüköztürk, 2011).
Further, principal components analysis was applied as a factorization technique.Moreover, in order to determine the items expressing a high level of association with a given factor and facilitate the interpretation of the factors, a vertical rotation technique called varimax rotation was employed (Büyüköztürk, 2011).
In order to select the factor count applicable to the scale, concepts such as the eigenvalue, variance rate, and line graph were considered.Some scholars note that eigenvalues of 1 or higher should be considered.It is noted that the higher the variance rate, the stronger the factor structure, and that the value should be 30% or higher for single-factor scales in social studies and more than 30% for multi-factor scales (Büyüköztürk, 2011).The decision regarding the number of significant factors or structures assessed by the scale was made based on a line graph that was drawn in accordance with the factor eigenvalues.Scree plots reveal dominant factors and thereby help in limiting the number of factors, which is the primary purpose of factor analysis.In the scree plot shown in Figure 1, the eigenvalues are plotted along the y axis, whereas the components are plotted along the x axis (Çokluk et al., 2012).The factor showing sudden accelerated falls on the graph reflects the number of factors that must be considered (Büyüköztürk, 2011).
Reliability can be defined as the consistency of the participants' responses to various items on the test (Büyüköztürk, 2011).The use of Cronbach's alpha (α) factor is acceptable in determining the reliability of a Likert-type attitudes scale (Tavşancıl, 2010).According to Kalaycı (2010), the scale is considered extremely reliable if the alpha factor lies in the 0.60-0.80range.A higher α factor of the scale implies that the scale items are more consistent and assess the same characteristic (Tavşancıl, 2010).
To this end, the relationship between the scores received for individual items and the overall score for the scale (item-overall score correlation) was examined in order to determine the ability of the scale items to assess, in terms of comparable behavior, the students' attitudes toward sustainable development.The item-overall score correlation, which describes the relationship between the scores for individual test items and the overall score received in the test, should be positive and preferably more than 0.25.Such a correlation level denotes a high level of internal consistency.It is recommended to remove the items that do not meet this requirement from the scale (Büyüköztürk, 2011).In this context, an examination of the adjusted item-overall score correlation for the Likert-type scale was followed by the removal of items having a lower correlation rate.
Another method employed in the context of reliability analyses is the comparison of the top 27% and bottom 27% segments.The t-test results of 27% of the segments at the top (N: 137) and bottom (N: 137) of the scale were checked to test for the existence of a significant variation between the groups.

Results and Discussion
This section presents the findings of the validity and reliability analyses discussed in Section 2. First, the data structure was checked to understand whether it was suitable for factor analysis.The KMO and Bartlett's tests were used for this purpose.The KMO factor used to evaluate the suitability of data for factor analysis can be reviewed using Bartlett's test of sphericity (Büyüköztürk, 2011).A significant Bartlett's factor indicates a normal distribution of data (Tavşancıl, 2010).From Table 1, it can be seen that the KMO test score was 0.845, and Bartlett's test score was 3826.460(p<0.05).These two values indicate that the data set is suitable for factor analysis.In addition, a significant Bartlett's factor suggests that the distribution of data is normal.A KMO value more than 60, combined with a significant result in Bartlett's sphericity test, reveals that the data are suitable for factor analysis (Büyüköztürk, 2011).Based on the applied factor analysis, items with a factor loading value lower than 0.40 were removed from the scale, along with the items that were considered to overlap since they had a factor loading value margin lower than 0.10 with respect to more than one factor.Varimax rotation was applied to facilitate the association of the factors with their related items and make their interpretation easier.Vertical rotation was applied during factor analysis, and the varimax method that is applicable for more than two factors was used to establish factor loading.
Second, various measures, such as eigenvalue and line graph techniques, were applied to establish the factor count.Based on Figure 1, which shows the line graph drawn for scale items, the items on the scale can be classified into two factors.A review of the Rotated Components Matrix reveals that items 38,37,33,22,36,4,34,18, and 27 are grouped under factor 1. Items 32, 26, 24, 28, 30, 19, and 31 are grouped under factor 2 (Table 2).
Figure 1.Line graph showing the factor count Factor 1 comprises nine items, exhibiting 27.708% of the overall variance.An examination of Table 3 reveals that factor loading figures in the range 0.675-0.568are observed for factor 1. For this factor, the students' negative views on sustainable development are prominent.Factor 2 comprises seven items, exhibiting 10.353% of the overall variance.For this factor, factor loading values range from 0.626 to 0.473.Further, factor 2 depicts the positive attitudes of students toward sustainable development.Therefore, the scale effectively comprises 16 items.
The Cronbach's alpha factor was calculated to establish the reliability of the scale, which was found to comprise 16 items.In this context, the Cronbach's alpha for the negative attitudes factor (F1), consisting of nine items of the scale, was calculated to be 0.809, whereas the corresponding value for the positive attitudes factor (F2), consisting of seven items, was found to be 0.603.Consequently, items 24 and 28 were removed from the scale since they violated reliability.The Cronbach's alpha derived from the overall scale of 14 items was found to be 0.811.According to Kalaycı (2010), a scale is considered extremely reliable if its alpha factor is in the 0.60-0.80range.In this sense, the scale developed by this study can be considered very reliable, since its alpha factor is within the specified range.The differences between the average scores for each item, as per the responses of the students in the top and bottom 27% segments, were reviewed to check whether the scale can identify the corresponding students.The results of the analysis are presented in Table 3. Note.n=509; n 1 =n 2 =137; p< 0.001.
From Table 3, it is clear that the variance between the average scores for the students in the top and bottom 27% segments was statistically significant (p<0.05).The overall item correlation for the items in the scale ranged from 0.27 to 0.59.It is noted that items with an overall item correlation of 0.30 or higher better differentiated among individuals and the items in the scale are highly reliable and focused on assessing the same set of attitudes.Hence, the 14 items in the scale were found to be capable of identifying the students in the top and bottom segments.In other words, each item in the scale was found to sufficiently differentiate among individuals.In conclusion, based on the evidence available for the reliability of the scale, one can argue that the sustainability attitudes scale is a reliable tool for assessing the attitudes of secondary education students regarding sustainability.
In addition, the existence of any relationship between the students' scores in the sustainable development education attitudes scale and the factors of the scale was investigated.Although the scores received by the students in the sustainable development education attitudes scale were highly and positively correlated with factor 1, their correlation with factor 2 was moderate, but positive (Table 4).Further, Table 4 reveals a positive but weak correlation between factors 1 and 2. This weak correlation is expected since factors 1 and 2 are different and assess different characteristics.Note.p<.001.
From Table 4, it is clear that a positive and significant relationship exists between the elements of the sustainable development attitudes scale and the scale in general.It can be argued that the results serve as the proof of the structural validity of the scale.
Some differences were identified by comparing the structure of this scale with the structures of similar scales in the literature.For instance, Kaya (2013) noted that the sustainable development attitudes scale developed for secondary education students was based on three factors (environmental, economic, and social) and had a Cronbach's alpha value of 0.929.For a scale developed to assess prospective teachers' attitudes toward education on sustainable environment, Afacan and Demirci (2011) identified six factors: awareness about sustainable environment, negative ideas about sustainable environment education, willingness to impart sustainable environment education, prudence in sustainable environment education, sensitivity in sustainable environment education, and efficient consumption in sustainable environment education.The internal consistency factor of the scale was found to be 0.904.In this study, however, the "negative attitudes toward sustainability" and "positive attitudes toward sustainability" were the two factors on which the scale was based, and a Cronbach's alpha internal consistency factor of 0.811 was obtained for the complete scale.Significant and positive correlations were found between the factors, as well as between the overall scale and individual factors.In this context, the structural validity of the scale is proved.

Conclusions
This study developed an assessment tool to determine high school students' attitudes toward sustainable development.The final scale comprises 14 items.In this context, 40 Likert-type items having five levels investigating the students' attitudes toward sustainability were developed.Following factor analysis, the item count was reduced to 16.The line graph review identified two factors to collate the items.Factors 1 and 2 depicted negative and positive attitudes, respectively.The reliability of factor 1 was found to be 0.809, whereas that of factor 2 was 0.603.The overall reliability of all factors was 0.811.Items 24 and 28 were removed from the scale, since they lowered the reliability.Factor 1 combines nine items (38, 37, 33, 22, 36, 4, 34, 18, and 27), and factor 2 comprises five factors (32, 26, 30, 19, and 31).
Early on in the study, the pool of items developed through the review of the existing studies and scales was reduced to a limited set of 40 items after the expert review removed the items that were semantically or structurally similar, incomprehensible or grammatically incorrect, or unrelated to the issue at hand.These 40 items were subjected to factor analysis to determine the validity of the scale.The factor analysis led to two factors that expressed 40.683% of the overall variance of the scale.In the final version of the scale, nine items were gathered under factor 1, and five were gathered under factor 2. A diligent analysis was applied for the development of the scale, and numerous statistical tests were employed in determining the final form.This showed that valid and reliable scales suitable for gathering high-quality data could be developed by diligent analyses and the application of correct statistical tests, which would result in marked improvements in data quality.This, in turn, enables more accurate and reliable studies.Hence, one can enthusiastically recommend the employment of the stages and statistical tests applied in this study to any future study for developing scales.
In conclusion, one can argue that the sustainable development attitudes scale developed herein is structurally valid and reliable and that it can be comfortably used in determining the attitudes toward sustainable development and sustainability of students from various demographics in other countries and from various types of high schools that are comparable to the classification of high schools in Turkey, based on science high schools, Anatolian high schools, and vocational high schools.

Table 1 .
KMO and Bartlett's sphericity test results

Table 2 .
Factor loading values for the attitudes scale

Table 3 .
Item analysis results

Table 4 .
Correlations between sustainable development attitudes scores and factors