Development and validation of a questionnaire on EFL teachers ’ attitudes towards classroom observation

2014 , 18  March 624 | P a g e  Development and validation of a questionnaire on EFL teachers’ attitudes towards classroom observation Hossein Khodabakhshzade, Azar Hosseini Fatemi PhD candidate, English department, Ferdowsi University of Mashhad hkhodabakhshzade@gmail.com Associate Professor, English department, Ferdowsi University of Mashhad azar.h.fatemi@gmail.com ABSTRACT This study was conducted to construct and validate an EFL teachers‟ attitudes towards classroom observation scale in the context of Iran. For this purpose, one hundred EFL teachers, who were teaching English in Jahan Elm, Institute of Higher Education and Hafez English Language Institute, Mashhad, Iran, were asked to participate in the study and fill in the questionnaire. First, the internal consistency of the instrument as measured by Cronbach‟s coefficient alpha reliability reached acceptable alpha(s). Then, Confirmatory Factor Analyses (CFAs) were conducted using LISREL 8.80. The results of the CFA as assessed by the CFI/NNFI, RMSEA and chi-x2, reached acceptable fits. The factor structures were defined and the factors were clearly distinguishable from one another. Based on the findings, the researchers concluded that the present scale has undoubtedly strong psychometric characteristics and good construct validity which is useful for assessing EFL teachers‟ views on classroom observation in Iran.


1.INTRODUCTION
puts forward the idea that, "no one can become a good supervisor, a good trainer, or a good teacher overnight.It is a gradual process, but can be speeded up with a concerted effort.The supervisor can help teachers in their professional development through ongoing guidance.By ongoing guidance a process in the course of which the supervisor guides his teachers step by step until they acquire the educational skills of a good teacher is meant".Although observation is receiving a wide spread interest in all educational settings around the world, and is implemented in numerous institutions regarding various fields of education, it is still looked upon as an area of foreign language teacher education which is replete with critical issues related to successful teaching especially for student teachers and teacher trainers.Classroom observation is one of these fields through which a systematic reflection on classroom experience can be resulted in professional development (Swan, 1993;Wallace, 1991).Although the benefits of classroom observation cannot be ignored (Barocsi, 2007) considering teachers" views in this regard can effectively help to take maximum advantage out of it.The main concern of how to make classroom observation more beneficial for teachers has shed light on EFL teachers" role as a policy maker in developing a procedure for observation.On the other hand, traditional forms of observations in fact, have led into a sense of hatred and indifference on the part of EFL teachers.In light of these considerations, this study draws on providing supervisors, & the educational systems with a validated questionnaire with the help of which they can gain a deeper insight of what are the best ways of helping teachers with the development of their educational skills, using their own opinions, along with finding some solutions of how to change teachers" negative attitudes toward classroom observation.

REVIEW OF LITERATURE
As the present study is twofold, that is both focusing on classroom observation along with the development & validation of a questionnaire, in the following literature, not just classroom observation in various contexts is searched, but the researcher has also reviewed some articles to pinpoint the practical ways of validating questionnaires.

Classroom Observation
With the help of the researches done, it has been repeatedly proved that classroom observations are really beneficial, resulting in higher proficiency scores (Douglas, Harris and Sass, 2007).That is what Ashton & Webb (1986) have explained that teachers have the greatest impact on student achievement.On the other hand, findings of some studies (Jacob & Lefgren, 2004) show no relation between teachers' participation in professional development activity and student achievement, while other findings (Brown, Smith, & Stein, 1995) show higher levels of students' achievements related to teachers" professional development directly related to the area in which they are teaching.Also, Darling-Hammond (2000) claims that teachers with higher preparation for teaching are more confident and therefore have more successful students than teachers who have little preparation or none (As cited in Garcia, 2011).Sadia Ali (2007) has mentioned six steps of a reflective teacher observation program in his article as: Analyze trainees" observation needs, Prepare trainees for observation procedures, Trainees observe trainers teaching, Observers contact trainees" school to build rapport, Observers observe trainees and trainees reflect on teaching, & Summative assessment.Also, Williams (1989, 86-87) has stated seven principles of teacher observation as pertinent to a reflective TOP.These principles are: Development, Limited and focused content, Course-link, Teacher centeredness, Future development, Positiveness, & Flexibility (As cited in Sadia Ali, 2007).
For classroom observation is considered as a kind of discouraging experience, & even at times leading to resentment, Wang & Seth (1998) carried a research at Qingdao University, aiming to turn classroom observation into an effective means for teacher development by employing a more collaborative approach, which had positive and encouraging results.Therefore, the research method was drawn from the self-development approach developed in the Masters dissertation of one of the Chinese counterparts on the project.The overall aim of using this approach was to help teachers adopt a more developmental attitude towards classroom observation by providing opportunities for self-development through selfassessment.As a result, the usual power relationship patterns between observers and teachers was dramatically changed by giving the teachers the opportunity to decide what they would like to do throughout the classroom observation process.This meant, for instance, that they were given the chance of choosing who they would like to invite to their classrooms, which aspects of their teaching would be observed, what was going to be discussed in the feedback discussion, etc.It also meant that they no longer worried about being evaluated by an outsider, because they were their own evaluators.In this way, the teachers came to welcome classroom observation as a practical aid to their self-development (As cited in Wang & Seth 1998, p 205).The typical procedural framework done in the aforementioned study was: Pre-classroom observation questionnaire, Classroom observation process, which was divided into: a. pre-observation discussion, b. classroom visit(s), c. post-observation discussion, & Post-classroom observation questionnaire.Malderez (2003) has reviewed some key observational concepts in a kind of ELT context.The idea of observation is put forward not just as "seeing" but using all the other senses available to gather information.In fact, observation is used in all educational contexts, but what matters here is that they are handled in a kind of descriptive way rather than being interpretative or evaluative.Also, four various observational purposes are stated in the aforementioned article, as observation for developmental purposes, training, evaluation & research.Each of which surly affects the observer, the person observed, learners, learning process, as well as what happens before & after the observation process.
Moreover; Tsai (2008) wrote an article based on the fact that observing existing classroom practices could bring forth insights into improving a language teacher"s instruction.To this end, he tried to do the data collection through videotaped classroom observations, a classroom observation protocol, a post-study open-ended questionnaire to the students and a post-study interview with the teacher.Later on, he went through four various stages in his study.Stage 1 dealt with preobservation discussion and negotiation with the teacher, stage 2 videotaped classroom observations which were conducted four times at a junior high school English class in Taiwan, stage 3 involved discussing with the teacher the results obtained from the observations, and seeking options to refine classroom practices, & the final stage aimed to implement and evaluate the options taken.Finally, the reactions gathered from the teacher and students indicated that the options implemented-preview activities, group discussion and controlled/guided writing practices -made English learning more impressive and effective.(Gebhard, 1999;Sheal, 1989;Wajnryb, 1992;Williams, 1989) as cited in Barocsi (2007, p 129).Therefore, he has tried to put more emphasis on professional development rather than assessment in his article (Bodóczky & Malderez, 1994).In his paper, he looks primarily at the field of Teaching English as a Foreign Language (TEFL) and outlines a qualitative research project which includes ethnographic elements with regard to the teacher trainer who is he himself in that current situation.The participants are student teachers in the course of teaching practice, & the data analysis is obtained through observation tasks which include the comparison of unstructured and structured observation in the classroom.The outcome revealed that teacher trainees can gain substantially from focused observation targeted on particular areas.
On the other hand, Williams (1989) also maintains the same modern belief regarding the fact of observations should have a kind of developmental nature and as a result her article describes a scheme of developmental classroom visits currently in use in an in-service teacher-training programme for primary teachers in Singapore.Also, the reasons behind the dissatisfaction with traditional observations are listed in the study, and the principles behind the new format such as: Developmental, Limited and focused content, Course-link, Teacher-centeredness, Future development, Positiveness, &.Flexibility is discussed.Murdoch (2000Murdoch ( -2009) ) has stated some general principles for making observations valuable & more applicable for teachers too in the current situations.He pinpoints the importance of having a kind of pre-observation discussion in which teachers can express ideas related to the common students" problems in that specific class as well as the style of teaching employed by the teacher during the term.Also, in the aforementioned discussion both the teacher & the observer can agree upon the focus of the observation process which makes everything more vivid for both sides.For the postobservation discussion, he suggests the idea of postponing it to some days later, as both the teacher & supervisor can reflect on the teaching procedure which has taken place, & also respecting the teacher"s perspective on the events happened during the observation session.Badre (n.d.) in his article "Toward a reflective classroom observation" has gone through two parts.First, he has tried to distinguish the existing differences between traditional observations & modern ones.And then he has endeavored to promote a kind of systematic incorporation of peer observation within the professional & pedagogical training of Moroccan public schools.To do this, his article is consisted of five parts: types & purposes of observation, traditional view versus reflective view of observation, phases of observation, strengths & downsides of peer observation, & observation instruments.
In addition, Chavez (2006) has undertaken a study at the National Autonomous University of Nicaragua, the main objective of which is to explore & analyze teachers" beliefs, knowledge, & perceptions concerning the teacher-learning process of EFL & the impact those elements exert on the teachers" performance as well as behavior in the classroom.The results did reveal some suggestions aimed at providing the teachers with the vital tools for teaching English effectively in the Nicaraguan EFL context.

Development and Validation of a Questionnaire
Due to the fact of needing an instrument, as the first step of any research, which has an acceptable degree of validation to gather information for any study, some researchers have aimed to do so regarding some sorts of questionnaires, therefore becoming a new trend in the field.Elahi  They have gone through a kind of exploratory factor analysis of the responses of a large sample of language learners & a subsequent confirmatory factor analysis with another large but different sample which resulted in a 21-item instrument with a high psychometric properties.Also, regarding its content validity, the researchers used expert judgment to review the list of strategies & self-regulatory abilities, at the same time trying to have the same administration procedures in all context as well as the tests they took.

Research Questions
To achieve the goals of the present study, the following research questions were posed: Q1: Does the questionnaire of EFL teachers' perceptions on classroom observation show good internal consistency?Q2: Does the instrument show good fit indices as measure by the Confirmatory factor analysis?

Participants
One hundred EFL teachers, who are teaching English in Jahan Elm, Institute of Higher Education and Hafez English Language Institute, Mashhad, Iran, were asked to participate in the study and fill in the questionnaire and give it back to the researcher within a week.

Instruments
The instrument used for this study was the Questionnaire of EFL Teachers' Perceptions and Views on Classroom Observation which was developed by the researcher.The questionnaire contains 2 main sections: the first one deal with the demographic items such as age, sex, years of experience, etc., and the second section, which is the main section of the questionnaire, consists of 20 items.
The items of the second section were the result of holding several interview sessions with about 25 EFL teachers at Hafez Foreign Languages Institute.The teachers were asked some questions related to their attitude towards classroom observation.Their answers were recorded, transcribed and verified by three different EFL expert teachers.
It is a teacher rated assessment of their perceptions and views on classroom observation.

Procedure
The theoretical validation of the instrument i.e. questionnaire was done by a comprehensive review of the related literature on classroom observation.The different standard methods for developing valid and reliable questionnaires were examined (Brown, 2001;Dornyei, 2003;Gilham, 2000).We also scrutinized the field for other existing instruments related to the classroom observation.
The researcher held several interview sessions with EFL teachers and asked several questions related to their attitudes towards classroom observation.I prepared field notes regarding EFl teachers' beliefs, attitudes and hunches related to classroom observation.The objectives of the issue i.e. were all specified.Comprehensive reports were provided as the result of the question and answer sessions accompanied with relevant discussions around the topic.All of the sessions were recorded to make sure no point has been missed during conducting the interview.The recorded report was then transcribed and was utilized as a guide to develop several items.
The initial list of items was constructed following these steps and then the items were subjected to expert judgment for clarity, content validity, and redundancy.After a revision along with discussion the items were rewritten and some of the irrelevant items were eliminated.The result of the item assessment rounds was 20 items.
Finally, the revised instrument was piloted with about 100 EFL teachers to receive additional feedback on clarity of the items.The teachers were asked to respond to the items using a 4-point Likert scale ranging from "strongly agree" to "strongly disagree".An anonymous personal information cover page was designed to get necessary information while assuring the participants that confidentiality will be respected.
The final version of the questionnaire of EFL Teachers' Perceptions and Views on Classroom Observation was given to the one hundred EFL teachers and they were asked to fill in the questionnaires and submit them to researcher within a week.

Statistical Analysis
The statistical analyses described in this study were conducted with SPSS 16.00 and LISREL 8.80.There are a variety of ways to validate an instrument, i.e. to assess its Construct validity.The first step is usually a test of internal consistency of the instrument using Cronbach"s coefficient alpha reliability estimate which has a general accepted target value of .74(Garson, 2005, Lewicki & Hill, 2006).
Confirmatory factor analyses (CFAs) were conducted using LISREL 8.80.Furthermore, in line with the work of Batinic, Wolff and Haupt (2007), the goodness of fit statistics was obtained emphasizing the Root mean square error of approximation (RMSEA), the comparative fix index (CFI), Non-normed fit index (NNFI) and incremental fit index (IFI).The X2 test was also used to test the fitness of the model.RMSEA values less than .05indicate good fit and values as high as .08represent acceptable errors of approximation (HU and Bentler, 1998).The CFI/NNFI and IFI differ along a 0 to 1 continuum in which values greater than .90and .95are considered to show an acceptable and outstanding fit of the data (Bentler, 1990).However, an index of .90 and above is considered as acceptable fit (Harrington, 2009).

Internal Consistency
The Cronbach"s alpha coefficient estimate for the instrument and subscales was acceptable and reached the target reliability of at least .70(Garson, 2005, Lewicki & Hill, 2006).

Confirmatory Factor Analysis
The CFA model estimated the ability of the eleven factors to clarify the relationship among the 20 items.This model with 20 items positing four factors provided a good fit to the data ( CFI=.94, NNFI=.93,IFI=.92,RMSEA=.07 and x2 =972.09,df=569,p=0.0).The solution was entirely proper and the factor structure was well definite with all factor loading being positive and significant and were larger than .30(from .43 to .88)See table 2. The correlations among the three factors were small to moderate (rs=.32 to .58),indicating that the factors were clearly distinguishable from one another (see table 3).In sum, there was support for the eleven factor model in Iranian sample based on the finding of; (a) a reasonable model fit (i.e.CFI, NNFI, IFI=.93),(b) good factor loading for the model (.43 and above for each item loading on the respective factor), and (c) reasonably low correlation among the eleven factors (<.7).Since the Chi -Square equals 6.51 and the p-value is larger than 0.05, then we can draw conclusion that the model fits the data, i.e. the model is appropriate.This model only shows the relationship between the latent variables and the observed variables of the study.
The values which are written on each arrow are demonstrated in the Estimated Mood, and they cannot be appropriately interpreted.In all SEM models run in LSREL software, the values of Estimated Mood are not interpretable because there is no principle to which one can compare these values.In order to make the values interpretable, we should change the mood from Estimated Mood to T-Value Mood.Having changed the mood to T-Value, we see that all the values written on the arrows of the above model changed and are higher than 1.96 (1.96 is a predetermined principle value to which all the values are to be compared, when the Critical Ratio (CR) is > 1.96 for a regression weight, that path is significant at the .05level, i.e. its estimated path parameter is significant) (Ullman, 2001).As a result, we can conclude that there is a meaningful relationship between the observed variables (personal Attitudes, Educational and Teachers' Demands) and their latent variable.

DISCUSSION
As seen in table 1, the internal consistency analysis of the Questionnaires of EFL Teachers' Perceptions and Views on Classroom Observation (PCOQ) utilizing Cronbach"s coefficient alpha reached acceptable alpha(s) which rejected the first hypothesis.The results of the CFA as assessed by the CFI/NNFI, RMSEA and chi-x2, reached acceptable fits.Table 2 showed that the factor structures were defined and table 3 showed that the factors were clearly distinguishable from one another; this also rejected the second hypothesis and is also consistent with the finding of the original author.

CONCLUSION
As it has been proved elsewhere & repeatedly throughout all previous related researches, supervisors do play crucial roles in teachers" professional status.But what matters here are the teachers" perspectives toward observation, observation procedure, & specifically supervisors" role in each & every academic setting.Therefore, by what Midttun (2007) puts forward about observation as an ongoing guidance, supervisors can come to the point that they should be there to do at top of their best to help teachers with their educational skills step by step until they are considered as a thorough good teacher.While doing that, they should keep some points in mind in order to make that procedure more valuable & effective, here particularly, to teachers.That is trying to have pre-observation, observation, & post-observation meetings with the teacher being observed, Murdoch (2000Murdoch ( -2009)).To do so, first of all it"s better to gain some information out of the view points of what teachers in a specific setting may look forward within a supervisor, what he/she should focus on more, how he/she should behave, & the like to make that procedure more fruitful.To this end, the researcher has tried to develop & validate a questionnaire as an instrument to gather more information on EFL teachers" attitudes towards classroom observation, which did reach robust psychometric characteristics as well as a good construct validity to help those supervisors who are looking to find a deeper insight out of what teachers think about classroom observations in order to overcome the tough situations faced with.After all, due to the limitations of the present study, more research is needed in the aforementioned area.Similar studies can be carried out to find about some ways of how those debilitative obstacles on part of teachers can be overcome.Also, further related studies may be helpful to see any existing differences regarding supervisors" roles in various academic settings.
Barocsi (2007) has reviewed classroom observation backgrounds, talking about different observational purposes such as: one teacher observing another teacher for the purpose of self-improvement or research (Cohen, Manion & Morrison, 2000; Gebhard, 1999; Mackey & Gass, 2005), & also carrying observation for assessment; which was indeed traditional, and often the only reason for observing teachers and classrooms & Ghonsooly(2010) tried to validate a Locus of Control questionnaire and then examining its relation to general English achievement.To do so they used Internal Control Index to measure the participants" locus of control, & regarding validation they checked the content as well as the physical appearance of the questionnaire with the help of three experts.Therefore, they validated the Pearson version of the LOC instrument which resulted in the identification of eight factors.In addition, Pishghadam & Khosropanah (2011) examined the validity of the English language teachers" competency test.To this end, they made use of Exploratory Factor Analysis to check the construct validity of the test.And the factorability of the intercorrelation matrix was measured by the use of two tests: Kaisar-Meyer-Olkin test of sampling adequacy & Bartlett"s Test of Sphericity.The results proved that the underlying factors of the test were what the test-designers claimed to be.That"s ten factors represented the underlying structure of ELT-CT.Considering the metacognitive awareness listening questionnaire, Vandergrift et al. (2006) has developed & validated one, too.

Table 2 .
Summary of the Item -factor loading

Table 3 .
Sub-factor correlation of the eleven factors of MRQ