Constructing Understanding Through Critical Reasoning: A Comparative Study Of First Year Undergraduate And Postgraduate Writing And Communication Classes

Reflections on Practice

Sykes, A. H., & Azfar, J. (2019). Constructing understanding through critical questioning: A comparative study of first year undergraduate and postgraduate writing and communication classes. Asian Journal of the Scholarship of Teaching and Learning, 9(1), 59-69.

View as PDF      Current Issue  

Abstract

This study explored how the forms and depth of critical questions asked by students differ between first year undergraduate students and first year postgraduate students in the Faculty of Engineering. The authors hypothesised that students who were of greater seniority, and would have had more opportunities to encounter, and respond to, higher-order questions were likely to (1) field questions more frequently, and (2) ask higher-order critical questions compared to their peers who are new to university life. To address these hypotheses, a comparative observational study was conducted with one undergraduate class and one postgraduate Engineering class to find evidence of the relationship between learning to think and developing thinking skills required for learning. An adaptation of Bloom’s framework was used to categorise the types of questions raised. The results showed that (1) there was no clear difference between the number of questions asked by the undergraduates and postgraduates, and (2) academic experience appears to be significant in determining the types of questions asked. There was a difference in the types of questions asked by and the level of thinking skills of undergraduates and postgraduates. The former displayed lower-order skills, while the latter displayed higher-order skills.

Background

As teachers playing as minor a role in the whole process as possible, acting only as facilitators adopting a constructive approach (King, 1993), we were keen to investigate whether the number of years of academic experience that students have indeed affects questioning in the classroom. This research was based on the principle of “Guided Reciprocal Peer Questioning” (King, 1989) by which students respond to situations by asking questions which are both generic and specific. Generic questions, which are content-free, such as “What conclusions can you draw about..?”, become a guide for students to then move on to specific questions that force them to identify ideas and show how they relate to one another. In the process, these questioning steps are assumed to induce higher-order thinking that at the same time, induces collaboration as the class discussion takes a dialogic form.

Objectives

This collaborative study aimed to situate the idea of “critical thinking” in a constructivist framework, where critical thinking in the form of questioning is used as an instrument to create meanings of presentations or ideas students are encountering for the first time. Whilst acknowledging that in the realm of higher education, critical thinking has been touted as a “free-floating entity” (Moore, 2004) with no specific measurements or indicators, the investigators decided to adopt Bloom’s educational taxonomy (Bloom, 1956, and revised 1989; 2001), and adapt it in a way which applies to the context of the classrooms being studied.

For this study, the authors hypothesised that students of greater seniority, who would have had more opportunities to encounter, and respond to, higher-order questions will (1) field more questions during presentations, and (2) tend to ask higher-order critical questions compared to their peers who are new to university life.

Methods

This was a comparative observational study, which involved collaborators conducting observations of each other’s classes. In particular, this study focused on the final presentations in each module, in which students were required to present to their peers. Both teachers observed the presentation sessions, but gave very little guidance or support to the questioning process. The constructivist approach that students adopted to understand their classmates’ presentations was assumed to be a means that demonstrated critical thinking (Tsui, 2002) by reflecting how their thinking was stretched and challenged (Gershon, 2015). This process could then be evaluated by the observers through the application of Bloom’s Taxonomy. By comparing undergraduate and postgraduate students, we hoped to find evidence of the relationship between learning to think and developing the thinking skills needed for learning (Pohl, 1999) at different levels of university learning and beyond.

Context of the classes being studied

Both classes were reading writing and communication modules tailored specifically to engineers. In the postgraduate module ES5101 “Technical Communication for Engineers”, there were 15 students in a class. Question-fielding formed part of the class participation score, which was worth 5% of the total module grade. Students shared a briefly written general overview of their presentation several days prior to the actual presentation—they would have been given prior opportunity to reflect and consider the feasibility of the presentation in preparation of the questions to field.

In the undergraduate module ES1531 “Critical Thinking and Writing for Engineers”, there were 18 students in a class. Question-fielding formed part of the class participation score, which was worth 10% of the total module grade. Students also had one week prior to the lesson to read the reports of the presenting team, and were tasked to generate questions to be fielded during the presentation.

Questioning techniques had not been specifically taught in both classes—as such, the study sought to investigate if students were able to employ higher-order questioning techniques without explicit scaffolding. Teachers, however, did role-model and employ questioning techniques during mini-class discussions and informal presentations throughout the semester.

Academic experience and seniority

In the context of this study, the passage of time between undergraduate and postgraduate studies would presumably have exposed students to higher-level types of questions that they would have to answer, as well as opportunities to ask such questions in class (either to their teachers or during peer presentations). With specific reference to Engineering students, the “academic experience” in this context refers to the opportunities for them to encounter and respond to higher-order questions like the Evaluate and Createtypes of questions, illustrated in the following section. These questions are particularly pivotal when students are in the process of completing their final-year theses. Such questions are also constituent components of other higher level engineering modules, as well as writing and communication courses that students may need to attend.

Bloom’s Taxonomy

Bloom’s Taxonomy of educational objectives (Figure 1) was used as a multi-tiered framework for evaluating the types of questions fielded by the students during the presentations. This taxonomy classifies knowledge (or in this case, the questions posed by students) in six tiers in ascending order, as follows: Remember (basic level of thinking skills), Understand, Apply, Analyse, Evaluate, and Create(highest level of thinking skills).

For this study, the questions fielded by students were classified into four categories: Remember, Apply, Evaluate, and Create. Remember and Apply are considered lower order or basic thinking skills, while Evaluate and Create are considered higher-order, abstract thinking skills. These four were deliberately chosen to more clearly delineate and categorise different forms of questions fielded by students. Table 1 below illustrates the different types of questions hypothesised for this study, using a modified version of Bloom’s Taxonomy.

Coding and inter-rater agreement

Questions were coded according to the predetermined categories identified from Bloom’s Taxonomy because these were the areas of interest to the authors. There was only one rater for each group. The authors cross-checked each other’s ratings of the questions from each other’s groups to verify the correct placement of each question into the most appropriate category. There were no disagreements.

Discussion of Findings

Undergraduate class

For the undergraduate class, students were keen to field questions and had genuine interest in projects they had not heard of before. Many of these questions, however, were used to clarify knowledge and construct synthesis between different parts of the question, rather than provide a critique of the ideas discussed. Only one student posed a Create question that might potentially chart new directions and recommendations for the presentation. It was also noted that often, all members of the “audience” would ask at least one question related to the presentation. Overall, about 60% of the questions asked involved Apply questions. The authors hypothesised that most of these questions were meant for clarification purposes, as students in the audience attempted to construct connections between different parts of the presentation. The next main types of questions observed were Evaluate questions (about 30%). It was noted that students were clear about the meaning of “evaluation” in terms of their questions raised, as many posed questions about the benefits and limitations of the solutions presented by their peers.

Postgraduate class

For the postgraduate class,the most common types of questions raised were Create questions (55%). These questions forced the presenters to not only critique their given paper, but to go beyond stating the study’s strengths and weaknesses and suggest substantiated and feasible alternatives. The next most common types of questions raised were Evaluate questions (38%). This finding indicates that students were consciously formulating judgements about the relative merits of the ideas and conclusions drawn by the presenter. They were not prepared to simply accept the presenter’s critique without pressing the latter to provide specific details and examples to corroborate their comments.

Analysis

Based on the findings presented in the previous sections, a clear difference is evident between the types of questions and, therefore, the level of thinking skills of first year undergraduate and first year postgraduate Engineering students. The former display lower-order skills, while the latter display higher-order skills, as shown by the frequency of the types of questions asked in response to the presentations given in each group.

There is some common ground between the undergraduate and postgraduate students in relation to Evaluate questions. Both groups had Evaluate questions as the second most frequent type of question asked in response to their peers’ presentations. This finding might indicate that the undergraduates are developing thinking skills beyond the lower-order ones and are gaining the knowledge and confidence to be evaluative when considering the ideas, problems, and solutions presented to them. As for the postgraduate students, they should be accustomed to making judgements on the relative merits or shortcomings of ideas, procedures, processes, and materials because this type of evaluation is essential for identifying knowledge gaps that lead to research questions. This is a skill necessary for postgraduate studies.

The findings of this study, that most of the questions from the postgraduate students were Create questions, indicates that by the time they reach their first year of postgraduate studies, students appear to have developed the high-order thinking skills required for formulating higher-order questions. Based on this observation, it can be assumed that as students progress through their academic careers, and with greater exposure to new knowledge, new experiences, new disciplines (electives), and new ideas, they are likely to become more adept at developing Create questions. Perhaps by their second or third year, undergraduate students will be able to suggest to their presenter classmates new ways or methods to improve both the structure and content of their presentations.

Reflections

Observations from the study

For this study, the authors hypothesised that students who were of greater seniority, and would have had more opportunities to encounter and respond to higher-order questions like Evaluate and Create types of questions were likely to (1) field questions more frequently, and (2) ask higher-order critical questions compared to their peers who are new to university life. With regard to the first hypothesis, there was no discernible difference between the number of questions asked by the undergraduate and postgraduate groups. Often the number of questions depended on the level of interest the presenters generated in their respective topics and the level of knowledge students in the audience had of the particular topic. The latter was particularly true in the case of the postgraduate students. For the second hypothesis, it was found that academic seniority appears to be significant in determining the type of questions asked. A correlation was found between lower level (first year undergraduate) students and the greater likelihood of Apply and Evaluate questions. Apply questions are generally considered to be indicative of lower-order thinking skills. On the other hand, a correlation was found between higher level (first year postgraduate) students and a greater likelihood of fielding Create and Evaluate questions. Both Create and Evaluatequestions are considered to be evidence of higher-order thinking skills.

Recommendations to develop questioning skills

In relation to teaching strategies used to facilitate good questioning, this study situates the role of the teacher as a facilitator of good questioning techniques appropriate to the academic seniority of the student. For instance, at the postgraduate level, classroom lessons that are teacher-led should “role model” Create types of questions that would spearhead students’ thinking beyond the boundaries of an issue, leading to more constructivist outcomes. This is because Create questions constitute a fundamental component of postgraduate studies—thinking about gaps in knowledge and contributing to the development of a field. Likewise, in a first year writing and communication class, the teacher can role model asking intermediate-level types of questions, such as Apply and Evaluate questions in order to provide a framework through which students can sharpen their questioning techniques. Having said that, teachers teaching first year undergraduate courses can also expose their students to Create questions, in order to spearhead curiosity about the various types of higher-order questions that they may ask in future. In this section, we propose three strategies which teachers can consider using to develop their students’ thinking and questioning skills.

Facilitation of critical questioning

It is proposed that teachers can facilitate the development of their students’ thinking and questioning skills, from lower- to higher-order levels, by initiating and demonstrating such questioning in their lessons and through the comments they provide as feedback. It is envisioned that by consistently placing emphasis on questioning techniques, rather than simply grading presentation content, teachers will be able to cultivate a community of inquiry (Lipman, Sharp, & Oscanyan, 1980) in the classroom that is self-sustaining, with students gradually developing the ability to steer class discussions independent of teacher intervention. This is of course premised on constant scaffolding on the teacher’s part through learning activities where students are consistently given opportunities to formulate and ask various types of higher-order questions. Such a technique is characteristic of the transfer of responsibility for the task of critiquing presentations to the student (Fisher, 1993), which will gradually reduce student dependence on the teacher.

Assessment of questions, rather than answers

Students could also be assessed and graded on the types of questions they ask during class presentations. If students are aware that they are being graded on the quality and depth of their thinking and questioning during such sessions, they have a greater incentive to think about and question their peers’ work more critically. It has been suggested, as a result of this study, that the authors revise the rubrics for assessing classroom presentations. Instead, the revised rubrics should include components that assess the types and quality of questions posed by students during their peers’ presentations, rather than grading students only on the quality of their own presentations

Q&A sessions as a collaborative and social learning experience

Thirdly, we also suggest that the question-and-answer (Q&A) sessions in the classroom be given more emphasis in any discipline or subject, as they are not only effective in assessing students’ understanding of content (via the presentation of ideas and peer critique), they are also useful to induce social learning (Bandura, 1977). Such sessions also give students the opportunity to apply and evaluate one another’s ideas, which is a direct application of teaching within Vygotsky’s zone of proximal development (Wells, 1999). Developing a culture of peer questioning amongst students also promotes mutual respect in the classroom, the inclusivity of ideas, and fosters a sense of open-mindedness and independence.

Conclusions

This study focused on an exploration of the extent of students’ critical questioning skills during graded class presentations. The research objectives were generated from our desire to investigate questioning skills as a manifestation of students’ thinking skills. We found that the postgraduate students posed more Evaluate and Create questions, which indicates that they were equipped with higher-order thinking skills as compared to the first year undergraduates. We can conclude from this observational study that on the undergraduate to postgraduate learning journey, students’ thinking skills would progress on a continuum, from the ability to pose basic Remember and Apply questions to being able to formulate higher-order Evaluate and Create questions.

References

Bandura, A. (1977). Social learning theory. Prentice Hall, Englewood Cliffs, NJ.

Bloom, B. S. (1956). Bloom’s taxonomy of educational objectives. New York: Longman.

Fisher, E. (1993). Distinctive features of pupil-pupil talk and their relationship to learning: How discursive exploration might be encouraged. Learning and Education, 7(4), 239-258. https://doi.org/10.1080/09500789309541363

Gershon, M. (2015). How to use Bloom's Taxonomy in the classroom: The complete guide. CreateSpace Independent Publishing Platform.

King, A. (1989). Effects of self-questioning training on college students' comprehension of lectures. Contemporary Educational Psychology 14(4), 366-381. https://doi.org/10.1016/0361-476X(89)90022-2

King, A. (1993). From sage on the stage to guide on the side. College Teaching, 41(1). 30-35. https://doi.org/10.1080/87567555.1993.9926781

Lipman, M., Sharp, A.M., & Oscanyan, F. (1980). Philosophy in the classroom (2nd Ed.), Temple University Press, Philadelphia.

Moore, T. (2004). The critical thinking debate: How general are general thinking skills? Higher Education Research and Development, 23(1). 3-18. https://doi.org/10.1080/0729436032000168469

Pohl, M. (1999). Learning to think–Thinking to learn: Models and strategies to develop a classroom culture of thinking. Cheltenham: Hawker Brownlow Education.

Tsui, L. (2002). Fostering critical thinking through effective pedagogy. Research in Higher Education, 40(2). 740-763. Retrieved from https://www.jstor.org/stable/1558404.

Wells, G. (1999). Dialogic inquiry: Towards a sociocultural practice and theory of education. New York: Cambridge University Press.

Julian AZFAR is an instructor at the NUS Saw Swee Hock School of Public Health, and formerly taught writing and communication courses at the NUS Centre for English Language Communication. His educational research interests include the sociocultural contexts of teaching and learning, comparative education, (inter)national education and citizenship. He currently teaches introductor y and interdisciplinary courses in public health, and has a keen interest in multi-disciplinary teaching and curriculum development.