Christopher Miller has been involved in education professionally since 2006.  He has worked at the secondary level in the United States, the Republic of Moldova, and since 2010, the Republic of Korea.  Christopher holds an MSEd in TESOL from Shenandoah University.  He is especially interested in quantifying the benefits of reflective practice.  Email: cmiller112@su.edu

Several years ago, one of the teachers from the TESOL MSEd program which I was enrolled in told me something which has stuck with me up until the present day: “Assessment drives everything.”  I’m not sure I fully grasped some of the deeper implications of that statement until rather recently, but when reflecting on my initial attempts at designing a discussion based speaking test for students, her words seemed apropos.  This article will detail my first foray into assessing students’ conversational skill in a project I dubbed “Global Issues Discussions,” as well as some of the lessons learned along the way.

 

How the Global Issues discussions unit came into being

There were a variety of issues at my school, my classes, and my professional development which led to the instructional unit that described in this article.

The school I currently work at is a special-purpose high school specializing in foreign languages based in Seoul, South Korea.  Students select a major language and receive a minimum of 8 hours weekly instruction in the major language and (again minimally) 7 hours in English.  Students meet in so called conversational classes, instructed by NESTs, twice weekly for 50 minute sessions.  Class sizes range from 11-14 students.  There are currently four NESTs at the school and I presently hold the position of head teacher among the NESTs.  Traditionally, the school has recruited exceptionally performing students throughout South Korea.

However, due to the declining birth rate of South Korea (Song, 2019), the school has begun to recruit from students who have varying degrees of English proficiency. Hence, in brief, my supervisor a few years back requested that I make the curriculum “simpler.”  That led to a year-long process of attempting to simplify the curriculum.  After mixing and matching a few different projects, such as producing humorous skits, traditional “TOEIC style” essays, and public speaking, our conversational English class still had a five lesson gap for the upcoming semester.

Around that time, two things happened.  First, I started reading a book by Alexis Wiggins called “The Best Class you Never Taught,” which focuses on spider web discussions.  I cannot do justice to that book in this article, nevertheless, I highly recommend it.  This book focused on promoting autonomous student-led discussion.  The book emphasized balanced speaking time, creating a supportive atmosphere, and cultivating 21^st century (communication) skills among students. 

Concurrently, I had a “critical incident (Farrell, 2014)” that really brought home the need for me to give greater emphasis to respect in my assessment of students’ spoken output.  When students were preparing for an upcoming graded humorous skit, I listened to another student ridiculing another student’s output, saying:  “That’s such a stupid idea, you’re such an idiot.”  This was delivered in a playful/humorous manner and the students engaged in this exchange were technically on task and communicating in the target language.  However, I still felt uncomfortable with the utterance, realizing that such statements were not appropriate in a school setting.  I did have an allotted slot in my assessment to deal with such behavior from students, namely the participation portion of the grade (which was about 10% of a student’s total grade).  However, through this incident and by reading Wiggins’ fine text, I realized I could make respectful communication a more integral part of both my curriculum and assessment.

 

Key features of Global Issues discussions

I have attached the rubric and schedule for this brief unit in the appendix (see appendix 1 and 3).  The key features of the unit are as follows:

• Balanced speaking times among members, deliberate respect, and fostering a healthy supportive atmosphere were graded components of the speaking test. With this in mind “9 principles of effective group communication” were drawn up with the assistance of other teachers and students were expected to display them during subsequent classroom discussions (see appendix 2). 
• In the first iteration of this unit, students spoke for an average of 1:30 per speaker.
• Students were given ample planning time: students would receive a list of 40 possible test questions on the 2^nd lesson of the unit (the test was on the 5^th unit (see appendix 3 for a schedule), each class was 50 minutes in length.  Students were given their test question about 8 minutes prior to the test, and then with the members of their group, they were allowed to plan aspects of their discussion prior to actually taking the test.  Likewise students did a practice test on the 4^th lesson. 
• Students were allowed to have access to notes taken during the 8 minute planning session during the actual test.
• Besides assessing students’ English competency, other instructional goals included promoting pro-social norms and interdependence, as well as providing students opportunities to communicate in a semi-formal register in the target language (such as might occur during a professional meeting—see appendix 4).
• Students would provide a form of peer feedback to each other in the 4^th lesson during the practice test.  Feedback models were provided to students in the 2^nd lesson, and based on principles derived from Marzano, et al (2003). 
• Each student was responsible for asking one question during the speaking test.
• The test was administered in the 5^th lesson.

 

Speaking test features (1^st Version)

Before the Test	During the Test
Students receive a list of 40 potential test questions on lesson 2. Students receive actual test question 8 minutes before the test. Students have 8 minutes to plan for the speaking test. Students are given a blank sheet of paper for writing notes during the test. Students give and receive peer feedback on lesson 4. Students perform a practice test on the 4th lesson of the unit.	Students speak for 1:30 on average. Each student must ask at least one question. Students are able to view notes taken during the planning time.

Test questions/prompts were designed to get students to take a position on a variety of social issues.  Test questions, which were produced in collaboration with the teachers administering the conversational English classes included:

• Agree or Disagree: When I am older, I will want children.
• What is best: single-sex schools, co-ed schools with mixed classes, or co-ed schools with single-sex classes?
• Should students be forced to repeat a school year if they don’t do well enough?
• Can computers be as good as or even better than teachers?
• Should students be able to skip a grade in school if they are smart enough?

 

Issues during the first run and subsequent modifications

There were some challenges that I, nor my team of teachers, had not adequately foreseen related to multiple aspects of this unit.  These included:

A) Wasting time: Students would at times re-read the discussion prompt or repeat a prior speakers ideas, in what appeared to be a way to fill out their expected speaking time.

B) Maintaining balanced speaking time.

C) Asking questions: Students would often elaborate a rebuttal to a student’s position, then add “what do you think about that?”—rather than directly asking a more focused/effective question.  For instance on the topic of using strong punishments to modify student behavior, on student formulated a “question” in this manner: “I think instilling fear would have negative effects.  Like students would be afraid to ask questions to the teacher.  What do you think about that?”

D) Over-rehearsing: As mentioned previously, students received 40 questions during the 2^nd lesson.  The hope was that students would be cued about potential test questions (hence somewhat mitigating their test anxiety), but that it would be impractical for students to rehearse all of the possible test questions in advance.  Nevertheless, a number of students attempted to do just that.  This led to my supervisor indicating to me that students were complaining about a lack of sleep as many students elected to stay up until three or four a.m. on the night before the test to prepare for all possible test questions.  Hence, the next time we administered this unit, students were only able to know their test discussion question 8 minutes prior to the start of the exam.

G) Direct reading from the planning sheet: again the intention was for students to outline discussion points and to brainstorm content for their upcoming discussions with this sheet given during planning time.  Unfortunately, a subset of students chose to write out their comments and directly read from the planning sheet during the speaking test, thereby limiting the authenticity of the communication among students. 

With all of these issues under consideration, a series of modifications were integrated into the assessment criteria in the 2^nd version of the global discussions unit (see appendix 5).  First, respect and support were combined, thus reducing the total weight on the grade from 20% to 10%.  It was specified that students needed to ask content-based questions, and that statements such as “what do you think?” or “How about you?” were insufficient if students hoped to receive full credit for their grade in that component of their assessment.  “Content-based questions” refers to questions that are generated from listening to the output of other students during the discussion test.  For instance, a content-based question might be requesting further information related to another student’s utterance: “you mentioned failed policy, could you provide me with an example of a failed policy of the school?”  To limit student reading, a new category was added to the grading criteria: responsiveness (i.e. students are genuinely interacting).  Also, to add more granularity to assessing student speaking, the category of English usage was subdivided into both fluency and accuracy.  Finally, speaking time was increased from 1:30 per student to 2 minutes each.

Beyond the grading criteria, there were also additional instructional modifications.  For instance, to help students formulate better, direct questions, a framework of question types were introduced, such as elaboration, clarifying, and challenge (i.e. another speaker’s views) questions.  To help students have a better sense of organization when generating their content: it was suggested that students divide their speaking time into two parts: in the first part, perhaps the first 1/3, students could state their opinion on the topic, give reasons and provide additional support (such as personal examples, details, et al.).  In the remaining portion of a group’s speaking time, students could engage in a general question and answer session with each other.  It was emphasized that this was merely a suggested framework and that if a student chose to not follow this suggested framework it would not have a negative effect on his/her final grade.  During planning time students were not allowed to use a full sheet of paper, instead students had access to a note card, which they were free to fill up however they saw fit and could utilize throughout the speaking test.  Furthermore, students could use a blank sheet to write questions or jot down additional notes during the discussion test.

 

Problems and modifications for the 2^nd version of the Global Issues discussion unit

Problem	Modification
Excessive reading Superficial questions by students Students attempting to prepare thoroughly for each possible test question.	Students can only bring in a prepared note card/ grading has criteria for “responsiveness.” Content-based questions were required for assessing student performance. Not providing students with a list of possible test questions in advance.

 

Sample of student output

After putting these revisions in place, I felt there was an overall improvement in output among students.  While I Recognize the presence of several prescriptive grammar and word choice issues, here is a quality example where student interaction produces a clear use of questions, substantial logical support (details and examples), and also features students displaying a healthy amount of respect.  The prompt for this sample was: “Should Korea use foreign services (i.e. facebook) or concentrate on developing their own domestic products.”

A: Can I ask you a question?  So don’t you like somewhat think originality is needed in those domestic products….I don’t…we shouldn’t follow up to the point that, how Western or sites from other countries how their thriving, we need originality and people to understand Korean culture and to understand it better?

B: And I think our originality is better and like Kakao Talk, like there’s some games involved like different to facebook or google like and there’s those especially features that Kakao talk do only have, so we should sustain that and just learn what we don’t have (um…) not just copy (yeah), but learn from (yeah I get your point) it.

C: Can I interrupt?  What kind of advantages foreign sites have?

B: Foreign sites like I think facebook has those good advantage, like they mixed those messengers and feed and posts all together, but our Kakao talk only has the ability to just messenger or news, so I think it’s a good way to add some functions about posts or what else…

 

Recurring questions and unresolved issues

With the other teachers in our conversation class, we addressed, but did not fully resolve the following issues:

1. Planning time vs. spontaneous speaking.  One teacher strongly criticized this project, feeling that by giving students plenty of practice time over multiple lessons, that the authenticity of the task was compromised.  I justified the decision by citing the research on the value of planning time for enhancing the quality and quantity of student output (see Ellis, 2003).
2. Accuracy vs. fluency.  This speaking test assessed both accuracy and fluency.  A few team members noted that assessing both simultaneously can have a negative influence on student output.  One colleague’s argument was that if students know that they will be assessed for accuracy they will excessively monitor themselves, and that will limit their fluency.  Other team members justified assessing both dimensions as the pressure on conversation class teachers at our school to differentiate student scores is substantial.  By using both accuracy and fluency we were able to assess students in greater detail and via objective criteria. 
3. Micromanaging student discussion.  Though students did have plenty of freedom for how they went about engaging in this discussion test, suggested frameworks were provided as well in the second version of this instructional unit.  How much did those frameworks influence students?  How much did these frameworks lead to somewhat artificial responses, thereby reducing the spontaneity of student conversations, and the display of performative discussions by students?  I don’t have clear answers to these concerns.

 

Top take-aways

There were a variety of insights gained in the process of crafting and modifying this curriculum with my excellent group of teachers.  Some of the key insights included:

1. There are additional opportunities to give students more ownership over the curriculum.  For instance, students could write the test questions in advance of the conversational unit instead of having teachers produce the prompts (which is what was in fact done during both versions of this instructional unit).  Of course, teachers would have to both edit and vet the questions, nevertheless student could undertake some responsibility for producing essential aspects of the curriculum.
2. Acknowledging certain items in the assessment criteria is often enough.  After making respect and support components of the grading criteria for this assignment, students provided ample amounts of respect and kind words to each other during this speaking test.  Even though the weight of those items was reduced by 50% in the second version of this unit, students still continued to communicate to each other in a kind and considerate tone.  I doubt students would have been as considerate to each other had those items not been in the grading criteria.  However, the items did not have to be weighted heavily to produce desirable results.
3. Assessment can be a force for good.  Much has been made of washback effects in ELT for decades (see Taylor, 2005).  While washback does have a wide array of undesirable consequences, it can also have positive benefits.  Remember the words of my professor from graduate school which I noted at the start of this article: “assessment drives everything.”  Yes, that is valid.  If assessment takes account of pro-social behaviors—then students magically start to communicate in a more pro-social way.
4. Finally, as a follow-up to the previous point, performative isn’t necessarily bad.  While I was impressed by the kindness and respect students exhibited to each other during this speaking test, I recognized that many students were doing it to complete the assignment successfully and so as not to harm the grades of their fellow teammates (as there were interdependent aspects to the individual student’s final grade).  Nevertheless, taking a longer-range view, I’m not sure that displaying “performative behaviors” is a major problem.  Yes, students might be faking it, or as one of my more memorable undergraduate teaching methods instructors put it: “doing school,” nevertheless, I don’t see that as a huge problem for this particular assignment.  Students were able to uptake a few polite and appropriate expressions, they received training in communicating in a respectful fashion.  This practice may help to lay the groundwork for further considerate communication when students face a more authentic communicative situation.  This performative display may aid their future performance regardless of the incentives used to initially produce the desired behavior.

 

Conclusion

I have reviewed the factors that led to implementing the global issues discussion unit in the curriculum for conversational English at my current high school, its key features, challenges, subsequent modifications, lingering issues with the instructional unit itself, and finally mentioned the key insights I personally derived from teaching and reflecting on this series of conversational lessons.  After my experiences with the global discussions unit, I further affirm Alexis Wiggins words: “A rubric…helps keep the end goals in mind and allows [students] to self-assess accurately, creating direction and accountability.” 

 

References

Ellis, R. (2003).  Task-based language learning and teaching.  New York, New York: Oxford. 

Farrell, T.S.C. (2015).  Promoting teacher reflection in second language education: A framework for TESOL professionals.  New York: Routledge.

Marzano, R.J., Pickering, D.J., Pollock, J.E. (2003).  Classroom instruction that works: Research-based strategies for increasing student achievement.  Alexandria, Virginia: Association for Supervision and Curriculum Development.

Song, J. (2019, August 28). South Korea’s birth rate falls to new developed world low.  Financial Times.  Retrieved from https://www.ft.com/content/16505438-c96c-11e9-a1f4-3669401ba76f.

Taylor, L. (2005).  Washback and impact.  ELT Journal, 59 (2), 154-155.

Wiggins, A. (2017).  The best class you never taught: How spider web discussion can turn students into learning leaders. Alexandria, Virginia: Association for Supervision and Curriculum Development.

 

Appendix 1: 2018 Semester 2 Global Issues Discussion Unit

What	Point Value	Reasons for Securing Points	Reasons for Losing Points
Respect	1	Acknowledges the value of differing viewpoints (-.2 for each instance)	Ridicule Sarcasm
Self-Assessment (L3)	1	Orally Engaging in the Task during Lesson 3 (-.5 for poor performanceà -1 point for very weak levels of participation)	Not Doing the Task/ Not Being on Task
On Task for all Five Lessons	1	Self-Explanatory (decided by the individual teacher)	Self-Explanatory
Questions	1	Questions by students to Clarify an utterance by a conversational partner Critically, yet respectfully challenge another speaker’s perspective To promote further elaboration by a conversational partner (decided by individual teacher)	Absence of questions   Excessively superficial questions
Balance	1	Group members speak for approximately the same amount of time (decided by individual teacher)	Imbalance in speaking times among members (both speaking too much and too little)
Support	1	Conversation partners are supported both when difficulties in communicating ideas emerge and when expressing their personal viewpoints (decided by individual teacher)	A lack of support when communication difficulties arise, or sharply different viewpoints are expressed (i.e. there is a degree of hostility to divergent viewpoints)
English	1	Flawless grammar and word choice (-.1 for each “major” grammatical mistake -.1 for two of the following: preposition/articles)	Grammar mistakes/ word choice issues/ excessively simple utterances Reading (excessively) from the planning document
Written Peer Feedback Sheet (L4)	1	Students demonstrate a legitimate effort to complete the task based on criteria delivered in L3. (incomplete -1 point)	Students do not complete the task or do so in a manner that displays limited effort
Arguments are well supported	1	Evidence/ anecdote/ displays of logic and clear reasoning are presented by a speaker (decided by the individual teacher)	Lack of evidence/ anecdotes/ logic or clear reasoning   Excessively superficial
Demonstrates Comprehension	1	The member demonstrates comprehension of the perspectives expressed by the other members in his or her group (through questions asked and responses provided) (decided by individual teacher)	Repeated failure to demonstrate comprehension and an absence of effort/ attempts to repair misunderstandings

 

 

Appendix 2: 9 Principles for Effective Group Communication

#	Explanation	Summary
1	There should be a balance of speaking time among all members of the group.	Balance
2	When confusion exists there should be questions asked to clarify.	Question
3	Only English	English
4	The group should be supportive of all members.	Support
5	Sarcasm is minimal and more often than not should not exist.	No Sarcasm
6	All ideas are heard and respected.	Respect
7	Ideas are not ridiculed.	No Ridicule
8	For the most part one person is speaking at a time.	One
9	The discussion is generally on task.	Focused

 

Appendix 3: Schedule for Global Issues Discussion Unit

Lesson	General Focus	Key Features
1	Communicating Expectations; Appropriate Conversational Behaviors for a Semi-Formal Meeting; Vocabulary	Sample video Communicate rubric Appropriate questioning Key Chunks/expressions for disagreeing/ demonstrating active listening/ supporting and acknowledging alternative perspectives/ promoting Turn-Taking, et al. Pick teams
2	Vocabulary and Training Continued; Feedback training; Positive/Negative Video; Brainstorming Training;	Appropriate questioning Key Chunks/expressions for disagreeing/ demonstrating active listening/ supporting and acknowledging alternative perspectives/ promoting Turn-Taking, et al. 10-15 minute instruction on delivering feedback Sample video Give Students list of all 40 potential test-day questions Brainstorming training
3	Review Feedback, vocabulary, and conversational norms; Informal Rehearsal; Teacher Feedback	Topic selection and 9 minute planning time Simultaneous mock speaking test Teacher feedback with group/approximately 2-3 minutes each group Students orally complete self/group assessment reflection exercise
4	Mock Test	Observing students complete a peer assessment form, which will be graded Topics are selected, students have 9 minutes to plan Students will speak for 6 minutes (roughly 1:30 per speaker) Each student will have a teacher provided planning sheet during planning time and at the start of the test a blank piece of paper supplied by the teacher Teacher will provide some form of written evaluation (assessed rubric/written feedback)
5	Actual Test	Topics are selected, students have 9 minutes to plan Students will speak for 6 minutes (roughly 1:30 per speaker) Each student will have a teacher provided planning sheet during planning time and at the start of the test a blank piece of paper supplied by the teacher

 

Appendix 4: Expressions for Discussion Test Unit

Dimension	Expressions
Expressing agreement/ disagreement	I agree with you that… I see things a little bit differently That is one perspective, but…
Demonstrating active listening	So what you’re saying is… In other words… Are you trying to say…
Supporting and acknowledging alternative perspectives	That’s a very good point Let’s look at this from the perspective of… This can be a sensitive topic
Promoting turn-taking	Why do you feel like that? Would you like to take that one ________? Let’s hear from _________.

 

Appendix 5: Rubric for Global Issues Discussion Unit 2019 Semester 1

What	Point Value	Reasons for Securing Points	Reasons for Losing Points
Respect and Support	1	Acknowledges the value of differing viewpoints (.5 value)   Conversation partners are supported both when difficulties in communicating ideas emerge and when expressing their personal viewpoints (.5 value)	Ridicule Sarcasm   A lack of support when communication difficulties arise, or sharply different viewpoints are expressed (i.e. there is a degree of hostility to divergent viewpoints) (-.2 for each instance)
On task for all Five Lessons	1	Self-Explanatory	Self-Explanatory
Questions	1	Questions by students to Critically, yet respectfully challenge another speaker’s perspective To promote further elaboration by a conversational partner and reflect an understanding of the content	Absence of questions   Excessively superficial questions (-.5 for lack of questions, or repetition of superficial questions: “what do you think?”)
Balance	1	Group members speak for approximately the same amount of time	Imbalance in speaking times among members (both speaking too much and too little) (10 second gaps above or below can be used to lower grade by -.2)
Responsiveness	1	Students are engaging in actual dialogue	Students provide ”mini-speeches” which exceed 30 seconds; students are providing obviously canned responses; students are looking down at notes excessively or reading from the sheet; starting the discussion by taking time to read out loud the discussion topic again
Accuracy	1	Flawless grammar and word choice	Grammar mistakes/ word choice issues/ excessively simple utterances; Pronunciation; (determined by teachers, ultimately influenced by the total number of mistakes by a given student)
Fluency	1	Excellent Rate of Speaking	Excessive pauses (including lengthy ums and uhs); repeating words or ideas; (-.1 for each 2 instances)
Written Peer Feedback Sheet (L4)	1	Students demonstrate a legitimate effort to complete the task based on criteria delivered in L3.	Students do not complete the task or do so in a manner that displays limited effort
Arguments are well supported	1	Evidence/ anecdote/ displays of logic and clear reasoning are presented by a speaker	Lack of evidence/ anecdotes/ logic or clear reasoning Reasoning is excessively general or abstract; Excessively superficial
Demonstrates Comprehension	1	The member demonstrates comprehension of the perspectives expressed by the other members in his or her group (through questions asked and responses provided)	Repeated failure to demonstrate comprehension and an absence of effort/ attempts to repair misunderstandings

 

Please check the 21st Century Thinking Skills course at Pilgrims website.

Please check the How to Motivate Your Students course at Pilgrims website.

Please check the How to Teach Environmental Studies course at Pilgrims website.