Vocabulary teaching: Data analysis

給各位協助參與此試驗的同學，我不知道你們有多少人會看到這則訊息，但我非常感激你們和徐老師給我機會作試驗。我也承諾大家會把試驗結果發布在這裡。所以請往下看吧。有任何更新我會註明，這篇我是隨時修改。如果想直接聯繫我，請寫信到 102551020@nccu.edu.tw 或是 karst10607@gmail.com

(I use English here, and I would be posting another Chinese version or a Chinese notated version later)

Title of the research:

From whole to parts, incorporating Gestalt theory into vocabulary teaching.

 

Introduction and Literature Review:

From the beginning of any English courses, teachers and students face similar problems: how to teach/learn new vocabulary effectively? There are cults of phonetics, morphology and even teaching methods. But here I provide a simpler and more effective solution: the Gestalt way of teaching new vocabulary. This is a relatively ignored part of teaching/learning new vocabulary. I could only find very little information about applying or researching Gestalt theory in teaching new vocabulary. Nevertheless, some pioneers had contribute valuable findings in using Gestalt method. Shari Sabeti had investigated a way to use graphic novel to teach in secondary schools. It turned out to be an effective way to make students less alienated and more engaged. This is a beautiful yet simple example of incorporating Gestalt theory into teaching textbook and new vocabulary. I will now enhance it and apply it on 10th graders in PCSH. 

Methods:

The full test materials and answers are available at http://fall-cicada.blogspot.tw/2014/12/vocabulary-teaching-v9.html 

We had pretest, immediate posttest and multiple-choices questions. We had no time for cloze test at all. But we do need more questions to make the statics more representative and objective. So this would be a critical question to solve for future experiments. 

• Numbers of Participants:

Some answer sheets are missing or haven’t been counted into the statics due to their lack of uniform answer sheets. We need further clean up even after the investigation is over. (There are two piles of them in folders)

	pretest	multiple-c	post-test
109 (contr)	33	33	33
118 (exp)	34	37	33

• The background of participants:

They are 10th graders in PCSH (the first semester, freshmen for 2 months in high school). There are about 80 of them in total. They have more approximate proficiency since they just pass the entrance exam. If they are 11th or 12th graders, their variation would be quite huge.

Issues found, categorized based on the following subcategories:

1. difficulty of tested vocabulary varies
2. powerful suggestion by the instructor
3. too many questions in test materials for the participants in such limited time
4. Instability of the presenting media in the lecturing
5. some participants refused to join the experiment from the very beginning

Difficulty of tested vocabulary varies：

• The post-test is a little bit easier than the pre-test. The selected words are not actually in the same levels. We judge the levels of the words by the tool developed by Taiwan test central. (Judge by the online tool http://www.taiwantestcentral.com/Toolbox.aspx?MainCategoryID=14 )  But it’s ok since both groups receive the same pre and post-test.
• see the table, these are the words in pre and post-test.

  	level 1	level 2	level 3	level 4	level 6	others
  Pretest	0	2	1	3	1	2 ->	ammo	skydive
  Posttest	1	2	3	2	1	2 ->	oppressive	chopper

Powerful suggestions by instructors:

As an instructor, I kept encouraging them to “finish” the test even by guessing. Some of them are demotivated by the difficulty of the questions. I had to encourage them to keep going.

The numbers of question are too many:

• They are too many questions for them. (But actually over half of them could finish them in time. Some even tell me their classmates lied that they can’t finish it in time.)
• The questions must be more representative (surly!) The questions shall be more representative in order to be objective. Although the text and the questions were carefully crafted, I am still worried by their representativeness.
• We need a mechanism to force the students complete the whole test. They procrastinate all the time, making the experiment uncontrollable and less objective. A huge count down clock might be a good tool to spur them. Note: in some cases, they would not be appealed by candies. Try something else with little budget. For example, share them your research results in this experiment or tell them the immediate benefits after receiving the lecture (in a way to know themselves better or help them with a new mode of studying.)

Instability of the media we used in the experiments and lecturing:

• The projection is still too small for them to read. At least not so comfortable to read with.
• Redesign and optimized the text layout for projection. (I had done so but obviously not enough for this time)
• Try Google Answer Sheet or design an app for real-time answering (eg. hi-learning voting system for smart phones) A more efficient way of grading and collecting data (with the help of online questionnaire or apps we developed to collect data) is needed. This would allow us to collect more feedback and save time for interview after the test.

 

Finally, from the videos we found some other issues:

• the performance of the teacher is not stable enough.
• some students do not participate actively throughout the session.

The first issue is obvious and expected before I conducted the experiment. I have not been teaching students with a whole bunch class for at least six months. I got nervous at the beginning in class 109. But in the second class, the class 118, I am totally alright and got into the condition.

As for the second issue, it’s a little bit harder to resolve. This is an non-compulsory lecture. It did interfere a lot if some of them don’t participate the session. I don’t think gifts are good alternatives to them because they want more than that. The purpose of the study or some immediate boost on their ability is what they desired most. So I told them what they will learn after the session, and the benefit of it will last until they enter and leave college. This is a useful way of making them engaged at the very beginning. 

Video recordings of the lecture:

• Controlled group Class 109: (traditional)  ways of lecturing

 

pretest, posttest, and m-choices

 

• Experimental: Class 118 : “Gestalt” way of lecturing 1: former prisoner Tom’s story

“Gestalt” way of lecturing 2: the soldiers

pretest, posttest, and m-choices 

(You may find all of them on my youtube channel. I only placed some representative clips here.)

Audio Recordings:

From the audio recording, you may found more details in the interaction between teacher and the students. You may also figure out why it’s difficult to bring new vocabulary to students in a short period of time in the traditional way of teaching textbook.

As for the audio file provided below, Ogg format is an open source and better quality alternative for wma, mp3, or aac file. Play it with any of your audio programs on your computer.

Control (Class:109)

http://kmimayo.no-ip.org/pluginfile.php/109/mod_forum/post/11/Record_0001%2864kbps%29.ogg

Experimental (Class:118 )

We forgot to do the audio recording, but we had more complete video recording for class 118. Check the video recordings to see their interaction and engagement.

 

Data Analysis:

We found the difference between the the classes is not as huge as expected. I had made some huge mistakes in the initial analysis of the old diagrams. I had revised them. The mistakes shall be noted and never happen again.

1. The second diagram shows stacked data. (superposition), but the first does not.
2. Grid lines shall be provided at the very beginning.
3. Nodes shall be provided at the very beginning.
4. Notice that the horizontal line starts from point “zero” – when participants score zero in the pre/post-test, and some of them did score zero.

The first mistake is the most serious one. I therefore judged the whole situation wrongly and considered the experimental group receives dominating success over the controlled. But if you look carefully into the following diagrams and pie charts, you will find the statics only shows they are classes with different properties. Further analysis will be done in the following discussion. We still obtain some precious findings in the experiment. ( If you’d like to read you yourself, please check the the statics from the online document. https://docs.google.com/spreadsheets/d/1IB6dS-zf9cTmOBcEk8MqmJxt7GOWBKEtJFezeY54Wq8/edit?usp=sharing )

Colored examples:

• red: pre-test
• orange: post-test
• blue: m-choices

 

Further analysis on the statics:

Basically we measure their improvement by comparing the pre-test and the post-test. From what we saw in the diagram, we found both groups have salient improvement in the immediate post-test. The problem is which group has better performance? Since they have different proficiency at the beginning, and the initial components of the participants in two classes are not homogenous, we have to compare the actual improvement by the following standards.

• Average improvement
• Proportion of proficiency level after instruction (%)
• Interviewing after the tutoring (not completed in this test)
• students’ engagement (accompanied with the videos provided)

By looking into these questions, we would have a more objective observation on the statics. The following two pie charts are class 109 and class 118.

 

 

 

I listed out my findings based on the sequence of previous questions.

• Q: What is the average improvement in both groups?
• A: Both classes have their majority move up to level 4 (scoring 4) in the post-test

Pre-test

scoring	0	1	2	3	4	5	6	7	8	9	10
109 (con)	2 (6.1%)	10 (30.3%)	7 (21.2%)	6 (18.2%)	5 (15.2%)	1 (3%)	0	2 (6.1%)	0	0	0
118 (exp)	2 (5.88%)	7 (20.6%)	8 (23.5%)	7 (18.2%)	6 (14.7%)	2 (5.88%)	2 (5.88%)	1 (2.94%)	0	1 (2.94%)	0

Post-test

scoring	0	1	2	3	4	5	6	7	8	9	10
109 (con)	0	0	2 (6.1%)	4 (12.1%)	14 (42.4%)	7 (21.2%)	3 (9.1%)	2 (6.1%)	0	0	0
118 (exp)	0	0	5 (15.1%)	5 (15.1%)	13 (39.4%)	7 (21.2%)	3 (9.1%)	0	0	0	0

 

• Q: What’s the percentage of proficiency level after instruction (%) ?
• A: After the instruction, experimental group has become less diverse in proficiency than the controlled. See the tables.

From the tables above we could clearly find class 118 (exp. group) had a more approximate level of improvement. If we look for more details from the pre-test table, we would figure out the reason: class 118 has a relatively more smooth performance on proficiency test at the very beginning. But even this is the fact, class 118 has a wider span of proficiency level in the pretest. The result of this experiment only reveals the Gestalt way of teaching vocabulary can possibly make a group more homogenous in proficiency improvement. This would be a useful property in teaching.

I assume the reason for this phenomenon is simple: they are not used to the new method yet, so they perform relatively approximate in the post-test. In the traditional way of teaching new vocabulary, some are good at fetching information from the oral lecturing of the text. This may be the reason why they are more familiar in using the old method to memorize new vocabulary. But some students with lower proficiency would not obtain better results way since they had not

 

• Q: Is there any interviewing after the tutoring?
• A: No. We didn’t have time for this. Although we had planned to do so in the beginning.

We surly didn’t have time for interviewing, but we will return the findings to the participants and ask for any further interest in helping our future investigation. We have at least two running web forums available for the participants to contact us. They are:  http://kmimayo.no-ip.org/  and https://www.facebook.com/windabc 

I also informed them my email and contact information on the blackboard before I left. These will serve as reliable channels for contacting and following the participants in near future for longitudinal research.

• Q: How is students’ engagement in class?
• A: It’s far better in the experimental group.

From the recordings we provided, the experimental group did have better engagement during the lecturing. It’s easier for them to picture the scenarios in mind if they have some comic to look at – at least a basic framework available for them to develop from. This is the core of this experiment, we found that even the smartest students benefit from a new way of learning vocabulary. Incorporating new words into existing schemata and a vivid, meaningful story.

Findings and conclusions:

We have come to the conclusion that students would be better focused and interested in the Gestalt way of teaching. They would have relatively closer improvement, and this make next teaching session more pleasant because they will be easier to teach. It seems that traditional method only benefits students with proficiency from intermediate to high. But the Gestalt way of teaching avoids the problem of phonetics and morphology, providing them with an alternative to start from. I believe this would be a useful tool for learners in all levels of proficiency.  

 

Some Final remarks: budget is everything essential

From the experiment, we realize why beget is everything essential even before you start an experiment. We need some necessary equipments and budget for the preparation. But as a regular graduate student, our instructors often failed to empathize how poor we are in real life. We have very limited budget for living, yet we still need to pay for everything the experiment needed. It is not that simple just handing out the Tax numbers to the stores, we need the money before we purchase anything!

It’s hard for instructors to imagine why his advisees need to work part-time and got exhausted. Because we are so poor that we even need to choose the shortest yet most dangerous path for commuting to save the fuel consumption every week! It’s just about 150 NTD per week but I can’t afford more than that. We risk our life to ride scooters passing the busy roads and bridges, facing threat everyday just to complete our academic pursuing. Even though we had support from our family, we still need at least 8000 – 10000 NTD per month to survive in Taipei. Otherwise we could not possibly finish each semester. Any AT who work for university receive about 300 NTD per month. But the salary is not delivered by month, but by semester! This would be an impossible scenario for anyone to live in Taipei without any extra financial support.

Many of our predecessors sneer at our hustle life style, but they probably never understand how difficult it is to live a life in such a consumption level nowadays. We graduate students are forced to work at the bottom of lowest entry part-time jobs in order to survive. This is the life of a regular graduate student. Do not be misled that they drive a minivan to school or flying abroad whenever they want to join a conference! They can not possibly afford it. If you find some live such a life, you have to doubt where his money come from.

As I had said, this is probably the most important findings in this experiment: we graduate students must take it as a priority that finishing our research topic systematically, and economically surviving in real life before and after graduation.  Only by following these guidelines can we guarantee that we would be an independent individual in the future, who can work and live individually without any interference from higher authority or financial reasons.

None of us shall take it for granted such a dependant life would last forever or throughout our academic life. We shall not mess up our health, friendship or relationship with our family since it is certainly not worth it. During the process of pursuing our goal of independence, some may feel unsatisfied to the way we act. But do remember: we could not make each satisfied all the time, yet it would be enough to make some of them satisfied at the needed time.

Minimal requirement on needed equipments:

• Tripod for DSLR
• A workable DSLR or camera for video recording
• audio recording device (smart phone could be an available alternative)
• head-wearing, reliable microphones for lecturing
• A lot of A4 papers for uniform test papers.
• a over head projector and reliable big screen

References: 

1. Sabeti, S. (2012). Reading graphic novels in school: texts, contexts and the interpretive work of critical reading. Pedagogy, Culture & Society, 20(2), 191-210.
2. https://www.coursera.org/course/learning (learning how to learn; chunking skill; illusions of learning)

 

Reading graphic novels in school: texts, contexts and the interpretive work of critical reading
Shari Sabeti*
School of Education, University of Stirling, Stirling, UK
This paper uses the example of an extra-curricular Graphic Novel Reading Group in order to explore the institutional critical reading practices that take place in English classrooms in the senior years of secondary school. Drawing on Stanley Fish’s theory of interpretive communities, it questions the restrictive interpretive strategies applied to literary texts in curriculum English. By looking closely at the interpre- tive strategies pupils apply to a different kind of text (graphic novels) in an alternative context (an extra-curricular space), the paper suggests that there may be other ways of engaging with text that pupils find less alienating, more pleasurable and less reminiscent of ‘work’.
Keywords: graphic novels; critical reading; interpretive communities; extra-curricular space; secondary English teaching