Up An evaluation of Learn to speak English
Submitted by Ezzelden Ibrahim. Mohammed
For the module EEL880
February 2003
Learn to speak English, produced by Learning company is designed to improve English skill: listening, reading, writing and speaking. Learn to speak English also help you to speak everyday English words and phrases, in addition to that through it we can evaluate our pronunciation by speak recognition technology that it provide, moreover we can record our voice and compare it with an American speakers consequently our accent will soon improve, not only these, but also supposed to develop English grammar. This program contains pronunciation, vocabulary, communication skills and games. According to Alessi and Trollip (2001) This is drill program, because it follows blew procedure: first an item is selected, then the item displayed after that the learner response and the program judge response, finally the learner receive feedback about response (p182). In order to run this program we require: Pc and speaker and microphone DVD Room Drive files version of this Drive5.00.2165.
Files of this Drive are:
C://winnt/system32/Drives cd rom .sys.
C://winnt/system32Drives/Red book.sys.
C://winnt/system32/stoprop.dl.
The program was tested on 40 students from Sudan University of science &Technology 24femal and 16 male, from different states. The evaluation includes 5-Engliush teachers specialized in speaking skill.
There are many questions that asked by the evaluator before the evaluation takes place:
Is this program going to be easy for the students to follow?
Is it going to improve student’s performance in speaking?
Is it suitable for the Sudanese and not away from their cultural background?
To what extent does the program help in teaching speaking?
It was found that the program was cosmetically good at the animation, graphics and screen space but need to enlarge the font size in order to make text clear enough for the users particularly for those who have eye ache.
The situation in Sudan nowadays, in great need for new ways of technology in order to cope with globalization era and its discoveries that are challenges and struggle between the nations. The Sudanese people are away from the new technology, because there’re many factors that hindered them. Firstly the serious one is the fighting between the northerner and southerner that wasted their efforts, money and youth and all of these elements are important of development and prosperity. As the result of this war we didn’t look at education and even we didn’t understood the meaning of civilization .Our attention is how to find weapons for fighting, but the real one (knowledge) which is develop day after day was neglected. Although we have creative people who established books and references we didn’t take care of them as the result of previous Era . As we know that, Sudan is the biggest country in Africa, contains different tribes with different cultures, colors and believes that is second reason leads to hinder the Sudanese from the development. As it was pre mention the civil war wasted most of our income, which it supposed to direct to education and its needs that to facilitate education process with its all sides and support and encourage: students, teacher's educators and scientists to work cooperatively in order to produce curriculums that assist to create dependent generations.
In fact, we are in different situation, attitude and needs, that because we are from different tribes resident in different areas and each has its own request this is clearly appear when we want to build curriculums that cover all those people with such, characteristic, forexamle we can not be able to draw a picture in this curriculums that satisfy with its indication, color and shape needs of those people and even the use of names in examples is matter of difference in the methods for example if we use names associated to northerners like Ali ,Taha, Sami,the Southern complained why those names are chosen instead of those like John, Gebrial…etc they see there is bias. The Sudanese people look to the marginal thing instead of looking to the main one, like information technology revolution and its good effect on the development of the world that we are apart from it. Really we are in great need for this program, if it didn’t yield what was intended to, at least it take people attention towards the subject and to look with consideration to it, not only this but also direct them to understand their situation compared with other. This program was chosen to be evaluate in order to examine is it suitable for the Sudanese with their differences? Are users going to cope with its color, graphic, animation and cultural background is it easy or difficult to use? Was the lesson logically organized and it is easy for them to follow? Are the users going to understand its content? Is it support them toward the speaking?
This program is supposed to improve English skills and also help the user to correct pronunciation and increase the vocabulary. We take this program in consideration, because it supposes to enable the users to speak English accurately and fluently that is our Sudanese need in order to express their idea and find access too many source of knowledge. This program was chosen to evaluate, because it suppose to brought both teacher and student to right way to cope with technology besides it safe their time and efforts, moreover the subject matter of this program is one of the an essential social studies” English speaking” that encourage to evaluation, because it is an important means of communication through it people exchange their ideas, experiences and it is important element of technology .The Sudan is rich by human sources how need to be invested in good way via new technology, but they still follow classical method in their schools, even in university level ,so they need awareness to this living language and make use of it in their daily life communication.
This program was chosen to evaluate with those learners, because this program was designed to certain ages, level of education .The purpose of evaluation is to check whether it is suitable for the Sudanese people? For the previous reason it need representative sample for whole people with different cultural background. The program is about learn to speak English and the sample is from English department so they are suppose to now subject area more than other, also this program suppose to support them towards speaking which is necessary for them to exchange their ideas among each other and with their teachers. In addition to that most effective way to acquire the language is speaking, also there is wisdom behind the selection of this group, since the sample from Education College, so they should be responsible from spread of it is subject to coming generation consequently, we need to educate the educator in order to produce qualified person.
According to Hannifin and Peck (1988) there are several aspects of the lesson that we must consider during evaluation. This fall into four basic categories: instructional adequacy, cosmetic adequacy, program adequacy and curriculum adequacy (p303.)
Cosmetic adequacy
Cosmetic adequacy refers, to degree to which feature like sound, color and graphics which are used in computer assisted instructional lessons to attract and retain user attention” This component refer to what extent lesson is like, focusing on sound, graphics, animation, video and screen space …etc
A according to Hannifin and Peck (1988) cosmetic adequacy can seen through the following: the screen space must used effectively, color and sound are capable to enliven the instruction so need to be systematically and for certain purpose, (p311).
Olson and Wilson, (1985)”color enhance the impact and legibility of the screen”
(Dwyer, 1978) there is evidence that color can enhance the learning.
Color can be used to attract attention.
The more color is used the less attention is will attract
Manni (1993) color is closely akin of graphic and it enhances learning and motivation. Sound especially speech is increasingly important for educational multimedia. (Alessi& Trollip (2001, p74). Bett and Wilson (1996) contradict with Hannifin and Peck “color doesn’t seem as powerful as other techniques such as animation”. Alessi Trollip (p76).Hannifin and Peck, (1988) “animation can be useful in creating motion during the games and simulation and can be very effective technique in improving cosmetic appeal of instruction (p313)According to Hannifin and Peck (1988) scrolling, caused by failure to control the cursor systematically, so scrolling must completely avoid. (P312).Olson and Wilson, (1985) there should be no need to scroll to complete reading information. Each screen should be well laid out combination of text, graphics and animation, which work together, to make particular point clear Olson and Wilson, (1985)Menus and icons according to Shnederman (1998) for the program to cosmetically adequate the menus must be simple and few in number and for global control such as choosing the next activity or obtaining help or existing the program
Text layout according to Alessi and Trollip the paragraphs should be well formats; spacing between the lines should be adjusted to make text clear and attractive. (P85)
According to Olson and Wilson (1985) for the text to be well layout the mount of information should be limit to one point plus supporting idea’
Technical adequacy:
This refers to extent which CAI lesson or any program can be easily executed by the users.
According to Hannifin and Peck (1988) the following is primary concern of evaluation of technical adequacy: there should be logic in the lesson this concern manner in which execution is consistently with the kind of lesson input, output and control prescribed besides that the lesson must perform free from loops, and it should be self starting require minimal disk handling moreover the lesson should, simplify program Protection so security must include sufficiently for both students and disk itself .the designer must ensure that an unauthorized cannot access on other individual data this can be accomplished with password and careful management of the disk also there must be good backup procedure. (Pp312-313)According to Alessi and Trollip (1991) the program should keep permanent record of learner's performance for the instructor, and to allow the learner to carry on from where he or she left of, also program record and file should be secure from tempering and accident delete especially when it operate on the network and access to any one.
Curriculum adequacy
According to Wholeben (1982) “this component refers to extent in which lesson procedure, activities and formats cope with a accepted standard”. For the program to be curriculum adequate: to extent possible the lesson must be consistent with style of both teacher and students; easily incorporated into existing curriculum activities and structure and cope with the kind of lesson activities and procedures already in place. Generally lesson should supply fairly compatible procedure for the lesson execusion both within and across related lesson.
The designs and procedures should provide possibility for expansion. Lesson protocol and conventions should permit the development of additional lessons and materials that might also utilized the same basic protocol, response, formats and other lesson features. Hannafin and Peck (1988, PP314-315)
According to Reeve (1994) program lesson should allow flexible, true to life sequence and moving through the lesson and enable the learner to use what they gain in the real life context
Pedagogical adequacy
According to Hannafin and Peck (1988) instructional adequacy refers to extent to which CAI lesson provide, the essential kind of support and features to accomplish objective at hand”. Instructional adequacy can be measure through the following: the instructional objectives are important side to consider in instructional adequacy; features and design of the lesson are examine to attribute cause to observed effects .The important features that characterize the pedagogical adequacy are that: the lesson must provides clear guidance to user; the objective must well stated; lesson activities and assessment question must be aligned also lesson must move smoothly from frame to frame and from section to section to make the learner attention effectively. The lesson should be easily to follow; the lesson should use directly and readily understood technique exit for conveying; the lesson activities and procedures must be logically and effective by reducing dead wood and moreover the lesson must emphasize the most important aspect and provide opportunity for meaningful interaction, (p303)
According to Alessi and Trollip (1991) instructional adequacy can achieve through motivation. (Pp31-33)
According to Sapair (1921) language is purely human and non-instinctive method of communicating ideas, emotions and desire by the means of voluntarily produced symbol .The word non-instinctive mean the language not born with person?
From this definition the speaking is efficient tool of communication. There are many aspects that you must consider in teaching this subject such as firstly; the student must have knowledge of phonetics that is scientific study of sounds in order to get correct pronunciation of the words that are going to produce in their communication. Secondly, to produce perfect students we must teach them how to produce dialouques and short story and give them more information about stress and intonation to emphasize of the word. The learner need to join native speaker in order to get the pronunciation, as it is, so native speaker must invite in the class and .we must take the learner to learn from situation .The learner need to know the speaking through pairs and group work and conversation in form of short story and dialogues.
Evaluation methodology
According to Trochim (1977) this is dynamic evaluation because the program evaluated while the target group, interaction between the programs, is using it and the user is one of our focusing and we do little or no help to users. According to (Gane et al, 1992) this is summative because, the evaluation was done after the company conducted the program. We concentrate to draw conclusion about how well instruction work (p338.
In addition to that we concerned with to what extent that the features of instruction support the learning? We do this to see the performance of users using this program and acceptance of instruction design. Finally we evaluate to see whether we a adopt or not?
Basically, Kirkpatrick suggest that there are four levels of evaluation at which program should be evaluated
According to Kirkpatrick (1996) level1 is concern evaluation reaction of learner to program. I used this level to know the feeling of the user about this programming to assess how much users like it? Although interpret attitudinal data cautiously, they usually provide valuable information.
Level 2 assessing learning is other important goal-validating program is to ensure that the learner learns what was intended. I used this level to know to what extent the user improved? Are they encouraged to learn English speaking through this program?
Perspective
The quantitative approach was chosen according to Huysamen (1997)That because I used the questionnaire, interview, expert review and observation and after that I used the matrix to analysis the data all these for the purpose to find evidences regardless it supports or doesn’t support an existing hypothesis
Target population
This program done on students from Sudan University of Science and Technology, faculty of Education, English department, first class their ages between 18 to 21 .The samples is representative to the Sudanese people in their age and education level and specialization, because they are from different regions with different cultural background and concepts resident in different regions in Sudan. I selected from this class, which include 60 students 40. The evaluation includes 5 English teachers specialized in speaking skill and are expert in the curriculum and closely relate to student and their actual needs.
Sample of the evaluation following this characteristic: -
The entire sample is joining the
English language department by the desire.
The entire samples are
single.
All of them obtain high grade in the
last speaking test and also from the activity that I gave to them before the
evaluation take place.
Their economic situation is different.
24 of them are female, 14 of them have
knowledge about computer. They was using it in their home, they are belong to
upper class 9 of 14 from Khartoum and 5 of14 from different states the rest 10
of 24 are from different states7of10 are from lower class and 3 of 10 are from
upper class and all the 10 have theoretical knowledge about computer.
In the sample there are 16 male 5 of
them from Khartoum 3of 5 know the computer but they only use in games and
listening to songs 2of 5 from upper class. The 11 are from different states and
7of 11 are know the computer and 4of 11 never come across the science of
computer. According to Reeves table of matrix is summary of information
required from evaluation of learn to speak English and guide to pronunciation
|
Instruments
Questions |
Expert review |
Observation |
Personal interview |
Questionnaire |
|
Is the program appearance is clear and attractive |
ü |
ü |
ü |
ü |
|
Does the lesson-executed as intended |
ü |
|
ü |
ü |
|
Is the content of the program useful to gain the objective |
ü |
|
ü |
ü |
|
Are lessons smoothly run |
ü |
|
ü |
ü |
I used observation to gather information about participants’ physical behavior this instrument help me to gathered directly information about the users while using the program and it yielded objective information
I gave to the user immediately after they finished the program and I received useful feedback from it.
I gave this program to 5 English teachers and after they investigated it carefully then I gave them questions to answer is elected those teacher because they’re expert more than other in teaching particularly in speaking skill also I can depend on their answers in my evaluation because they are suppose to authentic source.
How you are going to evaluate? The planning is important element in any evaluation
Firstly, I took the program and began to install in order to examine whether it is easy to install or not? This is the first problem that faces me in this evaluation. Prof, Iskender helped me to overcome it.
I began to think about the sample that I’m going to evaluate with them this program. The content of program direct me to choose the first English class and from them Select representative sample.40 students
Sudan University of science and technology associated with university of Pretoria (CIE) and explained to them the objective of this study and ensure to them that their information will be in top secret. Gave them open test in order to know their level and through this asked them about personal information in order to know their address.
Operated the program and l explain to the users the contexts the and how to use this program then I gave them chance to arrange them self according to their knowledge of computer and divided them into group of 6 students 1 computer.
Each 6students spent 1hour. While they used the program they asked each other and when they didn’t find answer they return to me to guide them and Idol and I advised them to use help icon (?) At that time my classmates Atoka and Rayne observed what the users did
In conclusion, I gave them questionnaire to fill and they do with little help then I collected their answers and after short time they feel relax Introduce them in personal interview.
Validity and reliability
According to Gay, (1987) Validity refers to the degree to which the evidence supports that these interpretations are correct and the manner in which the interpretations are used is appropriate.
Repeat the program with same group and I gain the same result; what they comment about in the program is not change. I found they feel more about the program content Reliability refers, to consistency of assessment scores
Figure Questionnaire response

In this section the findings will be discussed in the sequence in which they answer the research questions. The most important results will be presented first, followed by other results that support or contradict the findings.
The instrument that yielded the best information to answer this question was the questionnaire The findings are backed up by the following instruments, that yielded the following 80%of users answers showed that the program is clear and sound and text laid out and animation are clear 20% of users answer showed that there is defect in text. The personal interview support questionnaire and most of the users opinions about the color and animation and sound support the questionnaire The findings were contradicted by the following information from both expert reports observation during my observation Observed that, the text is not clear enough and there is problem in the color distribution It would seem then that there is really problem in text particularly in font size and color need more control.
Are lessons executed as indented
The instrument that yielded the best information to answer this question was expert review, which yielded the following information that there is difficult in installation and the use of icon misleads the users. The exit icon is hidden and so it cause difficult to the user and also there is no security for the user uses confused and did not know the icon that sing them out the lesson and when they press (?) help icon it did not support them with instruction or any assistance The findings were contradicted by the following information that yielded from the 75% of the users answer showed that the questionnaire which yielded that the icons are helpful and it is easy to enter the program without user name that because they don’t know the purpose of the security f the program and they look only for the surficial thing It would seem then that this program need security and good use of icons and solution to installing problem.
The instrument that yielded the best information to answer this question was expert review yielded the following. That according to their long experiences in teaching speaking skill and their closed relation to both curriculum and students the content of program is useful to students and it suppose to help them to speak English but it need to be provide with more examples and vocabulary section need to be order in related subject in addition to that it is very nice if it should provide with emphasize to important aspect. Users support this idea, yielded that this program is useful and motivate them to carry on in learning English and they asked to copy this program to take it home if it is allowable that mean they are interested by this program
Observed that the user really began to accept the subject immediately after they finish the program in personal interview they attempt to answer in English moreover in the next watching of the program Noticed that they are really improved. It would seem then that the findings are backed up by the following instruments that yielded the following information.
The findings were contradicted by the following information.
It would seem then that.
Are lessons run smoothly?
The instrument that yielded the best information to answer this question was questionnaire 75%of the users answers showed that the program is clear and easly to them to move from frame to frame.
25% showed that there is defect and it is not easy to move from section to section.
This idea is supported by all other instruments it would seem then that, the expert have seen there is only need to be emphasized of the important aspects.
When the above findings are considered the following conclusions can be made.
Cosmetically
According to Hannafin and Peck (1988) cosmetically adequate the program should characterized by the following: screen space must use effectively, color and sound are capable to enliven the structure so need systematic and for certain purpose (p311)
Olson and Wilson (1985) each screen should be well layout combination of text graphic and animation which work together to make particular point clear compare this with the finding I concluded that this program is enjoyed with clear sound, screen space, graphics and animation
What exactly need is to look carefully at text lay out (font size and color distribution.)
According to Hannafin and Peck (1988 the following are the primary concern in technical adequacy
There should be logic in lesson this concern the manner in which execusion is consistent with the kind of the lesson input, out put and control prescribed compare this with the finding it can be concluded that there is difficult installation moreover the use of icons made lesson difficult execute and there is no security to the users…
According to Wholeben (1982) curriculum adequacy refers to extent in which lesson procedure, activity and format cope with accepted standard for the program to curriculum adequate to extent possible the lesson must be consistent with style of both teacher and learners and easily incorporated into existing curriculum activities and structure and cope with the kind of lesson activities and procedure already in place, Hannafin and Peck (1988, pp314/315) compare this with the finding it can be concluded that the content of the program is useful to the learner and motivated them to speak English, but the serious conclusion is that there is cultural bias in this program and most of it example is away from Sudanese environment and the vocabulary is not order according to related topics…
According to Hannafin and Peck (1988) for the program to pedagogical adequate the lesson activities and procedures must be logically and effective by reducing the dead wood and emphasize the most important aspect and provide opportunity for meaning full interaction compare this with the finding it can be concluded the program need more emphasize to important
Recommendations
Based the table below shows the recommendations that arise from the conclusions above.
From the table we can see that the most important conclusions relate to
Conclusion |
Recommendation |
|
Cosmetically |
|
|
Technically |
|
|
Curriculum |
… |
|
Pedagogically |
|
From the above tale it can be seen that the most important recommendations relate to curriculum adequacy…
Alessi, S.M
&Trollip, SR. (2001). Multimedia for learning: Methods and development.3rd
edition. United Statesof America.pp
.
Alessi, S.M&
Trollip, SR. (1991) computer based instructions: methods and development.
available online
http://hagar.up.ac.za/catts/learner/
(Dwyer,
1978) available online
http://hagar.up.ac.za/catts/learner
Gane
et al, (1992) available online
http://hagar.up.ac.za/catts/learner/1999kagarimetsa-rj…/assign1.ht
Gay,
(1987) available
…Scoring rubric deve online lopment valid reliability_moskal, BarbaraM&A_leyd Rational equivalence reliability
Gay&Airasian,
(1999). available onli
http://www.sil.org/lingualinks/literacy/refrencematerial
Hannafin,
M.J., & Peck, K. (1988). The design, development and evaluation of
educational software. New York, NY: Macmillan.pp (Kirkpatrick,
Huysamen,
(1997) available online
http://www.gsu.edu/~mstswh/course/it7000/papers/the1.htm
Olson,
S &Wilson, D. (1985) available online
http://hagar.up.ac.za//catts/learner/eel/conc/conceot.htm
Reeves,
(1994)
http://hagar.up.ac.za//catts/learner/eel/conc/conceot.htm
Reeves (“EvaluationTechnology-based instruction”,) pp15-20 available online
http://www.und.ac.za/users/clarke/kzb/emat2.htm
Sapir, (1982) Language & Linguistic, John Lyon (1993)-British University Press
Trochim,
W.M.K. (1977) available
http://(Trochim.human.cornell.edu/kb/interval.htm
Wholeben,
(1982) available online
http://www.sidst.sk/science/evaluation/
Mohammed Ibrahim Ezzelden
Name of Program: -Learn to Speak English.
|
Introduction provides clear program information. |
1 |
|
|
|
|
|
Introduction describes program. |
1 |
|
|
|
|
|
Introduction describes evaluation setting. |
1 |
2 |
|
|
|
|
Introduction contains important findings & conclusions. |
1 |
2 |
|
|
|
|
Background explains why this program was evaluated. |
1 |
2 |
|
|
|
|
Background adequately describes evaluation context. |
1 |
2 |
3 |
|
|
|
Background provides rationale for evaluation. |
1 |
2 |
3 |
|
|
|
More than 10 literature sources consulted. |
1 |
2 |
3 |
|
|
|
Literature review integrated. Not just listed. |
1 |
2 |
3 |
4 |
5 |
|
Literature review logically arranged in sub-sections. |
1 |
2 |
3 |
4 |
5 |
|
Supporting and contradicting literature cited. |
1 |
2 |
3 |
4 |
5 |
|
References correctly cited, acknowledged and quoted. |
1 |
2 |
|
|
|
|
Quotes in context of argument and other quotes |
1 |
2 |
3 |
4 |
5 |
|
Evaluation methodology adequately defended. |
1 |
2 |
3 |
|
|
|
Research perspective clearly stated. |
1 |
2 |
3 |
|
|
|
Methodology appropriate to research perspective. |
1 |
2 |
3 |
4 |
5 |
|
Target population adequately described. |
1 |
2 |
3 |
|
|
|
Sampling adequately described. |
1 |
2 |
3 |
|
|
|
Sample adequately defended. |
1 |
2 |
3 |
|
|
|
Research matrix provides for triangulation. |
1 |
2 |
3 |
4 |
5 |
|
Matrix shows effective and efficient use of instruments. |
1 |
2 |
3 |
4 |
|
|
Instruments clearly described. |
1 |
2 |
3 |
4 |
5 |
|
Instruments clearly defended. |
1 |
2 |
3 |
4 |
5 |
|
Instruments appropriate for required results. |
1 |
2 |
3 |
4 |
5 |
|
Instruments match research perspective & methodology. |
1 |
2 |
3 |
|
|
|
Method describes pre-evaluation activities. |
1 |
2 |
3 |
|
|
|
Method describes evaluator activities. |
1 |
2 |
3 |
|
|
|
Method describes subject activities. |
1 |
2 |
3 |
|
|
|
Method describes activities after treatment. |
1 |
2 |
3 |
|
|
|
Method likely to ensure validity and reliability. |
1 |
2 |
3 |
|
|
|
Findings organized according to research questions. |
1 |
2 |
|
|
|
|
Findings arranged from most useful instrument to least. |
1 |
2 |
3 |
4 |
|
|
Contradictory findings reported. |
1 |
2 |
3 |
4 |
|
|
Tentative explanation of findings. |
1 |
2 |
3 |
4 |
|
|
Findings reported in sufficient detail. |
1 |
2 |
3 |
4 |
5 |
|
Sufficient number of findings given |
1 |
2 |
3 |
4 |
5 |
|
Conclusions based on integration of findings. |
1 |
2 |
3 |
4 |
5 |
|
Findings adequately integrated into conclusions. |
1 |
2 |
3 |
4 |
5 |
|
Conclusions more than just summary of findings. |
1 |
2 |
3 |
4 |
5 |
|
Conclusions refer back to relevant literature. |
1 |
2 |
3 |
4 |
5 |
|
Recommendations based on conclusions. |
1 |
2 |
3 |
4 |
5 |
|
Recommendations practical and useful. |
1 |
2 |
3 |
4 |
5 |
|
Figures clearly labeled. |
1 |
2 |
3 |
|
|
|
Figures introduced by a full explanatory sentence. |
1 |
2 |
3 |
|
|
|
Figures followed by a full contextual sing sentence. |
1 |
2 |
3 |
|
|
|
Tables introduced by a full explanatory sentence. |
1 |
2 |
3 |
|
|
|
Tables followed by a full contextual sing sentence. |
1 |
2 |
3 |
|
|
|
Every reference listed in bibliography. |
1 |
2 |
3 |
4 |
|
|
Bibliography correct in every way. |
1 |
2 |
3 |
4 |
|
|
Every item in bibliography also in text |
|
2 |
|
|
|
|
Text completely free of grammatical error |
1 |
2 |
3 |
4 |
5 |
|
Level of language appropriate to academic writing |
1 |
2 |
3 |
4 |
5 |
|
Style appropriate for academic writing |
1 |
2 |
3 |
4 |
5 |
|
General impression and layout |
1 |
2 |
3 |
4 |
5 |
|
|
|
|
|
|
|
|
|
3 |
36 |
39 |
60 |
25 |