An Investigation of the Problems that Young Learners of  English Have Using Bilingual Dictionaries

 Peter Beech

submitted in partial completion of the MSc degree in Teaching English, Aston University  

November 1997 

 Abstract 

This project examines the question of whether young Greek learners can  find a correct translation equivalent for words they look up in their English-Greek dictionaries. Several different dictionaries were used and it was expected that variations in the quality of dictionaries would be a significant factor. We therefore focus on the features that distinguish the Oxford English-Greek Learner’s Dictionary, which was used for the majority of the look-ups,  from its competitors. 

Of the 718 look-ups, 92 failed to produce an adequate translation. The most significant cause of failure, accounting for  54 cases, is that students reported finding a translation which is equivalent to some sense of the headword, but not the sense which was required in their context. Particular difficulty was experienced with derivatives, which, together with compounds, account for a significant proportion of the failures to locate an entry. 

Apart from the 92 look-ups which were judged to be unsuccessful, many more produced translations which were inadequate because they failed to convey the sense of the word with sufficient accuracy to permit discrimination between near synonyms. The OEGLD was better than its rivals in that respect, as it provides a large number of examples to support the translations. 

Learner training is necessary to overcome the tendency to select the wrong translation when several senses of a polysemous word are translated in one entry. Many such errors could also be overcome by improved dictionary design, particularly the integration of features from monolingual and bilingual dictionaries, which could best be achieved in electronic form.

“The value of a work must be estimated by its use: it is not enough that a dictionary delights the critic, unless at the same time it instructs the learner.” 

(Samuel Johnson, quoted in Carter (1987): 125)

Contents

Chapter One - Introduction

           1.1 Monolingual versus bilingual dictionaries                              

           1.2 Review of previous research                                                

           1.3 Description of the survey

                    1.3a Method of data collection                                          

                    1.3b Dictionaries used in the project                                  

Chapter Two - Results of the Survey  

           2.1 Introduction to the results                                                   

           2.2 Successful searches                                                           

           2.3 Categorisation and analysis of unsuccessful look-ups      

                      2.3a The word was not located 

                             2.3ai The word is not contained in the dictionary          

                     2.3aii The word is contained in the dictionary but the student failed to locate it

                       2.3b The word was located but in a wrong meaning 

                             2.3bi Because the dictionary translation is inaccurate                                                      

                              2.3bii The student misread the translation                  

                              2.3biii The wrong sense of a polysemous word was found                                                     

                       2.3c The word was located with a basically correct sense but the translation was not
                                        entirely satisfactory                                                       

         2.4 Categorisation and analysis of inadequate translations        

Chapter Three - Proposals Based on the Results

           3.1 Learner training                                                                     

           3.2 Bridging the gulf                                                                     

           3.3 Future dictionaries                                                                   

Chapter Four - Conclusion                                                                    

 References                                                                                           

Appendix One - Survey Form                                                                   

Appendix Two - Complete List of the 718 Look-ups                                

Appendix Three - Description of the Dictionaries Used 

          A3.1 The selection of headwords                                                 

           A3.2 The treatment of entries                                                      

          A3.3 List of headwords in FR-                                                    

Chapter One - Introduction

1.1: Monolingual versus bilingual dictionaries

It is generally believed that while bilingual dictionaries may be useful in the early stages, more advanced learners should use monolingual dictionaries. As cited in Carter (1987: 126), “Baxter (1980) concludes that prolonged dependency on bilingual dictionaries probably tends to retard the development of second language proficiency...”. The main reason for the bias against the bilingual dictionary is that it reinforces the belief that for each word in the L1 there is an equivalent in the L2, and vice versa. An additional reason is that the learner browsing through a monolingual dictionary will benefit from the incidental exposure to the target language. However, as Béjoint & Moulin testify (1985: 3), “The superiority of the monolingual over the bilingual is not as obvious as many of us would think or say. According to Ellegard (1978: 240-241), the main advantage of the monolingual dictionary is that, as it is commercially more profitable, it can offer more for the same price. It does indeed offer more information on syntax, according to recent research by T. Herbst. It also obviously includes more words in the foreign language (all things being equal). Apart from that, the only difference is that for each lexical unit the monolingual dictionary gives a definition while the bilingual dictionary gives equivalents.”   

Thus their relative usefulness depends on how accurate and comprehensible the monolingual’s definition is, and on how closely the  bilingual’s equivalents equate. As Scholfield (undated: 85) points out, “there is no reason in principle why English-L1 BDs [bilingual dictionaries] should not be as good as good monolingual English dictionaries in the information they contain”. While it is  generally assumed that the definitions in monolingual dictionaries are at least as accurate as the translations in their bilingual counterparts, it is not certain that they are understood. As Carter (1987: 127) remarks, even in the case of monolingual learner’s dictionaries with restricted defining vocabularies, there is no guarantee that the words used will be known by the learner. On the other hand, whatever the limitations of the bilingual dictionary, at least the learner knows the words on the right hand side. The question remains whether the bilingual succeeds in producing translations that are satisfactory. This project sets out to examine that question by surveying our learners’ use of several English-Greek dictionaries.

1.2: Review of previous research

There have been many surveys collecting data about what kinds of information students look for in dictionaries, ownership of and attitudes towards dictionaries, coverage of lexis from various registers, the effect of dictionary use on performance, and the “sociology of dictionary look-up situations” (Knowles & Roe 1995-96: unit 7, pages 11-19). Research has also been undertaken into dictionary users’ reference skills and dictionary consultation heuristics (op. cit., pages 19-28). One finding of previous research, whether with monolingual or bilingual dictionaries, is that users are primarily interested in meaning, and secondarily in syntactic information and synonyms, with much less interest being expressed in the other kinds of information a dictionary may provide (Jackson 1988: 197).  

Given the dissatisfaction with definitions described in Quirk’s 1973 study of native speakers (reported in Béjoint 1994: 143), it is questionable whether monolingual learner’s dictionaries are successful in conveying meaning. Béjoint concludes from his own 1981 study (op. cit.: 147) that as students are not interested in the additional kinds of information provided in learner’s dictionaries, native speaker dictionaries could serve them almost as well, but this overlooks the fact that learner’s dictionaries differ not only in offering additional information, but also in the style of the definitions. Whether definitions  of any kind are the best method of showing meaning is unclear, as the results of various studies conflict (Béjoint 1994: 165). In particular, there is no conclusive evidence for the purported superiority of monolingual over bilingual dictionaries.

Atkins & Knowles (1988) set out three main aims for their study: “We hoped to find out something about how effective dictionaries are in carrying out various operations...; whether bilingual and monolingual dictionaries are equally effective aids.... We also hoped to find out something about how dictionaries fail students...” They note (p.385) that any conclusion regarding the correlation between monolingual dictionaries and high achievement and between bilingual dictionaries and low achievement “would almost certainly depend on the quality of the particular dictionaries used; it seems probable that the distinction between a pocket dictionary offering single-word translation equivalents and a more serious bilingual dictionary would be more significant than the rather crude distinction between all monolinguals on the one hand and all bilinguals on the other”. 

The poor quality of bilinguals is evidenced by Atkins & Varantola’s  description (1997: 28-29) of the search for an appropriate translation for APUVALINE, which is given as INSTRUMENT, MEDIUM, VEHICLE, FACILITIES: “The four English words are by no means even partially synonymous.... They are certainly different enough to support some semantic differentiation.... Probably the most useful way of giving that type of information is in example sentences.” 

Indeed the corresponding entry of the Greek-English volume of  the Oxford English-Greek Learner’s Dictionary has one sense given as MEDIUM, VEHICLE, MEANS, WAY supported by thirteen examples. Cowie (1989: 55) points out that examples have “two major functions - that of clarifying a sense and that of distinguishing between related senses.” In order for both of those functions to be fulfilled adequately it  may well be necessary to have a large number of examples, and on that criterion the OEGLD scores highly.

However good the dictionary is, the learners must have the skill to use it effectively. As Atkins & Varantola state (1997:1), “There are two direct routes to more effective dictionary use: the first is to radically improve the dictionary: the second is to radically improve the users. If we are to do either of these things - and obviously we should try to do both - the sine qua non of any action is a very detailed knowledge of how people use dictionaries at present.” But while several of the findings of the above studies provide “detailed knowledge of how people use dictionaries at present”, it seems that no previous studies have looked at the results of the dictionary lookups that students naturally perform in their own learning situation using their own dictionary. In the present project, the range of dictionaries used was relatively small, and there was  a clear preference for the OEGLD, which meant it was possible to conduct a detailed study of the dictionary itself in parallel with the processing of the students’ data. It was felt that the  knowledge  acquired would be most useful if it pertained to the natural situation of the learners, rather than an artificial selection of items to look up, with the proviso that there should be a sufficient quantity of data to allow significant patterns to emerge. However, unlike the participants in Atkins & Varantola’s study, the learners in the present project were not asked to rate their own satisfaction with the result of the lookup, as that was considered to be an unreliable indication (cf. Béjoint 1994: 147).The methods by which the data were collected and analysed form the topic of the following section. 

1.3: Description of the survey

           1.3a: Method of data collection

In contrast to the studies described in the previous section, the intention in the present project was to investigate the degree of success with which students used dictionaries during their learning activities. This meant that, firstly, each student would use whatever dictionary he used habitually, and in fact one participant used two different dictionaries during the period in which he was supplying data. Secondly, it meant that students were not prompted to look up certain words, or given specially-designed tasks to do, but were simply asked to record the dictionary look-ups which they made during the normal course of their  language learning activities.

The students were asked to complete a survey form, which is reproduced in Appendix One, each time they did an activity for which they used a dictionary. They provided a reference such as book title and page number on each form so that we could trace the original context in which they had encountered the word. They also recorded which dictionary they had used for the look-ups. They were told that they should record each word that they looked up in the form in which they had encountered it, and the translation which they had found from their dictionary that corresponded to the sense of the word as used in that context. On some occasions they reported  not finding  the word, or not finding any suitable translation for it, as will be discussed in section 2.3a. Occasionally, students recorded more than one translation. A total of  718 valid lookups were recorded, and these are listed in Appendix Two.

During the period of the data collection, a data base was constructed using dBase III Plus for the storage and manipulation of the data. The fields in the data base were: 

Name; Date; Class;

Dictionary used;

Type of task;

Word sought; Word found; Result;

Number of  meanings; Rank sought; Rank found;

Reason for failure. 

The Result  field in the data base was a logical field which stored a binary evaluation of the search as a success or a failure. The evaluation depended solely on whether the translation which the student had found in the dictionary was appropriate in the context in which the word had been encountered. In order to establish this, for each look-up reported the researcher examined both the original context and the entry for the word in whatever dictionary the student had been using. In the case of polysemous words the number of senses given for the word was also recorded, along with the number of the sense of the translation selected, and the number of the sense that should have been selected. 

As we will see in section 2.3b.iii, by far the most common reason for failing to produce a correct translation was that students selected the wrong word from amongst the various translations of polysemous words. In the majority of such cases, they noted the first translation given without regard to its appropriacy in the context. The last field in the data base provided space (a maximum of 240 characters) for a preliminary evaluation of  the reason for the failure of those look-ups that did not produce a correct result. The use of dBase III Plus allowed the manipulation of all the data to produce reports like the one that constitutes Appendix Two, or of  parts of the data, so that look-ups could be arranged not just in alphabetical order of the word looked up, but also according to any of the other variables specified as fields. 

The students who supplied the data were studying General English as a Foreign Language at levels between Pre-intermediate and the level of the Cambridge Certificate of Proficiency in English, and were aged between eleven and seventeen. A breakdown of the proportion of successful look-ups made by each student, and also by each class of students is provided in section 2.1. The initials of  each student as well as the class he belonged to are also provided in the full list of look-ups in Appendix Two. It should be noted that in that appendix the OEGLD is referred to by the initials of its editors (SH). 

An indication of the types of tasks that these learners were working on can be gained from section two of the references, which lists the books that they used. The coursebooks used were Generation 2000, level 3, Reward Intermediate, First Certificate Passkey, Focus on First Certificate, Focus on Advanced English, Proficiency Masterclass, and a series of grammar books. There was also a book of passages for translation used by one individual who was preparing for university entrance examinations, together with past examination papers for these examinations as well as the Cambridge Certificate of Proficiency in English. Look-ups were recorded of words encountered in extensive reading as well as intensive study. As well as the breakdown of searches according to individual student and class, the proportion of successful look-ups according to task type is also analysed in section 2.1. The final section of Chapter One is a brief description of the dictionaries used in the project, an area that is covered more extensively in Appendix Three.   

1.3b: Dictionaries used in the project

As can be seen in section 2.1, a total of eight different dictionaries were used in the survey. By far the most popular was the Oxford English-Greek Learners Dictionary, which is based on the Oxford Advanced Learner’s Dictionary. This was used for 449 of the 718 look-ups reported. Throughout the present analysis we will be giving special emphasis to this dictionary, both because it is the most popular amongst our students and because we believe that we can make a useful distinction of a category of hybrid dictionary which combines features of the monolingual and the bilingual. 

The dictionaries ranged in size from the Collins Gem, with 345 small pages to the Penguin-Hellenews with 926 pages. The OEGLD, with 839 pages, was amongst the largest. It contains 31,000 headwords, as against 63,000 in the OALD. About a quarter of this reduction is achieved through economy in the treatment of compounds and derivatives, the rest through omission of less common words. In general, its entries are also briefer, often because it distinguishes fewer senses of the headword, and also because it provides less information on usage. As in any bilingual dictionary, brevity is also promoted through the provision of translation equivalents rather than definitions. The average length per entry in the OEGLD is therefore about one third less than in the OALD, but this is still more than twice the length of any of the other bilinguals used in this project. A much more detailed examination of the dictionaries used in the survey, again with particular emphasis on the OEGLD, may be found in Appendix Three. 

Chapter Two - Results of the Survey

2.1: Introduction to the results

For each word that the students reported having looked up, the original context in which they had encountered it was located so that the sense in which the word was used could be ascertained, and it could be judged whether the word found was an acceptable translation. Where necessary, a monolingual Greek dictionary was used to aid that judgement. In those cases where the translation recorded by the student was judged to be incorrect, the dictionary entry was scrutinised to discover the reason for the failure. The explanation of the failure was then added to the data base which had been designed for the storage and manipulation of the data reported on the survey forms.  

As will be seen in section 2.3, the production of incorrect translations was sometimes due to error on the part of the student, sometimes due to deficiencies in the dictionary. In all cases the evaluation of correctness depended on the demands of the context in which the student encountered the lexical item and which inspired the look-up. Where the dictionary translation succeeded in producing a correct understanding of the meaning in the text, the lookup was judged to be successful. Thus, for example, MIST translated as ΟΜΙΧΛΗ was regarded as correct, as it is irrelevant to the understanding of the original context that this particular dictionary gives the same translation for MIST as it does for FOG, without any further distinction.  

Similarly, CONDITION translated as ΚΑΤΑΣΤΑΣΗ was accepted as correct; even though a more felicitous translation of WEATHER CONDITIONS would be achieved with ΚΑΙΡΙΚΕΣ ΣΥΝΘΗΚΕΣ, it is the Greek idiom rather than the English one which is not expressed, and the native speaker of Greek will automatically supply that deficiency. On the other hand, if the purpose of the original context, which might be a vocabulary exercise, is to establish such distinctions as obtain between FOG and MIST, IMPERSONATOR and  IMITATOR or even AROMATIC and SPICY, then failure to do so was regarded as critical. Thus FIT translated as ΤΑΙΡΙΑΖΩ was classified as a failure because it implies that FIT is about suitability in terms of colour or pattern rather than size, which is false in the context of trying on clothes. 

Of the 718 look-ups reported, 626 produced a correct result and 92 were incorrect. In the tables that follow, these figures are analysed for each student, for each class, for each type of task, and for each dictionary. The table below shows for each individual student the number of incorrect results produced, the total number of look-ups reported, and the number of incorrect results as a percentage of the total. 

Three students (AK, KF, and ML) were successful in all their searches, but these were relatively few. Apart from these three, by far the most successful was LV, who reported just one mistake in the course of 86 look-ups, compared to her classmate CB, who produced 10 mistakes in 73 look-ups. Both of these were using the OEGLD for a variety of tasks at the level of the Cambridge Certificate of Proficiency in English.

The students who did better than the overall percentage of 12.81% incorrect look-ups included representatives of all classes and the individual variations seem to be more significant than groupings according to the level of study. The table below shows for each class of students the number of incorrect results produced, the total number of look-ups reported, and the number of incorrect results as a percentage of the total. The classes are: 

C = Pre-intermediate

D = Intermediate

F = Approximately the level of the Cambridge First Certificate

P = Approximately the level of the Cambridge Certificate of Proficiency. 

It can be seen from this table that while there are great differences between the rate of success from one class to another, the differences are not systematic. If one were tempted to base conclusions on the fact that the highest level students have the lowest rate of failure, it would be necessary to explain also why students at the next highest level have so many unsuccessful look-ups. 

The next table shows for each type of task the number of incorrect results produced, the total number of look-ups reported, and the number of incorrect results as a percentage of the total. 

 Translation was recorded as a separate category, but not further analysed as only one student was doing it (OT). For each of the other three task types there is a substantial amount of data as these are tasks that most students in most classes have been engaged in and in the course of which they have reported dictionary look-ups. That also means that the effect of the variety of ability between the students is largely neutralised, and so the data can be interpreted with more confidence. It would thus seem that it is much easier for these students to find correctly the meanings of the words that occur in their grammar exercises than it is to find the meanings of the words occurring in the texts they read, with the words that are encountered in vocabulary exercises being even more difficult. Of all the incorrect results produced in the course of 718 look-ups, half occurred in the 241 look-ups inspired by vocabulary exercises, one fifth of which were unsuccessful.

The following table shows for each dictionary used the number of incorrect results produced, the total number of look-ups reported, and the number of incorrect results as a percentage of the total.  

While there are some variations in the success with which the different dictionaries were used, these are just as likely to be attributable to the individual users as to the choice of dictionary. The overall pattern suggests that there is no significant difference between the failure rate of those using the OEGLD and the average failure rate of those using the other dictionaries. However, it should be noted that the data are skewed to a certain extent by one individual (JH) who reports a total of 168 look-ups, 33 with the Boston Press dictionary, and 135 with the Collins Gem. If we remove his figures from the comparison, then the picture is rather different: