Tests and assessments are part of our daily lives. Throughout our schooling we are bombarded with tests, exams, and assessments. In adulthood we need to take driving tests, employers and professions require qualifying exams before they will admit you to their organisations. If that does not satisfy our need to be tested many magazine covers advertise Do-it-yourself tests; "Test your tolerance", "Find your fitness factor" and so on.
For Taekwon-do practitioners the test that marks a major milestone is the black belt grading test. However what can we say about this grading experience. Is it good, bad or just ugly?
This thesis will examine the meaning of tests and exams in our lives and assess where the black belt grading fits. Then it will suggest some criteria for evaluating whether a test is good or bad and use these to evaluate the New Zealand grading system and the grading system as recommended by the ITF in the condensed Encyclopaedia.
Current literature suggests that assessments and exams have the following purposes:
Motivation, Analysis & feedback, Certification, Standard setting, Selection and Recording & Profiling
The presence of a test motivates a student to work harder and do more practice than if there were no tests. Tests and exams are said to provide extrinsic and intrinsic motivation. In Taekwon-do the reward of obtaining the prestigious black belt at the end of the test provides extrinsic motivation. The coloured belt and black belt ranking system provides goals for students to aim for, giving them a purpose to continue training. The human characteristic of peer pressure causes a person to want to keep up with the grades of their peers and when they take their grading they try to outdo or at least do as well as their peers. The natural drive to succeed and to take up challenges means that the mere presence of a test will motivate students to keep coming to training to prove and improve themselves. This is the intrinsic.
Perhaps the original purpose of the Taekwon-do grading system was to measure a student's progress, however by establishing a structured system of rank the students were also given goals to aim for and thus motivate them to continue training and to progress through the ranks.
ANALYSIS & FEED BACK
This is the idea that the test results will help the assessor, the instructor and the student to analyse strengths and weaknesses. From an individual student's perspective, obtaining feedback is a useful part of the learning process. The more detailed the feedback the more useful it will be to help focus on development areas and to confirm positive achievement. An instructor is able to get feedback on the standards of the club as whole, about individual students and to gauge from the results how effective the teaching has been. The assessor will be interested to see how instructors are doing, whether standards are uniform, whether more attention needs to be put into any particular areas. For example, gradings can highlight weaknesses in a region or a particular club or in different areas of the syllabus.
It would be fair to say that most students aim to pass a Taekwon-do grading the first time. They are not generally taking the test to get feedback on what they are doing wrong; that is the role of instructors and mock gradings. Therefore it seems that analysis and feedback is currently an incidental purpose of gradings.
Certification is the formal recognition by an authorised body that a student has gained a certain level of knowledge and reached a certain level of competence. This is perhaps the most obvious purpose of the black belt grading as success means not only a certificate on paper but also gives a very visible stamp of approval in the form of the black belt. This is also the first instance where students are registered with the ITF (International Taekwon-do Federation) thus joining the international black belt society.
In any organisation that strives for excellence from it's members and respect from society the issue of standards is always at the forefront of debate. In Taekwon-do we find the issue of standards occupying a dominant role at national and international level. General Choi travels the world holding seminars to try to harmonise and continually improve standards internationally. Each country is concerned about having "high" standards among its practitioners. And one of the natural roles of the black belt grading is to set and maintain standards.
However standards without further clarification is an imprecise term. What standards are we talking about? Is it how high one can jump - the tower kick is a requirement of the 1st dan grading in New Zealand and the 3rd dan grading as recommended in the Encyclopaedia. Are these standards absolute?
In practice the black belt standard is not specified in any black and white terms. This thesis warns that the term standards is imprecise and subjective. Too often the standards debate is raised without clearly defining which standards are at issue. On an international level the standard for a black belt is far from clear. In New Zealand, because we have a comprehensive grading syllabus we can say what one must do to qualify as a black belt but we do not have one sentence that will encapsulate what that standard is. What the standards are now, what they should be and who should decide are issues for another thesis.
This thesis contends that although individual instructors will vary as to the standards required from their students ultimately the minimum standard is what is specified by the grading syllabus. For example most instructors will neglect some area of Taekwon-do for one reason or another. Say that self defence was rarely taught at club, by including this aspect in a grading we ensure that students will practise this area and get themselves to the minimum competence standards as required by the grading test. Instructors will also be reminded to teach areas they may otherwise neglect.
Therefore this thesis contends that the setting of and maintenance of standards is a dominant role of the grading system.
Where demand exceeds supply a selection is required to determine who gets the resources. The results of tests and assessments are often used to select people for such things as scholarships and jobs. In Taekwon-do the need to select has not been great and so this has been an insignificant role of gradings.
Should the need arise then the comparability of results from one grading to another will be important. If selection is required across national borders then international comparability of the grading system will also be important.
RECORD KEEPING AND PROFILING
When the results of many students are recorded along with other biographical data such as age, sex, height, or ethnic group then from this data further useful information can be created. From this "database" information such as the percentage of males passing their grading or the age distribution of people taking a particular grading can be extracted and used to help management (The Executive or Instructors for example) in making management decisions. These include decisions such as marketing strategies or teaching strategies or administration policies.
Currently Taekwon-do grading results are not being used to build a database but the results are available and no doubt they can be compiled to give some interesting profiles!
The table below summarises the different purposes of the black belt grading system and the significance of that purpose.
PURPOSE Motivation Analysis & Feedback Certification Standards Selection Recording
HOW DO WE JUDGE A GRADING?
It is one thing to take a test but how do we assess whether the test we are taking is fair and that it is a good test? This thesis submits that some tests are better than others and that a good test should have the following characteristics:
1. There should be clear information available on what is expected.
2. The grading tasks should be comprehensive and representative of the Taekwon-do syllabus as a whole.
3. The grading tasks should be relevant to the purpose of gradings.
4. The grading system should be flexible.
5. The grading should achieve its purposes.
ASSESSING T1IE ASSESSMENT
=> There should be clear information on what is expected before and during the grading.
The availability of information is desirable as it allows the instructor and the student to plan and assess. Instructors are able to better plan their classes if they know the dates of gradings and what is expected at them. Instructors can then prepare their students thoroughly. Students can assess whether or not they are ready for a grading and plan their training towards a particular grading.
Information should be available about the procedures of the grading and the content of the grading. Information on the contents of a grading should include what tasks will be required, how each task will be measured and what results if successful. For example the fitness test in the New Zealand grading is a good example. It specifies what is required; eg. sit ups, how this is measured; with arms across the chest coming up so that elbows touch knees; and how the result will be measured; a scoring of between 1-4 points depending on how many sit ups are done.
The procedures of a grading refer to how the grading will be conducted, the order of testing where there are numerous headings of tasks required and other basic procedural matters.
The alternative to having an informed approach seems to be the "if you grade then you should be prepared for anything" approach. This is where no one knows for sure what to expect and written information is non-existent. This type of grading format seems to be the traditional method and some national bodies still adopt this approach.
There are several weaknesses to this approach. Students are faced with uncertainty which leads to extra stress in an already stressful situation. Performance as a whole may then decrease and so maximum performance levels are not achieved. Gradings become far more variable from one to the other because there are no set guidelines as to content or format. An examiner's preference for some areas over others will lead to bias in the grading.
All these factors will have a negative impact on the public perception of our martial art as gradings are one of the main events where Taekwon-do is in the public arena. At international level the availability of information to students and instructors will be dependent on the policies and efficiency of the governing national body.
In New Zealand the information channel is via the Grading Sub-committee. This Sub-committee publishes a black belt application "pack" which is fairly comprehensive and self contained. This pack contains information on procedural matters as well as a detailed syllabus for each grade together with the necessary application forms. Proposed dates of grading are also published and circulated to the regions for confirmation. Overall New Zealand Taekwon-do practitioners are well informed about what to expect at a grading and can access that information readily. I would rate this aspect of the New Zealand grading system as excellent.
> The grading tasks should be comprehensive and representative of the Taekwon-do syllabus as a whole.
Comprehensive means that the many different aspects of Taekwon-do should be tested. When we refer to the cycle of Taekwon-do in the condensed Encyclopaedia we see five areas specified. These are described as fundamental movements, patterns, conditioning, sparring and self defence. When we read General Choi's philosophies then an even broader picture of Taekwon-do emerges. In particular General Choi stresses a strong theoretical component which includes studying the meaning of and acting according to moral culture and applying science to the application of techniques such as the theory of power. In the Encyclopaedia, General Choi writes with regard to System of Rank "In Taekwon-do, character development, fortitude, tenacity, and technique are graded as well as individual capacity.".
Since Taekwon-do has a knowledge base which is represented on paper by 15 volumes it would be impossible to comprehensively test everything. Therefore the emphasis of this criterion is that the grading tasks are representative of what Taekwon-do is about.
Taekwon-do gradings can be described as criterion referenced assessments because everyone who achieves the set standards should pass. The criterion-referenced assessment is a good thing because it emphasises positive performance and your score does not depend on who else is sitting the test. The other form of assessment can be described as norm-referenced because the pass rate is pre-determined by a distribution table or graph. Your score is determined by how you performed relative to how others performed. The old School Certificate examination was a norm-referenced assessment.
Since the grading is criterion referenced it makes it more important that the grading tasks are representative. If a grading only tests one or two aspects of Taekwon-do we must question the quality of the certification process. Under a narrow test a student who meets the competency levels in the areas tested should pass regardless of any competency or otherwise in the other aspects of Taekwon-do. If the tasks are not representative then we also must question the ability of the grading system to maintain or set standards.
Another weakness of a narrow grading syllabus is that it is biased towards those areas represented and this may help or hinder a student depending on whether they are strong or weak in those areas. In either scenario the grading will not provide optimum motivation for students because a student weak in the areas tested will be put off and a student strong in the areas tested will not find sufficient challenge.
The dan grading tasks in the Encyclopaedia is listed under three headings; Patterns, Sparring and Power. Sparring comprises two forms either free-sparring and foot technique for 1St and 2nd dans or free sparring and self defence for 3rd and 4th dans. At most 4 to 5 aspects of Taekwon-do will be tested. This is not a very comprehensive test when considered against the many non-physical aspects to Taekwon-do and General Choi's philosophies.
The weighting of these areas are not stated. If they have similar to equal weighting then power counts for 33% of the grading. This is not representative of the normal Taekwon-do syllabus nor of what a student can be expected to do in class.
When we evaluate the ITFNZ grading syllabus we see that it incorporates no less than 10 separate headings of tasks (fitness test, patterns, free-sparring, 1-step sparring, foot sparring, self defence, destruction, written theory, oral test to assess attitude/spirit, contribution credits and a thesis for 2nd dans and above). The New Zealand grading can be considered fairly comprehensive and representative of the Taekwon-do syllabus as a whole.
> The grading tasks should be relevant to the purposes of the grading.
The dominant purposes of gradings were identified as motivation, certification and setting standards.
Tasks can be non-relevant to the purpose of gradings and weaken the grading. In a certain karate style students taking the black belt grading in Japan must be able to walk the length of their training hall on their hands. This task may be relevant in some way to that style but seems to have tenuous relevance to our martial art.
Tasks may become outdated and have decreased relevance as time moves on. For example the requirement to break bricks seems to be a traditional requirement which is still used in some countries (such as the United Kingdom). There is some danger of causing permanent injury by this task so we need to assess what purpose this task serves and consider whether there are safer methods. Also we must question whether this task is representative of what we do in Taekwon-do. Could we expect to be breaking bricks at every second training?
As the black belt certificate does not specify exactly what we have certified we do not have a checklist to say that one task is relevant and that another is not. However the guiding principle ought to be that a task should be something that can be reasonably expected as part of Taekwon-do training.
> The grading system should be flexible.
Flexibility refers to the ability to change and develop. There are several aspects to the notion of flexibility in regard to gradings; flexible as to format and flexible as to content and this thesis further argues that there should be flexibility in the measurement of achievement in a task.
The format refers to how gradings are conducted and the process itself. It also refers to the choice of testing methods such as physical tests, verbal tests, written tests and impression assessment by the examiners. In New Zealand we clearly adopt a point score system. Pass or fail is determined by an aggregation of points achieved from the different grading tasks. No one task is absolute. This format incorporates a degree of flexibility. What we must not have is where some tasks are absolute because absolute standards are arbitrary and place undue emphasise on those aspects. It appears that the Encyclopaedia recommends the point score system as well but the testing methods are not as varied.
The content refers to the syllabus and the grading tasks. In New Zealand we have seen several examples of change in this area. The fitness test was introduced in 1995, the destruction requirements were changed to give it less emphasis and updates in techniques are continually being incorporated and tested via the patterns. The contents of the grading in the Encyclopaedia were significantly less comprehensive than the New Zealand grading. It is unclear whether the grading tasks have been revised in recent years. There is scope to improve this aspect of the recommended grading syllabus to make it more comprehensive and representative.
The third aspect refers to having a different measurement scale for different people such as that used to measurement achievement in the fitness test. On policy grounds flexibility in the measurement of performance is desirable and should be adopted where applicable. This is because Taekwon-do never was intended to be and is no longer solely practised by able bodied young men. The relative achievement levels of Mrs X, a middle aged woman, being able to kick to head level can be just as high or higher than for Mr Z, a young male, being able to jump and kick to an arms length above head level. The counter argument to allowing such flexibility in the measurement of grading tasks is that standards will be lowered. This thesis disputes this objection as in the eyes of the public seeing Mrs X kick to head height one would not say "That looks easy, anyone can do that." Instead what is more likely; "That woman is doing well, I wonder if I can do that too if I start training."
If we have flexibility we encourage individual achievement as opposed to absolute achievement. Absolute standards can cause students to be discouraged from training. Whereas if we set quality standards but allow for individual capacities we will encourage participation from a wider section of the population.
If past performance is indicative of the future then the New Zealand grading is in good shape to move with the times. Taekwon-do is a dynamic art and the courage to change shows that we are a modern martial art.
=> The grading should achieve its purposes
The primary purposes of the grading system were to motivate, to certify and to set and maintain standards.
Motivation is achieved if the grading system is stable and well structured so as to set high but achievable goals for students.
A quality certification process is one where students and instructors are well informed of the procedures and standards. As certification is about getting a certificate the examiners or administrators of the grading must follow through with the paperwork, updating details at national level and registering successful candidates with the ITF.
The setting and maintaining of standards is achieved if the grading tasks are comprehensive, representative and relevant.
For reasons discussed above the New Zealand grading system measures up successfully against these purposes. In comparison the guidelines in the Encyclopaedia are lacking in comprehensiveness and tends to over-emphasise power testing. This bias may inhibit students from taking their dan gradings.
Also the techniques to be used for power are specified in the syllabus which narrows the grading tasks and makes the grading less flexible. An alternative could be something like the NZ 2nd dan syllabus which allows a students to choose a flying hand and foot technique. The imbalance in the ITF guidelines leads to a poorer certification process and lessens the ability of the grading system to set and maintain standards.
Some grading processes are better than others. A good grading is one where all parties are well-informed and the grading tasks are representative of the whole syllabus and relevant to the purposes of having a grading. Further the grading system should be flexible and allow for development in the future.
New Zealand has chosen not to strictly adhere to the grading guidelines in the Encyclopaedia but has developed a far more comprehensive and flexible approach. Viewed in light of the roles and purpose of assessments and the five characteristics of a good assessment the NZ grading rates highly on all accounts. In comparison the narrower guidelines found in the Encyclopaedia show weaknesses in some areas and may indicate a need to update and revise those guidelines.
Obtaining a black belt is a significant milestone in a person's Taekwon-do life cycle and the importance of the black belt grading cannot be overemphasised. It is therefore in the best interests of our art to ensure that this process is well considered and continually revised.
Assessment; a teachers guide to the issues
Caroline Gipps and Gordon Stobart, Hodder & Stoughton 1993
Assessment and Examination in the Secondary School
eds. Richard Riding and Sue Butterfield, Routledge 1990
Testing and Assessment
Charles Desforges, Caswell Education Ltd, 1989
The Condensed Encyclopaedia of Taekwon-do
General Choi Hong Hi, 1995
The Search for Standards
ed. Harry Tomlinson, British Educatlonal Management and Administration Society, 1992