Research Article
Effectiveness of Repeated Basic Navigational Training Sessions Designed for Novice Natural Resource Managers
Warnell School of Forestry and Natural Resources, University of Georgia, Athens, GA 30602, United States of America
Navigation through the forested environment can involve the interpretation of a map, the use of a compass or GPS unit and perhaps pacing (counting walked steps). Even though advances in satellite-based navigational tools provide the ability to effectively orient and guide oneself, field professionals (foresters, biologists and others) continue to use compass and pacing skills in everyday management of natural resources. Orienteering is but one form of navigation and involves the completion of a set of connected traverses of various distances and directions. Orienteering events are usually performed using compass and pacing skills. Another similar event (geocaching) has introduced the use of GPS for navigational purposes. For field sampling and reconnaissance purposes however, distance and direction estimation skills (compass and pacing) are necessary tools for natural resource managers. Navigating through the natural environment using a compass and measuring distances using only a pace count, is a common way of travelling when sampling (inventorying) natural resources and a common way to navigate through terrain where no roads are present. The Future Farmers of America compass practicum is a standard method for testing the ability of a person to accurately determine direction and distance (Herl, 2006). In this exercise, students are asked to determine the azimuth and distance of selected straight line courses, with one point deducted for each two degree deviation from the correct azimuth and one point deducted for each two-foot deviation from the correct distance. Currently, ten courses are used to test compass and pacing skills, with ten points possible for each course. Earlier, four courses were used, where one point was deducted for each deviation in the direction and distance from four straight-line courses.
Introductory courses in natural resource management generally include one that involves learning how to effectively measure directions (Bourdeau and Taylor, 2007). The field compass that is typically used in natural resource management includes a compass needle encased in fluid within a dial, which has directional gradations in 2° increments, either in the form of azimuths (0-360°) or bearings (four 90° quadrants). A typical field compass, such as the Silva Ranger, also includes a base plate, which must be held level and a mirror, for sighting and for adjusting the dial. More advanced compasses that provide more accurate readings, such as a staff compass, can be used in the field, but these are typically involve more set-up time. Less advanced compasses (those lacking a mirror) can also be used, but more often than not these result in less accurate readings. As a result, the most common type of compass used in every-day natural resource management practices is a hand-held device that includes a mirror. Students are typically instructed on the proper methods for holding and using a mirror compass and are typically provided one or more practice sessions to allow them to become comfortable navigating through the natural environment.
In addition to the compass, another standard technique used in natural resource management involves the measuring of distances. Tapes and other devices (such as lasers) can be used to measure distances in the field, yet pacing is commonly employed. While pacing is less accurate than physically measuring distances with tapes or other devices, it is also usually less time-intensive, thus widely used when high levels of accuracy are not necessary. Students learn more about subjects with repetitive and frequent presentations of it and periodic testing of material covered in college-level coursework has been shown to produce a positive effect (Roediger and Karpicke, 2006). One question that arises during introductory natural resource management field training sessions involves the number of practice test sessions that are necessary to gain improvements in direction and distance estimation skills. Some argue that periodic testing should be viewed as a learning opportunity rather than simply a measure of retention of knowledge (Richland et al., 2008) and others suggest that feedback provided after periodic testing is an effective learning process (Kang et al., 2007). Therefore, this study tracked the progress of students in an introductory field measurements course over an extended period of time to determine whether accuracy could be improved with repeated distance and direction test sessions and feedback on their progress. The research was conducted in an established educational setting and involved normal educational practices and educational tests related to the effectiveness of the instructional techniques.
From August 15, 2008 to December 1, 2008, this study was conducted on the Whitehall Forest, near Athens, Georgia (USA). Over a 16 week period, 47 students enrolled in an introductory field measurements course were tested during 7 field laboratories to determine whether measurable improvements in their navigational skills could be observed. On the first day of the study, the students were instructed on the proper use of a mirror compass. They were given hands-on assistance in the importance of keeping the base level, in sighting the dial correctly in the mirror and in sighting an object to which they might navigate through the sighting slot. In addition, they were given hands-on assistance in turning the dial until the compass needle was appropriately positioned within the dial. This instruction was repeated on the second test day as well and was available throughout the test period to reinforce the methods of using a mirror compass. Pacing instructions were also provided. One's pace changes under different field conditions (e.g., field versus woods, uphill versus downhill), therefore, practice sessions were developed to help students understand how their pace might change in different environments. In the previous year that the course was taught, a few of these practice tests were administered and the consensus was that improvements could be seen in student's distance and direction estimation work. These tests became a common aspect of the course, but were not structured well enough to determine significant differences. To test the notion that improvements could be gained with repeated practice tests, a number of voluntary and ungraded practice compass and pacing tests were integrated into the course and made available to the students throughout a semester to allow them to hone their skills. Students were asked to re-examine their compass and pacing techniques on a sample 1-chain (20.1 m) straight course that included a known direction (azimuth). Each of the compass and pacing practice areas was situated in a forested environment, with a moderate amount of down wood and topographical variation.
For each of the seven field laboratories, care was taken to ensure that the practice courses were designed to provide similar experiences from one practice test session to the next. The distance was measured using a measuring tape and the direction was determined using a staff compass. Students were asked to check their pace and test their compass skills on this sample course. A second straight course, with the distance and direction unknown to the students, was also provided. Distances ranged from about 15 to 25 m on these courses and varied from week to week depending on the area chosen for the test. Again, the distance and direction were determined using a measuring tape and a staff compass. Students were then asked to determine their estimate of the distance (to the nearest whole unit) and direction (to the nearest degree) using their compass and pacing skills. Each of the directions were measured assuming a 0° declination, which is consistent with the Future Farmers of America testing protocol. Students wrote their answers on a 7.62x12.71 cm card and provided these to the laboratory instructors. Students were also instructed to trust their own work and not collaborate with others on the answers. After each of the seven practice tests, feedback was provided to the students regarding their progress and a number of discussions were entertained regarding their distance and direction measurement techniques.
The data collected was inserted into a spreadsheet, along with the correct distance and direction measurements and deviations from the correct measurements were determined. For the direction measurements, the absolute value of the deviations could range from 0 to 360°, although, no single response was greater than about 180°, due to either mis-reading the compass, or incorrectly writing the answer on the response card. For the distance measurements, the absolute value of the deviation from the correct answer was also determined. However, in this case, since, the unknown distances varied in length and since, absolute error is usually larger with longer distances, a relative error was determined by dividing the distance deviation by the length of each course.
The main assumption of these practice tests was that with repeated practice sessions and feedback on performance, students would become more proficient with their compass and pacing skills. These skills were further enhanced each week since, the actual field laboratories for the semester-long course involved sampling vegetation in either a systematic or random process, which provided more time for students to practice their navigational skills outside of a testing environment. However, the regular field laboratories may not be sufficient to improve navigational skills since bad habits developed early may be perpetuated throughout the term. A further assumption was that students would also work on their own time outside of class periods to practice these skills, although, evidence of this is lacking. Ultimately, we became interested in determining how many practice test sessions were necessary to reach a non-improving state in both distance and direction field measurements. Therefore, this analysis was undertaken to determine the point at which further improvements in student's field skills in these areas would level off.
Statistical tests were performed on the sets of data to determine whether the means of both distance and direction were significantly different at the 95% confidence level from one testing session to the next. First, the test data was analyzed to determine which distribution (e.g., normal, poisson) best represented the data. In one case (direction measurements), the data needed to be transformed to better represent a normal distribution. F-tests were then employed on the normally distributed data to determine whether the variances could be considered equal. Student's t-tests were then employed to determine whether the means of each practice session differed significantly.
Forty-seven students were involved in the study, however some students could not participate in all seven tests, due to absences, disinterest, tardy arrivals at the field laboratories and injuries sustained in other activities outside of class. Ultimately, only twenty-two of the students participated in each of the seven practice tests. Outright blunders (e.g., azimuth readings that were 30 to 180° in error) from either mis-reading the compass, or incorrectly writing the answer on the response card were omitted from the analysis. These results were attributed to carelessness rather than ineffective use of distance and direction measurement skills, since, a week-to-week assessment of the results did not indicate consistently poor results for these students. In order to adjust for the obvious blunders and to be consistent from one test date to the next, the largest three outliers from each practice test were removed from the data prior to analysis.
Chi-squared tests using BestFit software (Palisade Corporation, 1996) indicated that the distance measurement data were generally best represented by a normal distribution, while the directional data (in discrete form, since, it represents the number of degrees different from the actual staff compass measurement) was best represented by a Poisson distribution. For the distance measurements, an F-test for equal variances was conducted and the conclusion drawn was we could not reject the notion that the sample variance from one test measurement date to the next (for both the 47 student set and the 22 student set) was equal. As a result, student's t-tests were employed to determine significant differences between the means of the distance measurements from one test date to the next. For the direction measurements, the data were first transformed using the method suggested by Freeman and Tukey (1950) since, the original data contained a number of zero values. The transformation involved the following:
(1) |
Transforming data described by a Poisson distribution commonly involves the square root of the variate (Sokal and Rohlf, 1995), while other transformations involving the square root of the original data were attempted, this method resulted in the closest representation of normally distributed data. In sum, the Poisson distribution was a better fit to the original data, yet after transforming the data, the normal distribution was generally a better fit to the data. An F-test for equal variances of the square root transformed data was then conducted and the conclusion drawn was we could not reject the notion that the sample variance differed from one test measurement date to the next (for both the 47 student set and the 22 student set). As a result, student's t-tests were employed, using the transformed data, to determine significant differences between the means of the direction measurements from one test date to the next.
When assessing the changes in direction measurements among the test periods, it was found that the average deviation between student's estimates of a direction using a mirror compass and the actual direction measured using a staff compass was relatively constant (1.27 to 2.41°) from the first test session to the last (Table 1). The only test session that was significantly different (p = 0.05) from the others (with the exception of the fourth test session) was the sixth. The fourth test session was not significantly different (p = 0.05) from sessions 1-3, 5 and 7. And the seventh (and last) test session was not significantly different than the first five sessions. No trends were evident in the results and the maximum deviation (after removing the three largest outliers) was fairly large (6 to 9° in general). When we examined just the 22 students who had the opportunity to take all seven practice tests (Table 2), the results were very similar, although, the average deviation in direction measurements was slightly lower in most cases. Here, the sixth test session was only significantly different (p = 0.05) than the second test session.
The distance measurement tests involving all of the students suggest that improvements can be made with repeated practices (Table 3). Here, the sixth and seventh practice test sessions were significantly different (p = 0.01) from the first, when all students (less outliers) were assessed.
Table 1: | Results from compass tests, all students |
Table 2: | Results from compass tests, selected students |
Table 3: | Results from distance measurement (pacing) tests, all students |
Table 4: | Results from distance measurement (pacing) tests, selected students |
The sixth and seventh test sessions were not significantly different from each other at the p = 0.05 level. The average deviation, in relative terms, was 2.4% during the last test session, which implies that for every 100 m measured, student would estimate a distance that was approximately 2.4 m (longer or shorter) different from the actual distance. When the sub-set of twenty-two students who completed all seven practice tests was assessed (Table 4), again the sixth (p = 0.05) and seventh (p = 0.01) were significantly different than the first test session, although, the last two test sessions were not significantly different from each other.
This study supports earlier findings that repetitive and periodic testing of material covered in college-level coursework can produce a positive effect as suggested by Roediger and Karpicke (2006) on a novice natural resource manager's navigational skills. Curriculum in natural resource orientation and field measurements contains a wide variety of topics, from reading and making maps to learning how to measure and sample various forms of vegetation. The time spent on any one topic may therefore be limited and in many cases the theory behind material is presented, a field laboratory is followed and the course then moves on to another topic. However, gaining competence in a number of basic natural resource management skills may require repeated practice sessions or exercises. Thus, one question that has consistently been pondered is how much practice is sufficient for students to gain confidence in their ability to appropriately measure distances and directions. When compass and pacing skills are considered, practice test sessions are relatively easy to set up and administer prior to actual field labs that may involve other topics, such as the systematic samples of trees. While this became a normal part of the course during the semester it was taught, encouraging students to participate in the practice tests was problematic due to the fact that they were ungraded and voluntary.
This study also confirms that feedback provided after periodic testing is an effective learning process by Kang et al. (2007). While compass and pacing skills are used throughout the field measurements course, traditionally the methodology is presented in a single session, a practice session is provided and students are expected to hone these skills throughout the course with limited feedback. Rather than provide instruction on these important topics and move on to others, repetitive practice, testing and feedback for distance measurement skills seems necessary to improve accuracy in this area over time. One limitation of this study is that no control group was tested, mainly due to the size of the class and the researchers reluctance to disallow students the opportunity for additional practice in these areas. However, previous year's compass and pacing skills, as measured using the Future Farmers of America 4-course method and where no repetitive practice, testing and feedback was provided, averaged 84.4, 84.5 and 85.2 for the 2004, 2005 and 2006 years, respectively. The students involved in the current test had an average test score of 89.1 for a similar Future Farmers of America 4-course compass and pacing test. Although, about one-third of the students who completed all seven of the periodic practice tests (n = 22) did not show an improvement in distance measurement skills when comparing the first test to the last, the average improvement in accuracy was 1.6%, or 1.6 m over a 100 m distance. When considering the entire class, those who completed both the first and last of the periodic practice tests (n = 40) showed a 1.0% improvement in distance measurement skills. However, of these, 40% of the students did not show an improvement from the first practice test to the last.
Most of the previous work in the area of repetitive testing (Agarwal et al., 2008; Richland et al., 2008; Roediger and Karpicke, 2006) has focused on person's memory and its influence on the long-term retention of information. In our case, repetitive testing was used to help students arrive at a consistent method for performing a common natural resource management task. Upon initial introduction of the topic, distance measurements efforts may have been based on a person's forced gait, rather than a more natural gait. Repetitive feedback may have allowed students to understand that a more natural gait is required to more accurately measure distances. In addition, methods for navigating over or around obstructions may not have been honed until after several practice tests. Therefore, rather than introduce the subject once and allow students to use unrefined skills in field exercises, we suggest that periodic feedback with regard to distance measurements will result in higher- accuracy skill over the long run and induce a level of confidence and competence that will serve them well in the future careers.