Figure 26 shows the spatial accuracy of the ESA African land cover map for Ivory Coast, which shows large areas in which the accuracy is low, while spatial user accuracies by land cover class are shown in Figures 27 to 31. The main issue is overestimation of shrubland and underestimation of grassland as shown in Figure 25 as well as in the spatial accuracy maps (e.g., see Figure 28).
Figure 26: Map of the spatial accuracy of the ESA African land cover map for South Africa
Figure 27: Spatial accuracy of the ESA African land cover map for South Africa for the tree cover class
Figure 28: Spatial accuracy of the ESA African land cover map for South Africa for the shrub cover class
Figure 29: Spatial accuracy of the ESA African land cover map for South Africa for the grassland class
Figure 30: Spatial accuracy of the ESA African land cover map for South Africa for the cropland class
Figure 31: Spatial accuracy of the ESA African land cover map for South Africa for the bare area and sparse vegetation classes combined
Conclusions
This working paper has provided an accuracy assessment of the ESA 20 m land cover map of Africa for four African countries (Kenya, Gabon, Ivory Coast and South Africa). The results varied from 44% (for South Africa) to 91% (for Gabon). In the case of Kenya (56% overall accuracy) and South Africa, these values are largely caused by the confusion between grassland and shrubland. This may be due to the training data used by the classifier and should be carefully checked. The training data for the ESA African land cover map were partly taken from existing maps and may also go some way to explaining the classification errors. However, we have demonstrated that if a weighted confusion matrix is used, which diminishes the importance of the confusion between grassland and shrubs, the overall accuracy for Kenya increases to 79% and to 75% for South Africa.
The overall accuracy for Ivory Coast can be explained using different reasons. Ivory Coast has a highly fragmented land cover, which makes it a difficult country to map with remote sensing. Moreover, there will most likely be a low density of usable optical images that are cloud free, which may be compounding problems with the classification. The exception was Gabon with a high overall accuracy of 91% but can be explained by the high amount of tree cover across the country, which is a relatively easy class to map.
One might argue that doing a validation of a 20 m resolution map using 20 m resolution pixels is not the right approach due to geo-registration errors. However, we would argue that the issues are not related to resolution but rather misclassification of large areas. The South Africa example clearly demonstrates this since a different approach was used, i.e., the dominant land cover over a 100 m squared area was used in the validation yet this map had the lowest accuracy of all 4 countries. Hence aggregating to a larger area for validation does not improve the accuracy figures because the areas that are misclassified are very large. An example where aggregation might improve accuracy is Gabon. Although most is forest, there are occasional validation pixels at a 20 m resolution with a different land cover class such as cropland that could be considered noise. Aggregating to a larger area would remove these cases. However, as the overall accuracy for Gabon was already very high, this would be unlikely to make a huge difference.
Below is a list of suggestions for how high-resolution land cover mapping might be improved in the future:
• Improve the training data, particularly if they have been derived from coarser resolution maps rather than visual interpretation or in situ data collection. In areas where there are problems, LACO-Wiki could be used to gather a large training data set to improve those classes where there is currently large confusion, provided the resources exist to collect field data. The algorithm for creating a sample along a road network may help in more efficiently collecting the training data.
• Make use of additional training data that can be collected by using additional sources of data besides very high-resolution imagery (e.g., bioclimatic layers, field size maps, geo-tagged photographs (e.g., from Flickr and Mapillary)). There are numerous automatic object recognition algorithms that could classify photos into land cover types. This may be an additional source of training data to supplement the data that was used in creating the ESA African land cover map.
• Interact closely with local experts, e.g., to provide local insights into land cover types that are specific to an area of a country. The wetland example in Ivory Coast provides good evidence of the need to involve local experts. However, local experts will only provide additional value if they work very closely with the person who is involved as the global expert for training data collection. We have also experienced that validation points from local experts will need to be checked and possibly corrected since their personal view and interpretation of land cover classes many times does not match the general definition of the class applied. Hence just relying on local experts alone might result in unsatisfactory classifications. This issue becomes evident when classifications from different local experts are compared and large disagreements can occur. Hence there is a need to balance this interaction.
• Use additional sources of remote sensing imagery in the classification, e.g., Sentinel 1 in addition to Sentinel 2 or other imagery (e.g., Landsat) in a sensor fusion approach. This may help to filter out some of the noise, e.g., the occurrence areas of cropland in forest areas in Gabon.
References
Fritz, S. and See, L. 2008. Quantifying uncertainty and spatial disagreement in the comparison of Global Land Cover for different applications. Global Change Biology, 14, 1-23.
Lesiv, M., Fritz, S., McCallum, I., Tsendbazar, N.E., Herold, M., Pekel, J.-F., Buchhorn, M., Smets, B., et al.
2017. Evaluation of ESA CCI Prototype Land Cover Map at 20m. IIASA Working Paper. IIASA, Laxenburg, Austria: WP-17-021.
Lesiv, M, Tsendbazar, N.E., Herold, M., Buchhorn, M., Smets, B., Van De Kerchove, R., Pekel, J.-F., Maus, V., Duerauer, M., See, L., Fritz, F. 2019. Beyond Kappa and Overall Accuracy – A Spatial Accuracy
Assessment of Recent Land Cover Products. Living Planet Symposium, 13-17 May 2019, Milan, Italy.