Comparative study about computational techniques to Amazonia satellite imagery classification Aug.2019 - Present Multilabel Image Classification • Deep Learning • Ensemble Techniques

The problem consists in classify Amazonia satellite imagery in labels of weather conditions and land use patterns. It was taken from kaggle which, in partnership with Plannet, made available more than 40,000 classified images and more than 60,000 images to a blind test. The images were divided in chips, i.e instead of use an image with 6600×2200 pixels - representing 19877.88 hectares - we are using chips with 256×256 pixels - representing 89.7187 hectares. Each chip should be classified on one of 4 labels about weatherconditions and none or more of 13 labels about land use patterns. Figure 1 shows some examples of this classifications. The chips can be classified with none labels about land use patterns if the weather conditions cover the camera, as in case of 1st chip in Figure 1, the weather condition is "cloudy" and it doesn't allow us to se the land use patterns. The possible classifications are:
weather conditions: clear - when the sky is without any pertubation; cloudy; partly cloudy and haze
Land use Patterns: primary - when the image shows primary forest; agriculture; cultivation - which is like a familiar agriculture; road; water; habitation; bare ground; selective logging; artisinal mine; blooming; slash and burn; blow down and conventional mine.

Figure 1 - Chips and their labels

Source: Kaggle

Due to the low number of our interest classes a search for something that could compensate it became necessary. Figure 3 shows the correlation between labels, i.e the percentile of images labeled on X class that also have the label Y. As we can see, the graph isn't mirrored, i.e a big number of appearance of a class with another doesn't make the opposite true. We can see some obvious correlations like 'primary' with almost every other classes - once 'primary' appears in almost all images that is a thing that we could predict - but we also can se some interesting correlations, like the high percentile of correlation between 'artisinal mine' and 'water', 'conventional mine' and 'road' and 'slash and burn' with 'agriculture' and 'cultivation'. This correlations are interesting because they can help us to compensate the lack of examples that we have to give to the deep learning networks.

In order to explore more the relationships that could be helpful for the finding of our interests classes by the deeplearning networks, the graph in Figure 4 was created. Each node represents a label - excluding weather condition labels - and each edge represents how many times two labels appeared together - strong edges means that two labels appears together in a large set of images, whereas weak edges represents that two labels almost never appeared together. As we can se, our interest classes have less strong edges than the other classes. From that we could think that a set of other labels, that our interest class have a strong edge, can define if it will appear at the image.

Figure 4 - Labels correlation graph

Source: Own autorship

The Figure 5 presents the analysis about if a set of strong edges could define if a interest class will or not appear in some image. Here we get the 5 strongest edges of each interest class and combine them in order to find the best combination as possible. As we can see, despite of exists, this definition is still weak. The class with greater possibility of being described by another ones was 'artisinal mine' and it's biggest possibility of description was 28.13% what isn't neither a half of number of appearness of this class on the image set.

Five deeplearning networks were used to this problem: VGG16, ResNet50, InceptionV3, MobileNet and MobileNetV2. Table 1 shows the results of these networks on the validation set, it is sort by the number of appearness of the label in the validation set. Despite of don't represent neither 0,15% of the validation set the label 'conventional mine' wasn't ignored by four of the five networks, we can explain this with the correlations shown at Figure 5.

Table 2 shows the networks results on blind test - fbeta - and the time that the network spent classifying the 60,000 images of test set. We can see that it is align with the results at validation set.

Source: Own autorship

After the tests with the individual networks, some tests with ensembles of these networks were realized. Table 3 shows the performance of these esambles. This table is sort descending by Fbeta. The first thing that catchs our attention is that the best Fbeta among the ensambles - ensemble 05 - is very close from Fbeta of VGG16 alone - a difference of 0.31 percentual points. It can means that the networks found the same patterns on the classes so they can't do a big contribution to each other.
Another finding that this table provide to us is about the influence of the votes number. The votes systems that had an 'unanimous' policy - 'ensemble 03', 'ensemble 07' and 'ensemble 12' - had the worst performance losing to 4 of the 5 individual networks. On the other hand, in the votes systems that had a big freedom degree, with just need one vote, the Fbeta was dependent of the networks that composed the ensemble. We can see that when we put together the 3 bests networks - 'ensemble 01' - their mistakes didn't affect significantly the Fbeta, but when we add worsts networks - 'mobileNet' and 'mobilenetV2'- to this ensemble - 'ensemble 08' - their mistakes affected too mutch the Fbeta and it's performance was worst than 4 of the 5 individual networks.

Source: Own autorship

Finally, the Figure 6 presents a performance comparation - based on the time in seconds that each network spent to classify the tests set and the Fbeta obtained at blind tests - of all approaches used. We can see that among all of our options VGG16 was the best cost benefit.

Figure 6 - Performances comparation.

Source: Own autorship