Doccano: Annotation statistics counts number of annotated labels (not number of samples)

Created on 5 Dec 2019 · 4Comments · Source: doccano/doccano

for any multi-label annotation project, let's say you annotate 10 samples with 30 labels, the graph will show 30 annotations and not 10.
this doesn't sound like a reasonable behavior.

enhancement good first issue

Source

cohen-ori

👍1

Most helpful comment

Hey @icoxfog417 @Hironsan @cohen-ori

I am picking up this issue for development.

Thanks.

tusharmakkar08 on 7 Apr 2020

👍2

All 4 comments

@icoxfog417: Our team has also been confused as to why we observe different completion values for the same project across several accounts (example: 4/847 vs 263/847). We had thought that the progress bar and pie chart were representative of how many samples ("documents") had been marked as "approved".

Thanks @cohen-ori for explaining this behavior.
Interestingly, this observed behavior does not appear to be present in this screenshot from the tutorial.

As an end user, this is unexpected. If the counter is intentionally designed and implemented this way, then we find it to violate the principle of least astonishment.

Without looking at the source code, I would assume that this is an artifact from the way progress tracking works in classification projects; a classified sample ("document") has just one label, so tracking the number of labels over the number of samples functions as a completion counter.