Doccano: Metadata Fields not Populated on Export

Created on 17 Jun 2019  ·  4Comments  ·  Source: doccano/doccano


System information

  • Windows 10, running latest (pulled on 06/17/2019) Docker image
  • Python version: n/a, using docker version

Describe the problem

The documentation states:

Any other columns (for csv) or keys (for json) are preserved and will be exported in the metadata column or key as is.

But this is not the behavior I am experiencing. I am using the Docker image version of Doccano running on a Windows 10 host. For a sequence labeling project, I import data in this JSON lines format:

{"external_id": "abc12345", "other_field": 5, "text": "hello world"}
{"external_id": "abc12346", "other_field": 2, "text": "this is a test"}

After labeling the data, I go to _Edit Data_ > _Export Data_ and export in either the _JSONL_ or _JSON(Text-Labels)_ formats (issue appears either way). Here is the output I get with the _JSON (Text-Labels)_ output:

{"id": 101, "text": "hello world", "meta": {}, "labels": [[0, 6, "Something"]]}
{"id": 102, "text": "this is a test", "meta": {}, "labels": []}

There is no "metadata" field as described in the documentation, and the "meta" field does not contain any of my metadata required to join the annotated examples back to the other relevant data required for my use case.

Source code / logs

n/a

bug

Most helpful comment

As far as I understand, you need to explicitly use "meta" key both on import and export. Like in https://github.com/chakki-works/doccano/wiki/Import-and-Export-File-Formats

So your import would look like

{"text": "hello world", "meta": {"external_id": "abc12345", "other_field": 5}}

And then your export would also contain the same field. That works for me. But I guess documentation should to be updated on that.

All 4 comments

I tested this again today with the lastest version of Doccano from Docker, I am still seeing the same issue. I believe this would be better classified as a bug than a question as it is behavior that is inconsistent with the documentation.

As far as I understand, you need to explicitly use "meta" key both on import and export. Like in https://github.com/chakki-works/doccano/wiki/Import-and-Export-File-Formats

So your import would look like

{"text": "hello world", "meta": {"external_id": "abc12345", "other_field": 5}}

And then your export would also contain the same field. That works for me. But I guess documentation should to be updated on that.

Please add this to the documentation. I just ran into this issue today.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

gika815 picture gika815  ·  3Comments

callmeashish picture callmeashish  ·  3Comments

zhangxieyang2 picture zhangxieyang2  ·  4Comments

JordanSimba picture JordanSimba  ·  4Comments

rebby123 picture rebby123  ·  3Comments