Zotero-better-bibtex: url field is having its special characters escaped in BBT Bibtex

Created on 3 Jan 2021  ·  8Comments  ·  Source: retorquere/zotero-better-bibtex

Exporter used: Better Bibtex

Expected behavior:

@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-%C3%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}

Actual behavior:

@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-\%C3\%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}

The error is in the url field. Although the url field is marked as a verbatim field, when exporting using BBT Bibtex, special characters are escaping (in the example "%" is being escaped).
The error only happens in BBT Bibtex, using BBT Biblatex or the standard Zotero Bibtex, everything is fine.

url = {https://medium.com/@eam.avelar/o-que-\%C3\%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d} with % being scaped.

Version of the systems where the error occurs:
Zotero version: 5.0.94
Zotero version: 5.0.95-beta.3 + 4db61b8dc
OS: macOS 11.1

BetterBibtex Debug Json file:

{
  "collections": {},
  "config": {
    "id": "36a3b0b5-bad0-4a04-b79b-441c7cef77db",
    "label": "BetterBibTeX JSON",
    "localeDateOrder": "dmy",
    "options": {
      "Normalize": false,
      "exportFileData": false,
      "exportNotes": true,
      "keepUpdated": false
    },
    "preferences": {
      "DOIandURL": "both",
      "ascii": "",
      "asciiBibLaTeX": false,
      "asciiBibTeX": true,
      "autoAbbrev": false,
      "autoAbbrevStyle": "",
      "autoExport": "immediate",
      "autoExportDelay": 1,
      "autoExportIdleWait": 10,
      "autoExportPathReplaceDiacritics": false,
      "autoExportPathReplaceDirSep": "-",
      "autoExportPathReplaceSpace": " ",
      "autoPinDelay": 0,
      "automaticTags": true,
      "auxImport": false,
      "biblatexExtendedDateFormat": false,
      "biblatexExtendedNameFormat": false,
      "biblatexExtractEprint": true,
      "bibtexParticleNoOp": false,
      "bibtexURL": "url",
      "cacheFlushInterval": 5,
      "citeCommand": "cite",
      "citekeyFold": true,
      "citekeyFormat": "[auth][shorttitle2_2][year]",
      "citeprocNoteCitekey": false,
      "csquotes": "",
      "debugLogDir": "",
      "exportBibTeXStrings": "off",
      "exportBraceProtection": true,
      "exportTitleCase": true,
      "extraMergeCSL": true,
      "extraMergeCitekeys": true,
      "extraMergeTeX": true,
      "git": "config",
      "ignorePostscriptErrors": true,
      "import": true,
      "importBibTeXStrings": true,
      "importCaseProtection": "as-needed",
      "importCitationKey": true,
      "importExtra": true,
      "importJabRefAbbreviations": true,
      "importJabRefStrings": true,
      "importSentenceCase": "on+guess",
      "itemObserverDelay": 5,
      "jabrefFormat": 0,
      "keyConflictPolicy": "keep",
      "keyScope": "library",
      "kuroshiro": false,
      "mapMath": "",
      "mapText": "",
      "mapUnicode": "conservative",
      "newTranslatorsAskRestart": true,
      "parseParticles": true,
      "platform": "mac",
      "postscript": "",
      "postscriptOverride": "",
      "qualityReport": true,
      "quickCopyMode": "latex",
      "quickCopyPandocBrackets": false,
      "rawImports": false,
      "rawLaTag": "#LaTeX",
      "relativeFilePaths": false,
      "retainCache": false,
      "scrubDatabase": false,
      "skipFields": "keywords, file,  timestamp",
      "skipWords": "a,ab,aboard,about,above,across,after,against,al,along,amid,among,an,and,anti,around,as,at,before,behind,below,beneath,beside,besides,between,beyond,but,by,d,da,das,de,del,dell,dello,dei,degli,della,dell,delle,dem,den,der,des,despite,die,do,down,du,during,ein,eine,einem,einen,einer,eines,el,en,et,except,for,from,gli,i,il,in,inside,into,is,l,la,las,le,les,like,lo,los,near,nor,of,off,on,onto,or,over,past,per,plus,round,save,since,so,some,sur,than,the,through,to,toward,towards,un,una,unas,under,underneath,une,unlike,uno,unos,until,up,upon,versus,via,von,while,with,within,without,yet,zu,zum",
      "strings": "",
      "testing": false,
      "verbatimFields": "url,doi,file,eprint,verba,verbb,verbc,groups",
      "warnBulkModify": 1,
      "warnTitleCased": false,
      "workers": 0
    }
  },
  "items": [
    {
      "abstractNote": "Em aprendizagem de máquina, a medição de desempenho é uma tarefa crucial. As métricas ROC (Receiver Operating Characteristics) e AUC (Area…",
      "accessDate": "2020-11-23",
      "attachments": [
        {
          "dateAdded": "2020-12-31T15:41:19Z",
          "dateModified": "2020-12-31T15:41:19Z",
          "itemType": "attachment",
          "path": "/Users/rflmorais/Zotero/storage/JER6PBMW/o-que-é-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d.html",
          "relations": [],
          "tags": [],
          "title": "Snapshot",
          "uri": "http://zotero.org/users/7315465/items/JER6PBMW"
        }
      ],
      "citationKey": "AvelarQueAUC2019",
      "citekey": "AvelarQueAUC2019",
      "creators": [
        {
          "creatorType": "author",
          "firstName": "Adriano",
          "lastName": "Avelar"
        }
      ],
      "date": "22/09/2020",
      "dateAdded": "2020-12-31T15:41:14Z",
      "dateModified": "2021-01-03T14:17:34Z",
      "extra": "Citation Key: AvelarQueAUC2019",
      "itemID": 71,
      "itemType": "webpage",
      "key": "ZMDN7VRS",
      "language": "pt-BR",
      "libraryID": 1,
      "notes": [],
      "relations": [],
      "select": "zotero://select/library/items/ZMDN7VRS",
      "tags": [],
      "title": "O que é AUC e ROC nos modelos de Machine Learning",
      "uri": "http://zotero.org/users/7315465/items/ZMDN7VRS",
      "url": "https://medium.com/@eam.avelar/o-que-%C3%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d",
      "version": 255
    }
  ],
  "version": {
    "bbt": "5.2.102",
    "zotero": "5.0.95-beta.3+4db61b8dc"
  }
}

bug

All 8 comments

It looks like you did not upload an debug report. The debug report is important; it gives @retorquere your current BBT settings and a copy of the problematic reference as a test case so he can best replicate your problem. Without it, @retorquere is effectively blind. Debug reports are useful for both bug analysis and enhancement requests; in the case of export enhancements, I need the copy of the references you have in mind.

If you did try to submit a debug report, but the ID looked like D<number>, that is a Zotero debug report, which I cannot access. Please re-submit a BBT debug log by one of the methods below.

This request is much more likely than not to apply to you, too, _even if you think it unlikely_, and even if it does not, there's no harm in sending a debug log that turns out to be unnecessary. @retorquere will more often than not just end up saying "please send a debug log first". Let's just skip over the unnecesary delay this entails. Sending a debug log is very easy:

  1. If your issue relates to how BBT behaves around a specific reference(s), such as citekey generation or export, select at least one of the problematic reference(s), right-click it, and submit an BBT debug report from that popup menu. If the problem is with export, please do include a sample of what you see exported, and what you expected to see exported for these references.

  2. If the issue does not relate to references and is of a more general nature, generate an debug report by restarting Zotero with debugging enabled (Help -> Debug Output Logging -> Restart with logging enabled), reproducing your problem, and selecting "Send Better BibTeX debug report..." from the help menu.

Once done, you will see a debug ID in red. Please post that debug id in the issue here.

Thank you!

When I compile this:

\documentclass{article}
\usepackage[utf8]{inputenc}
\bibliographystyle{plainnat}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-\%C3\%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}
\end{filecontents}

\begin{document}

\nocite{*}

\bibliography{\jobname.bib}

\end{document}

I get the URL as expected.

When I compile this:

\documentclass{article}
\usepackage[utf8]{inputenc}
\bibliographystyle{plainnat}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-%C3%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}
\end{filecontents}

\begin{document}

\nocite{*}

\bibliography{\jobname.bib}

\end{document}

I get an error, and no URL is output. So it looks like for bibtex, the escaping is necessary.

BTW, if you send a debug log via the steps described in point 1, I get sent the reference in debug log format automatically; my workflow is built around that. This worked though.

Sorry about the debug stuff. Lesson learned.
Thanks for the quick response. I thought it was a bug in Zotero considering that is a verbatim field. But it seems is not. Im using a flavored bibtex from my university and I'm getting an error with the url with escape chars. I'm suspecting they using some old macros to parse the bib file

Thanks a lot

No worries about the debug log -- I'm happy you sent one. You wouldn't believe how stubborn people can be about sending them. It's one of the primary reasons I built @label-gun, because it would be practically a given people would not send one no matter how I pleaded.

WRT the URL -- things are more complicated than I had thought. It is true that

\documentclass{article}
\usepackage[utf8]{inputenc}
\bibliographystyle{plainnat}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-%C3%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}
\end{filecontents}

\begin{document}

\nocite{*}

\bibliography{\jobname.bib}

\end{document}

gives me an error, but

\documentclass{article}
\usepackage[utf8]{inputenc}
\bibliographystyle{plainnat}
\usepackage{url} % <------------------- this right here
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-%C3%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}
\end{filecontents}

\begin{document}

\nocite{*}

\bibliography{\jobname.bib}

\end{document}

compiles just fine. I'll have to think about how to deal with both situations.

:robot: this is your friendly neighborhood build bot announcing test build 5.2.102.247 ("fixes #1716")

Install in Zotero by downloading test build 5.2.102.247, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

Problem solved. It's worked perfectly.
Thanks a lot. It's save me a lot of time

Cool, thanks for the confirmation. A new release is building.

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

element4l picture element4l  ·  7Comments

SunHaozhe picture SunHaozhe  ·  3Comments

DrGo picture DrGo  ·  6Comments

bwiernik picture bwiernik  ·  7Comments

alisonketz picture alisonketz  ·  6Comments