Evalml: Add a classification accuracy objective

Created on 18 Dec 2019 · 4Comments · Source: alteryx/evalml

Definition (from wikipedia):
Screen Shot 2019-12-18 at 11.44.38 AM.png

This came up in the discussion around Dan Putler's initial work on a cross-platform performance comparison yesterday (notes in #293). Dan used a classification accuracy objective in Applied Modeling, H20 and R, but used recall in evalml. If we implement accuracy, that may provide a better performance comparison. Plus, the more metrics we support, the better.

It would be great to keep our implementation general enough to handle multiclass, but if we have to start with binary classification, so be it.

enhancement good first issue

Source

dsherry

All 4 comments

related to this, we should also implement balanced accuracy

kmax12 on 19 Feb 2020

👍1

Got it, will update the objectives project plan w/ this!

angela97lin on 19 Feb 2020

@angela97lin my suggestion is to list this issue in the future work section if you want, but to not allow it to affect the current objective API project scope. I think our priority should still be to finish the project; then this would be a great first issue to work on once we have it done. That sound ok?

dsherry on 19 Feb 2020

👍1

Ok after talking to Max and Angela, the plan is:
1) This is still separate from the objective API project
2) We can implement this at any point, before/during/after that project, which is why it's in the dev backlog. It's not a huge amount of work
3) Its low priority for now
4) This would be a good issue for a new hire

dsherry on 19 Feb 2020

👍1

Was this page helpful?

0 / 5 - 0 ratings