Evalml: Warning messages in unit test: "invalid value encountered in double_scalars" and others

Problem
Running locally, (on python 3.8, but I've seen similar on other python versions):

(featurelabs) ➜  evalml git:(master) pytest -v evalml/tests/automl_tests/test_autobase.py::test_pipeline_limits
====================================================================== test session starts ======================================================================
platform darwin -- Python 3.8.0, pytest-4.4.1, py-1.8.0, pluggy-0.13.1 -- /Users/dsherry/anaconda/envs/featurelabs/bin/python
cachedir: .pytest_cache
rootdir: /Users/dsherry/development/aie/featurelabs/evalml, inifile: setup.cfg
plugins: xdist-1.26.1, cov-2.6.1, nbval-0.9.3, forked-1.1.3
collected 1 item

evalml/tests/automl_tests/test_autobase.py::test_pipeline_limits PASSED       [100%]

======================================================================= warnings summary ========================================================================
evalml/tests/automl_tests/test_autobase.py::test_pipeline_limits
evalml/tests/automl_tests/test_autobase.py::test_pipeline_limits
...
  /Users/dsherry/anaconda/envs/featurelabs/lib/python3.8/site-packages/sklearn/metrics/classification.py:1436: UndefinedMetricWarning:

  Precision is ill-defined and being set to 0.0 due to no predicted samples.
...
  /Users/dsherry/anaconda/envs/featurelabs/lib/python3.8/site-packages/sklearn/metrics/classification.py:1436: UndefinedMetricWarning:

  F-score is ill-defined and being set to 0.0 due to no predicted samples.
...
  /Users/dsherry/anaconda/envs/featurelabs/lib/python3.8/site-packages/sklearn/metrics/classification.py:872: RuntimeWarning:

  invalid value encountered in double_scalars
...
  /Users/dsherry/development/aie/featurelabs/evalml/evalml/automl/auto_base.py:307: RuntimeWarning:

  invalid value encountered in double_scalars

Three warnings coming from sklearn and one coming from our code. I seem to get a slightly different combination/order of warnings every time I run the test.

More info
Here's the line the last warning is coming from, in AutoBase._add_result:
high_variance_cv = (scores.std() / scores.mean()) > .2

I suspect scores is all empty or all 0. But why? This is the next thing to look into. Perhaps empty test data when we're scoring the model?

My suspicion is that this dataset is too small or too uniform, and that the models trained on it are predicting all the same value or something like that. If I'm right, this reinforces that we need guard rails to detect this problem when the user uploads their data, and that mocking in the unit tests to avoid actual fitting (#275 ) is important (even if this particular test isn't mockable).

I encountered these warnings while debugging bug #167 , so it's possible this is related to that.

Next steps
We should determine why these warnings are showing up. If it's a problem with the test setup, let's change the test to fix or avoid it. Otherwise, it could be a bug. We shouldn't be printing out warnings like this under normal use anyways.