turicreate.evaluation.accuracy¶

turicreate.evaluation.accuracy(targets, predictions, average='micro')¶

Compute the accuracy score; which measures the fraction of predictions made by the classifier that are exactly correct. The score lies in the range [0,1] with 0 being the worst and 1 being the best.

Parameters:

targets : SArray

An SArray of ground truth class labels. Can be of any type except float.

predictions : SArray

The prediction that corresponds to each target value. This SArray must have the same length as targets and must be of the same type as the targets SArray.

average : string, [None, ‘micro’ (default), ‘macro’]

Metric averaging strategies for multiclass classification. Averaging strategies can be one of the following:

None: No averaging is performed and a single metric is returned for each class.

‘micro’: Calculate metrics globally by counting the total true positives, false negatives and false positives.

‘macro’: Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.

For a more precise definition of micro and macro averaging refer to [1] below.

Returns:

out : float (for binary classification) or dict[float] (for multi-class, average=None): Score for the positive class (for binary classification) or an average score for each class for multi-class classification. If average=None, then a dictionary is returned where the key is the class label and the value is the score for the corresponding class label.

See also

confusion_matrix, precision, recall, f1_score, auc, log_loss, roc_curve

References

[1] Sokolova, Marina, and Guy Lapalme. “A systematic analysis of performance measures for classification tasks.” Information Processing & Management 45.4 (2009): 427-437.

Examples

# Targets and Predictions
>>> targets = turicreate.SArray([0, 1, 2, 3, 0, 1, 2, 3])
>>> predictions = turicreate.SArray([1, 0, 2, 1, 3, 1, 0, 1])

# Micro average of the accuracy score.
>>> turicreate.evaluation.accuracy(targets, predictions, average = 'micro')
0.25

# Macro average of the accuracy score.
>>> turicreate.evaluation.accuracy(targets, predictions, average = 'macro')
0.24305555555555558

# Accuracy score for each class.
>>> turicreate.evaluation.accuracy(targets, predictions, average = None)
{0: 0.0, 1: 0.4166666666666667, 2: 0.5555555555555556, 3: 0.0}

This metric also works when the targets are of type str

# Targets and Predictions
>>> targets = turicreate.SArray(["cat", "dog", "foosa", "cat", "dog"])
>>> predictions   = turicreate.SArray(["cat", "foosa", "dog", "cat", "foosa"])

# Micro average of the accuracy score.
>>> turicreate.evaluation.accuracy(targets, predictions, average = 'micro')
0.4

# Macro average of the accuracy score.
>>> turicreate.evaluation.accuracy(targets, predictions, average = 'macro')
0.6

# Accuracy score for each class.
>>> turicreate.evaluation.accuracy(targets, predictions, average = None)
{'cat': 1.0, 'dog': 0.4, 'foosa': 0.4}