turi::evaluation::roc_curve Class Reference

#include <toolkits/evaluation/evaluation_interface-inl.hpp>

Public Member Functions
	roc_curve (std::unordered_map< flexible_type, size_t > index_map=std::unordered_map< flexible_type, size_t >(), flexible_type average=FLEX_UNDEFINED, bool binary=true, size_t num_classes=size_t(-1))
std::string	name () const
bool	is_prob_evaluator () const
virtual bool	is_table_printer_compatible () const
void	init (size_t _n_threads=1)
void	register_example (const flexible_type &target, const flexible_type &prediction, size_t thread_id=0)
void	gather_global_metrics ()
virtual variant_type	get_metric ()
virtual void	register_unmapped_example (const size_t &target, const size_t &prediction, size_t thread_id=0)

Detailed Description

Computes the ROC curve. An aggregated version is computed, where we compute the true positive rate and false positive rate for a set of 1000 predefined thresholds equally spaced from 0 to 1. For each prediction, we find which bin it belongs to and we increment the count of true positives (where y=1 and yhat is greater than the lower bound for that bin) and the number of false positives. When complete, these counts are used to compute false positive rate and true positive rate for each bin.

In order to use this class, there are two modes:

• binary mode: In this mode, the inputs are (target_class, prediction_prob) where prediction_prob is the probability of the "positive" class. Here the "positive" class is defined as the largest class as sorted by flexible_type semantics.
• multiclass mode: In this mode, the inputs are (target_class, prob_vec) where prob_vec are the vector of probabilities. In this case, the target_class must be integer

Definition at line 1466 of file evaluation_interface-inl.hpp.

Constructor & Destructor Documentation

roc_curve()

turi::evaluation::roc_curve::roc_curve ( std::unordered_map< flexible_type, size_t >  index_map = std::unordered_map<flexible_type, size_t>(), flexible_type  average = FLEX_UNDEFINED, bool  binary = true, size_t  num_classes = size_t(-1)  )

inline

Constructor.

Parameters

[in]	index_map	Dictionary from flexible_type -> size_t for classes.
[in]	average	Averaging mode
[in]	binary	Is the input mode expected to be binary?

Definition at line 1504 of file evaluation_interface-inl.hpp.

Member Function Documentation

gather_global_metrics()

void turi::evaluation::roc_curve::gather_global_metrics ( )

inline

Gather global metrics for true_positives and false negatives

Definition at line 1697 of file evaluation_interface-inl.hpp.

get_metric()

virtual variant_type turi::evaluation::roc_curve::get_metric ( )

inlinevirtual

Return the final metric.

Implements turi::evaluation::supervised_evaluation_interface.

Definition at line 1724 of file evaluation_interface-inl.hpp.

init()

void turi::evaluation::roc_curve::init ( size_t  _n_threads = 1 )

inlinevirtual

Init the state with a variant type.

Implements turi::evaluation::supervised_evaluation_interface.

Definition at line 1545 of file evaluation_interface-inl.hpp.

is_prob_evaluator()

bool turi::evaluation::roc_curve::is_prob_evaluator ( ) const

inlinevirtual

Returns true of this evaluator works on probabilities/scores (vs) classes.

Reimplemented from turi::evaluation::supervised_evaluation_interface.

Definition at line 1531 of file evaluation_interface-inl.hpp.

is_table_printer_compatible()

virtual bool turi::evaluation::roc_curve::is_table_printer_compatible ( ) const

inlinevirtual

Returns true of this evaluator can be displayed as a single float value.

Reimplemented from turi::evaluation::supervised_evaluation_interface.

Definition at line 1538 of file evaluation_interface-inl.hpp.

name()

std::string turi::evaluation::roc_curve::name ( ) const

inlinevirtual

Name of the evaluator.

Implements turi::evaluation::supervised_evaluation_interface.

Definition at line 1523 of file evaluation_interface-inl.hpp.

register_example()

void turi::evaluation::roc_curve::register_example ( const flexible_type &  target, const flexible_type &  prediction, size_t  thread_id = 0  )

inlinevirtual

Register a (target, prediction) pair

Parameters

[in]	target	Target of a simple example.
[in]	prediction	Prediction of a single example.
[in]	thread_id	Thread id

Implements turi::evaluation::supervised_evaluation_interface.

Definition at line 1614 of file evaluation_interface-inl.hpp.

register_unmapped_example()

virtual void turi::evaluation::supervised_evaluation_interface::register_unmapped_example ( const size_t &  target, const size_t &  prediction, size_t  thread_id = 0  )

inlinevirtualinherited

Register an unmapped (target, prediction) pair. Use this for performance only. Here the target and prediction are assumed to be integers to avoid flexible_type comparisons and flexible_type hashing.

Parameters

[in]	target	Target of a simple example.
[in]	prediction	Prediction of a single example.
[in]	thread_id	Thread id registering this example.

Reimplemented in turi::evaluation::classifier_accuracy.

Definition at line 287 of file evaluation_interface-inl.hpp.

---

The documentation for this class was generated from the following file:

• toolkits/evaluation/evaluation_interface-inl.hpp