turi::evaluation::supervised_evaluation_interface Class Reference

#include <toolkits/evaluation/evaluation_interface-inl.hpp>

Public Member Functions
virtual	~supervised_evaluation_interface ()=default
	supervised_evaluation_interface ()=default
virtual std::string	name () const =0
virtual void	init (size_t _n_threads=1)=0
virtual bool	is_prob_evaluator () const
virtual bool	is_table_printer_compatible () const
virtual void	register_example (const flexible_type &target, const flexible_type &prediction, size_t thread_id=0)=0
virtual void	register_unmapped_example (const size_t &target, const size_t &prediction, size_t thread_id=0)
virtual variant_type	get_metric ()=0

Detailed Description

Interface for performing evaluation in a streaming manner for supervised learning.

Background: Evaluation

An evaluation that can be computed in a streaming manner. All it needs is an aggregation over a sequence of individual statistics computed from individual evaluations.

What we need for a supervised evaluation scheme.

The interface makes sure that you can implement various types of streaming evaluations.

Each standardization scheme requires the following methods:

*) init: Initialize the state

*) register_example: Register a label and a prediction

*) get_metric: Final transformation required. eg. square root for rmse.

Definition at line 223 of file evaluation_interface-inl.hpp.

Constructor & Destructor Documentation

~supervised_evaluation_interface()

virtual turi::evaluation::supervised_evaluation_interface::~supervised_evaluation_interface ( )

virtualdefault

Default destructor.

supervised_evaluation_interface()

turi::evaluation::supervised_evaluation_interface::supervised_evaluation_interface ( )

default

Default constructor.

Member Function Documentation

get_metric()

virtual variant_type turi::evaluation::supervised_evaluation_interface::get_metric ( )

pure virtual

Init the state with a variant type.

Parameters

[in]

_state

Starting state.

Implemented in turi::evaluation::roc_curve, turi::evaluation::flexible_accuracy, turi::evaluation::recall, turi::evaluation::precision, turi::evaluation::fbeta_score, turi::evaluation::precision_recall_base, turi::evaluation::confusion_matrix, turi::evaluation::classifier_accuracy, turi::evaluation::max_error, and turi::evaluation::rmse.

init()

virtual void turi::evaluation::supervised_evaluation_interface::init ( size_t  _n_threads = 1 )

pure virtual

Init the state with n_threads.

Parameters

[in]

n_threads

Number of threads.

Implemented in turi::evaluation::roc_curve, turi::evaluation::confusion_matrix, turi::evaluation::classifier_accuracy, turi::evaluation::max_error, and turi::evaluation::rmse.

is_prob_evaluator()

virtual bool turi::evaluation::supervised_evaluation_interface::is_prob_evaluator ( ) const

inlinevirtual

Returns true of this evaluator works on probabilities/scores (vs) classes.

Reimplemented in turi::evaluation::roc_curve.

Definition at line 253 of file evaluation_interface-inl.hpp.

is_table_printer_compatible()

virtual bool turi::evaluation::supervised_evaluation_interface::is_table_printer_compatible ( ) const

inlinevirtual

Returns true of this evaluator can be displayed as a single float value.

Reimplemented in turi::evaluation::roc_curve, turi::evaluation::precision_recall_base, and turi::evaluation::confusion_matrix.

Definition at line 260 of file evaluation_interface-inl.hpp.

name()

virtual std::string turi::evaluation::supervised_evaluation_interface::name ( ) const

pure virtual

Name of the evaluator.

Implemented in turi::evaluation::roc_curve, turi::evaluation::flexible_accuracy, turi::evaluation::recall, turi::evaluation::precision, turi::evaluation::fbeta_score, turi::evaluation::precision_recall_base, turi::evaluation::confusion_matrix, turi::evaluation::classifier_accuracy, turi::evaluation::max_error, and turi::evaluation::rmse.

register_example()

virtual void turi::evaluation::supervised_evaluation_interface::register_example ( const flexible_type &  target, const flexible_type &  prediction, size_t  thread_id = 0  )

pure virtual

Register a (target, prediction) pair

Parameters

[in]	target	Target of a simple example.
[in]	prediction	Prediction of a single example.
[in]	thread_id	Thread id registering this example.

Implemented in turi::evaluation::roc_curve, turi::evaluation::confusion_matrix, turi::evaluation::classifier_accuracy, turi::evaluation::max_error, and turi::evaluation::rmse.

register_unmapped_example()

virtual void turi::evaluation::supervised_evaluation_interface::register_unmapped_example ( const size_t &  target, const size_t &  prediction, size_t  thread_id = 0  )

inlinevirtual

Register an unmapped (target, prediction) pair. Use this for performance only. Here the target and prediction are assumed to be integers to avoid flexible_type comparisons and flexible_type hashing.

Parameters

[in]	target	Target of a simple example.
[in]	prediction	Prediction of a single example.
[in]	thread_id	Thread id registering this example.

Reimplemented in turi::evaluation::classifier_accuracy.

Definition at line 287 of file evaluation_interface-inl.hpp.

---

The documentation for this class was generated from the following file:

• toolkits/evaluation/evaluation_interface-inl.hpp