Turi Create  4.0
Public Member Functions | List of all members
turi::parallel_sframe_iterator_initializer Class Reference
SFrame : Physical » Main SFrame Objects

#include <core/storage/sframe_data/sframe_iterators.hpp>

Public Member Functions

 parallel_sframe_iterator_initializer (sframe data, const size_t &_row_start=0, const size_t &_row_end=-1)
 
 parallel_sframe_iterator_initializer (const std::vector< sframe > &data_sources, const size_t &_row_start=0, const size_t &_row_end=-1)
 
void set_global_block (size_t _row_start=0, size_t _row_end=-1)
 

Detailed Description

Utlity to provide parallel iteration over an SFrame

Definition at line 27 of file sframe_iterators.hpp.

Constructor & Destructor Documentation

◆ parallel_sframe_iterator_initializer() [1/2]

turi::parallel_sframe_iterator_initializer::parallel_sframe_iterator_initializer ( sframe  data,
const size_t &  _row_start = 0,
const size_t &  _row_end = -1 
)
inlineexplicit

Create an sframe iterator initializer initialized with a single sframe. The sframe is divided into num_threads blocks of approximately equal size. This iterator claims the thread_idx block.

Parameters
[in]data_sourcesCollection of SFrames
[in]row_startFirst row to read
[in]row_endOne past last row to read (i.e. EXCLUSIVE). Row_end can be beyond the end of the array, in which case, fewer rows will be read. Default -1 reads all rows.

Definition at line 43 of file sframe_iterators.hpp.

◆ parallel_sframe_iterator_initializer() [2/2]

turi::parallel_sframe_iterator_initializer::parallel_sframe_iterator_initializer ( const std::vector< sframe > &  data_sources,
const size_t &  _row_start = 0,
const size_t &  _row_end = -1 
)
explicit

Initialize the sframe iterator with a vector of sframes. Each sframe is divided into num_threads blocks of approximately equal size. This iterator claims the thread_idx block.

With multiple sframes, elements in the current row can be accessed by it.value(sframe_index, column_index), where sframe_index refers to the index in data_sources, and column_index refers to the index of the column within that sframe.

Parameters
[in]data_sourcesCollection of SFrames
[in]row_startFirst row to read
[in]row_endOne past last row to read (i.e. EXCLUSIVE). Row_end can be beyond the end of the array, in which case, fewer rows will be read. Default -1 reads all rows.

Member Function Documentation

◆ set_global_block()

void turi::parallel_sframe_iterator_initializer::set_global_block ( size_t  _row_start = 0,
size_t  _row_end = -1 
)

Set the global block to read. This allows us to create the initializer only once and change the row_start and row_end multiple times.

Parameters
[in]row_startFirst row to read
[in]row_endOne past last row to read (i.e. EXCLUSIVE). Row_end can be beyond the end of the array, in which case, fewer rows will be read. Default -1 reads all rows.
The documentation for this class was generated from the following file: