turi::gl_sframe_writer Class Reference

#include <core/data/sframe/gl_sframe.hpp>

Public Member Functions
	gl_sframe_writer (const std::vector< std::string > &column_names, const std::vector< flex_type_enum > &column_types, size_t num_segments=(size_t)(-1))
void	write (const std::vector< flexible_type > &f, size_t segmentid)
template<typename T >
void	write (T begin, T end, size_t segmentid)
gl_sframe	close ()
size_t	num_segments () const

Detailed Description

Provides the ability to write gl_sframe. The gl_sframe is internally cut into a collection of segments. Each segment can be written to independently, and the resultant SFrame is the effective concatenation of all the segments.

// Writes an SFrame of 4 segments, and 2 columns "a" and "b", both of which
// are integers.
gl_sframe_writer writer({"a","b"},
                        {flex_type_enum:INTEGER, flex_type_enum::INTEGER},
                        4);

// for each segment, write a bunch of (i, i) pair values.
// segment 0 has 10 0's,
// segment 1 has 10 1's,
// etc
for (size_t seg = 0;seg < 4; ++seg) {
  for (size_t i = 0;i < 10; ++i) {
    writer.write({i, i}, seg);
  }
}

gl_sframe sa = writer.close();
// sa is now an SFrame of 40 elements comprising of
// four consecutive sequences of (1,1) to (10,10)

Different segments can be written safely in parallel. It is not safe to write to the same segment simultanously.

Definition at line 2997 of file gl_sframe.hpp.

Constructor & Destructor Documentation

gl_sframe_writer()

turi::gl_sframe_writer::gl_sframe_writer	(	const std::vector< std::string > &	column_names,
		const std::vector< flex_type_enum > &	column_types,
		size_t	num_segments = (size_t)(-1)
	)

Constructs a writer to write an gl_sarray of a particular type.

Parameters

column_name	The column names of the SFrame.
type	The type of each column of the SFrame. Everything written to the writer (via write) must be of that type, is implicitly castable to that type, or is a missing value denoted with a FLEX_UNDEFINED value.
num_segments	Optional. The number of segments of the SFrame. Adjusting this parameter has little performance impact on the resultant gl_sframe. Modifying this value is only helpful for providing writer parallelism. Defaults to the number of cores on the machine.

Member Function Documentation

close()

gl_sframe turi::gl_sframe_writer::close

(

)

Stops all writes and returns the resultant SFrame.

num_segments()

size_t turi::gl_sframe_writer::num_segments

(

)

const

Returns the number of segments of the Aarray; this is the same value provided on construction of the writer.

write() [1/2]

void turi::gl_sframe_writer::write	(	const std::vector< flexible_type > &	f,
		size_t	segmentid
	)

Writes a single value to a given segment.

For instance,

gl_sframe_writer({"a","b"}, {flex_type_enum:FLOAT, flex_type_enum::STRING}, 1);
writer.write({1.5, "hello"}, 0);

Different segments can be written safely in parallel. It is not safe to write to the same segment simultanously.

Parameters

f	The value to write. This value should be of an array of the requested typse (as set in the constructor), or is castable to the requested type, or is FLEX_UNDEFINED.
segmentid	The segment to write to.

write() [2/2]

template<typename T >

void turi::gl_sframe_writer::write ( T  begin, T  end, size_t  segmentid  )

inline

Writes a range of values to a given segment.

Essentially equivalent to:

while(start != end) write(*start++);

Different segments can be written safely in parallel. It is not safe to write to the same segment simultanously.

Parameters

start	The start iterator of the range to write.
end	The end iterator of the range to write.
segmentid	The segment to write to.

Definition at line 3055 of file gl_sframe.hpp.

---

The documentation for this class was generated from the following file:

• core/data/sframe/gl_sframe.hpp