turi::gl_sarray_writer Class Reference

#include <core/data/sframe/gl_sarray.hpp>

Public Member Functions
	gl_sarray_writer (flex_type_enum type, size_t num_segments=(size_t)(-1))
void	write (const flexible_type &f, size_t segmentid)
template<typename T >
void	write (T begin, T end, size_t segmentid)
gl_sarray	close ()
size_t	num_segments () const

Detailed Description

Provides the ability to write gl_sarrays. The gl_sarray is internally cut into a collection of segments. Each segment can be written to independently, and the resultant SArray is the effective concatenation of all the segments.

// Writes an integer SArray of 4 segments.
gl_sarray_writer writer(flex_type_enum:INTEGER, 4);

// for each segment, write a bunch of 10 values.
// segment 0 has 10 0's,
// segment 1 has 10 1's,
// etc
for (size_t seg = 0;seg < 4; ++seg) {
  for (size_t i = 0;i < 10; ++i) {
    writer.write(i, seg);
  }
}

gl_sarray sa = writer.close();
// sa is now an SArray of 40 elements comprising of
// the sequence 10 0's, 10 1's, 10 2's, 10 3's

Different segments can be written safely in parallel. It is not safe to write to the same segment simultanously.

Definition at line 2009 of file gl_sarray.hpp.

Constructor & Destructor Documentation

gl_sarray_writer()

turi::gl_sarray_writer::gl_sarray_writer	(	flex_type_enum	type,
		size_t	num_segments = (size_t)(-1)
	)

Constructs a writer to write an gl_sarray of a particular type.

Parameters

type	The content type of the SArray. Everything written to the writer (via write) must be of that type, is implicitly castable to that type, or is a missing value denoted with a FLEX_UNDEFINED value.
num_segments	Optional. The number of segments of the SArray. Adjusting this parameter has little performance impact on the resultant gl_sarray. Modifying this value is only helpful for providing writer parallelism. Defaults to the number of cores on the machine.

Member Function Documentation

close()

gl_sarray turi::gl_sarray_writer::close

(

)

Stops all writes and returns the resultant SArray.

num_segments()

size_t turi::gl_sarray_writer::num_segments

(

)

const

Returns the number of segments of the SArray; this is the same value provided on construction of the writer.

write() [1/2]

void turi::gl_sarray_writer::write	(	const flexible_type &	f,
		size_t	segmentid
	)

Writes a single value to a given segment.

For instance,

gl_sarray_writer writer(flex_type_enum:FLOAT, 1);
writer.write(1.5, 0); // writes the value 1.5 to segment 0
writer.write(1, 0); // writes the value 1.0 to segment 0 (integer can be cast to float)

Strings are the most general type and everything can cast to it. hence:

gl_sarray_writer writer(flex_type_enum:STRING, 1);
writer.write("hello", 0); // writes the value "hello" to segment 0
writer.write(1.5, 0); // writes the value "1.5" to segment 0
writer.write(1, 0); // writes the value "1" to segment 0

Different segments can be written safely in parallel. It is not safe to write to the same segment simultanously.

Parameters

f	The value to write. This value should be of the requested type (as set in the constructor), or is castable to the requested type, or is FLEX_UNDEFINED.
segmentid	The segment to write to.

write() [2/2]

template<typename T >

void turi::gl_sarray_writer::write ( T  begin, T  end, size_t  segmentid  )

inline

Writes a range of values to a given segment.

Essentially equivalent to:

while(start != end) write(*start++);

Different segments can be written safely in parallel. It is not safe to write to the same segment simultanously.

Parameters

start	The start iterator of the range to write.
end	The end iterator of the range to write.
segmentid	The segment to write to.

Definition at line 2072 of file gl_sarray.hpp.

---

The documentation for this class was generated from the following file:

• core/data/sframe/gl_sarray.hpp