NonBlockingFileIO
public struct NonBlockingFileIO
NonBlockingFileIO
is a helper that allows you to read files without blocking the calling thread.
It is worth noting that kqueue
, epoll
or poll
returning claiming a file is readable does not mean that the
data is already available in the kernel’s memory. In other words, a read
from a file can still block even if
reported as readable. This behaviour is also documented behaviour:
poll
: “Regular files shall always poll TRUE for reading and writing.”epoll
: “epoll is simply a faster poll(2), and can be used wherever the latter is used since it shares the same semantics.”kqueue
: “Returns when the file pointer is not at the end of file.”
NonBlockingFileIO
helps to work around this issue by maintaining its own thread pool that is used to read the data
from the files into memory. It will then hand the (in-memory) data back which makes it available without the possibility
of blocking.
-
The default and recommended size for
NonBlockingFileIO
‘s thread pool.Declaration
Swift
public static let defaultThreadPoolSize: Int
-
The default and recommended chunk size.
Declaration
Swift
public static let defaultChunkSize: Int
-
See moreNonBlockingFileIO
errors.Declaration
Swift
public enum Error : Swift.Error
-
Initialize a
NonBlockingFileIO
which uses theNIOThreadPool
.Declaration
Swift
public init(threadPool: NIOThreadPool)
Parameters
threadPool
The
NIOThreadPool
that will be used for all the IO. -
Read a
FileRegion
in chunks ofchunkSize
bytes onNonBlockingFileIO
‘s private thread pool which is separate from anyEventLoop
thread.chunkHandler
will be called oneventLoop
for every chunk that was read. AssumingfileRegion.readableBytes
is greater than zero and there are enough bytes availablechunkHandler
will be called1 + |_ fileRegion.readableBytes / chunkSize _|
times, deliveringchunkSize
bytes each time. If less thanfileRegion.readableBytes
bytes can be read from the file,chunkHandler
will be called less often with the last invocation possibly being of less thanchunkSize
bytes.The allocation and reading of a subsequent chunk will only be attempted when
chunkHandler
succeeds.This method will not use the file descriptor’s seek pointer which means there is no danger of reading from the same
FileRegion
in multiple threads.Declaration
Swift
public func readChunked(fileRegion: FileRegion, chunkSize: Int = NonBlockingFileIO.defaultChunkSize, allocator: ByteBufferAllocator, eventLoop: EventLoop, chunkHandler: @escaping (ByteBuffer) -> EventLoopFuture<Void>) -> EventLoopFuture<Void>
Parameters
fileRegion
The file region to read.
chunkSize
The size of the individual chunks to deliver.
allocator
A
ByteBufferAllocator
used to allocate space for the chunks.eventLoop
The
EventLoop
to callchunkHandler
on.chunkHandler
Called for every chunk read. The next chunk will be read upon successful completion of the returned
EventLoopFuture
. If the returnedEventLoopFuture
fails, the overall operation is aborted.Return Value
An
EventLoopFuture
which is the result of the overall operation. If either the reading offileHandle
orchunkHandler
fails, theEventLoopFuture
will fail too. If the reading offileHandle
as well aschunkHandler
always succeeded, theEventLoopFuture
will succeed too. -
Read
byteCount
bytes in chunks ofchunkSize
bytes fromfileHandle
inNonBlockingFileIO
‘s private thread pool which is separate from anyEventLoop
thread.chunkHandler
will be called oneventLoop
for every chunk that was read. AssumingbyteCount
is greater than zero and there are enough bytes availablechunkHandler
will be called1 + |_ byteCount / chunkSize _|
times, deliveringchunkSize
bytes each time. If less thanbyteCount
bytes can be read fromdescriptor
,chunkHandler
will be called less often with the last invocation possibly being of less thanchunkSize
bytes.The allocation and reading of a subsequent chunk will only be attempted when
chunkHandler
succeeds.Note
readChunked(fileRegion:chunkSize:allocator:eventLoop:chunkHandler:)
should be preferred as it usesFileRegion
object instead of rawNIOFileHandle
s. In case you do want to use rawNIOFileHandle
s, please consider usingreadChunked(fileHandle:fromOffset:chunkSize:allocator:eventLoop:chunkHandler:)
because it doesn’t use the file descriptor’s seek pointer (which may be shared with other file descriptors and even across processes.)Declaration
Swift
public func readChunked(fileHandle: NIOFileHandle, byteCount: Int, chunkSize: Int = NonBlockingFileIO.defaultChunkSize, allocator: ByteBufferAllocator, eventLoop: EventLoop, chunkHandler: @escaping (ByteBuffer) -> EventLoopFuture<Void>) -> EventLoopFuture<Void>
Parameters
fileHandle
The
NIOFileHandle
to read from.byteCount
The number of bytes to read from
fileHandle
.chunkSize
The size of the individual chunks to deliver.
allocator
A
ByteBufferAllocator
used to allocate space for the chunks.eventLoop
The
EventLoop
to callchunkHandler
on.chunkHandler
Called for every chunk read. The next chunk will be read upon successful completion of the returned
EventLoopFuture
. If the returnedEventLoopFuture
fails, the overall operation is aborted.Return Value
An
EventLoopFuture
which is the result of the overall operation. If either the reading offileHandle
orchunkHandler
fails, theEventLoopFuture
will fail too. If the reading offileHandle
as well aschunkHandler
always succeeded, theEventLoopFuture
will succeed too. -
Read
byteCount
bytes from offsetfileOffset
in chunks ofchunkSize
bytes fromfileHandle
inNonBlockingFileIO
‘s private thread pool which is separate from anyEventLoop
thread.chunkHandler
will be called oneventLoop
for every chunk that was read. AssumingbyteCount
is greater than zero and there are enough bytes availablechunkHandler
will be called1 + |_ byteCount / chunkSize _|
times, deliveringchunkSize
bytes each time. If less thanbyteCount
bytes can be read fromdescriptor
,chunkHandler
will be called less often with the last invocation possibly being of less thanchunkSize
bytes.The allocation and reading of a subsequent chunk will only be attempted when
chunkHandler
succeeds.This method will not use the file descriptor’s seek pointer which means there is no danger of reading from the same
NIOFileHandle
in multiple threads.Note
readChunked(fileRegion:chunkSize:allocator:eventLoop:chunkHandler:)
should be preferred as it usesFileRegion
object instead of rawNIOFileHandle
s.Declaration
Swift
public func readChunked(fileHandle: NIOFileHandle, fromOffset fileOffset: Int64, byteCount: Int, chunkSize: Int = NonBlockingFileIO.defaultChunkSize, allocator: ByteBufferAllocator, eventLoop: EventLoop, chunkHandler: @escaping (ByteBuffer) -> EventLoopFuture<Void>) -> EventLoopFuture<Void>
Parameters
fileHandle
The
NIOFileHandle
to read from.byteCount
The number of bytes to read from
fileHandle
.chunkSize
The size of the individual chunks to deliver.
allocator
A
ByteBufferAllocator
used to allocate space for the chunks.eventLoop
The
EventLoop
to callchunkHandler
on.chunkHandler
Called for every chunk read. The next chunk will be read upon successful completion of the returned
EventLoopFuture
. If the returnedEventLoopFuture
fails, the overall operation is aborted.Return Value
An
EventLoopFuture
which is the result of the overall operation. If either the reading offileHandle
orchunkHandler
fails, theEventLoopFuture
will fail too. If the reading offileHandle
as well aschunkHandler
always succeeded, theEventLoopFuture
will succeed too. -
Read a
FileRegion
inNonBlockingFileIO
‘s private thread pool which is separate from anyEventLoop
thread.The returned
ByteBuffer
will not have less thanfileRegion.readableBytes
unless we hit end-of-file in which case theByteBuffer
will contain the bytes available to read.This method will not use the file descriptor’s seek pointer which means there is no danger of reading from the same
FileRegion
in multiple threads.Note
Only use this function for small enoughFileRegion
s as it will need to allocate enough memory to holdfileRegion.readableBytes
bytes.Note
In most cases you should prefer one of the
readChunked
functions.Declaration
Swift
public func read(fileRegion: FileRegion, allocator: ByteBufferAllocator, eventLoop: EventLoop) -> EventLoopFuture<ByteBuffer>
Parameters
fileRegion
The file region to read.
allocator
A
ByteBufferAllocator
used to allocate space for the returnedByteBuffer
.eventLoop
The
EventLoop
to create the returnedEventLoopFuture
from.Return Value
An
EventLoopFuture
which delivers aByteBuffer
if the read was successful or a failure on error. -
Read
byteCount
bytes fromfileHandle
inNonBlockingFileIO
‘s private thread pool which is separate from anyEventLoop
thread.The returned
ByteBuffer
will not have less thanbyteCount
bytes unless we hit end-of-file in which case theByteBuffer
will contain the bytes available to read.Note
Only use this function for small enoughbyteCount
s as it will need to allocate enough memory to holdbyteCount
bytes.Note
read(fileRegion:allocator:eventLoop:)
should be preferred as it usesFileRegion
object instead of rawNIOFileHandle
s. In case you do want to use rawNIOFileHandle
s, please consider usingread(fileHandle:fromOffset:byteCount:allocator:eventLoop:)
because it doesn’t use the file descriptor’s seek pointer (which may be shared with other file descriptors and even across processes.)Declaration
Swift
public func read(fileHandle: NIOFileHandle, byteCount: Int, allocator: ByteBufferAllocator, eventLoop: EventLoop) -> EventLoopFuture<ByteBuffer>
Parameters
fileHandle
The
NIOFileHandle
to read.byteCount
The number of bytes to read from
fileHandle
.allocator
A
ByteBufferAllocator
used to allocate space for the returnedByteBuffer
.eventLoop
The
EventLoop
to create the returnedEventLoopFuture
from.Return Value
An
EventLoopFuture
which delivers aByteBuffer
if the read was successful or a failure on error. -
Read
byteCount
bytes starting atfileOffset
fromfileHandle
inNonBlockingFileIO
‘s private thread pool which is separate from anyEventLoop
thread.The returned
ByteBuffer
will not have less thanbyteCount
bytes unless we hit end-of-file in which case theByteBuffer
will contain the bytes available to read.This method will not use the file descriptor’s seek pointer which means there is no danger of reading from the same
fileHandle
in multiple threads.Note
Only use this function for small enoughbyteCount
s as it will need to allocate enough memory to holdbyteCount
bytes.Note
read(fileRegion:allocator:eventLoop:)
should be preferred as it usesFileRegion
object instead of rawNIOFileHandle
s.Declaration
Swift
public func read(fileHandle: NIOFileHandle, fromOffset fileOffset: Int64, byteCount: Int, allocator: ByteBufferAllocator, eventLoop: EventLoop) -> EventLoopFuture<ByteBuffer>
Parameters
fileHandle
The
NIOFileHandle
to read.fileOffset
The offset to read from.
byteCount
The number of bytes to read from
fileHandle
.allocator
A
ByteBufferAllocator
used to allocate space for the returnedByteBuffer
.eventLoop
The
EventLoop
to create the returnedEventLoopFuture
from.Return Value
An
EventLoopFuture
which delivers aByteBuffer
if the read was successful or a failure on error. -
Changes the file size of
fileHandle
tosize
.If
size
is smaller than the current file size, the remaining bytes will be truncated and are lost. Ifsize
is larger than the current file size, the gap will be filled with zero bytes.Declaration
Swift
public func changeFileSize(fileHandle: NIOFileHandle, size: Int64, eventLoop: EventLoop) -> EventLoopFuture<()>
Parameters
fileHandle
The
NIOFileHandle
to write to.size
The new file size in bytes to write.
eventLoop
The
EventLoop
to create the returnedEventLoopFuture
from.Return Value
An
EventLoopFuture
which is fulfilled if the write was successful or fails on error. -
Returns the length of the file associated with
fileHandle
.Declaration
Swift
public func readFileSize(fileHandle: NIOFileHandle, eventLoop: EventLoop) -> EventLoopFuture<Int64>
Parameters
fileHandle
The
NIOFileHandle
to read from.eventLoop
The
EventLoop
to create the returnedEventLoopFuture
from.Return Value
An
EventLoopFuture
which is fulfilled if the write was successful or fails on error. -
Write
buffer
tofileHandle
inNonBlockingFileIO
‘s private thread pool which is separate from anyEventLoop
thread.Declaration
Swift
public func write(fileHandle: NIOFileHandle, buffer: ByteBuffer, eventLoop: EventLoop) -> EventLoopFuture<()>
Parameters
fileHandle
The
NIOFileHandle
to write to.buffer
The
ByteBuffer
to write.eventLoop
The
EventLoop
to create the returnedEventLoopFuture
from.Return Value
An
EventLoopFuture
which is fulfilled if the write was successful or fails on error. -
Write
buffer
starting fromtoOffset
tofileHandle
inNonBlockingFileIO
‘s private thread pool which is separate from anyEventLoop
thread.Declaration
Swift
public func write(fileHandle: NIOFileHandle, toOffset: Int64, buffer: ByteBuffer, eventLoop: EventLoop) -> EventLoopFuture<()>
Parameters
fileHandle
The
NIOFileHandle
to write to.toOffset
The file offset to write to.
buffer
The
ByteBuffer
to write.eventLoop
The
EventLoop
to create the returnedEventLoopFuture
from.Return Value
An
EventLoopFuture
which is fulfilled if the write was successful or fails on error. -
Open the file at
path
for reading on a private thread pool which is separate from anyEventLoop
thread.This function will return (a future) of the
NIOFileHandle
associated with the file opened and aFileRegion
comprising of the whole file. The caller must close the returnedNIOFileHandle
when it’s no longer needed.Note
The reason this returns the
NIOFileHandle
and theFileRegion
is that both the opening of a file as well as the querying of its size are blocking.Declaration
Swift
public func openFile(path: String, eventLoop: EventLoop) -> EventLoopFuture<(NIOFileHandle, FileRegion)>
Parameters
path
The path of the file to be opened for reading.
eventLoop
The
EventLoop
on which the returnedEventLoopFuture
will fire.Return Value
An
EventLoopFuture
containing theNIOFileHandle
and theFileRegion
comprising the whole file. -
Open the file at
path
with specified access mode and POSIX flags on a private thread pool which is separate from anyEventLoop
thread.This function will return (a future) of the
NIOFileHandle
associated with the file opened. The caller must close the returnedNIOFileHandle
when it’s no longer needed.Declaration
Swift
public func openFile(path: String, mode: NIOFileHandle.Mode, flags: NIOFileHandle.Flags = .default, eventLoop: EventLoop) -> EventLoopFuture<NIOFileHandle>
Parameters
path
The path of the file to be opened for writing.
mode
File access mode.
flags
Additional POSIX flags.
eventLoop
The
EventLoop
on which the returnedEventLoopFuture
will fire.Return Value
An
EventLoopFuture
containing theNIOFileHandle
.