NonBlockingFileIO

public struct NonBlockingFileIO

NonBlockingFileIO is a helper that allows you to read files without blocking the calling thread.

It is worth noting that kqueue, epoll or poll returning claiming a file is readable does not mean that the data is already available in the kernel’s memory. In other words, a read from a file can still block even if reported as readable. This behaviour is also documented behaviour:

  • poll: “Regular files shall always poll TRUE for reading and writing.”
  • epoll: “epoll is simply a faster poll(2), and can be used wherever the latter is used since it shares the same semantics.”
  • kqueue: “Returns when the file pointer is not at the end of file.”

NonBlockingFileIO helps to work around this issue by maintaining its own thread pool that is used to read the data from the files into memory. It will then hand the (in-memory) data back which makes it available without the possibility of blocking.

  • The default and recommended size for NonBlockingFileIO‘s thread pool.

    Declaration

    Swift

    public static let defaultThreadPoolSize: Int
  • The default and recommended chunk size.

    Declaration

    Swift

    public static let defaultChunkSize: Int
  • Declaration

    Swift

    public enum Error : Swift.Error
  • Initialize a NonBlockingFileIO which uses the NIOThreadPool.

    Declaration

    Swift

    public init(threadPool: NIOThreadPool)

    Parameters

    threadPool

    The NIOThreadPool that will be used for all the IO.

  • Read a FileRegion in chunks of chunkSize bytes on NonBlockingFileIO‘s private thread pool which is separate from any EventLoop thread.

    chunkHandler will be called on eventLoop for every chunk that was read. Assuming fileRegion.readableBytes is greater than zero and there are enough bytes available chunkHandler will be called 1 + |_ fileRegion.readableBytes / chunkSize _| times, delivering chunkSize bytes each time. If less than fileRegion.readableBytes bytes can be read from the file, chunkHandler will be called less often with the last invocation possibly being of less than chunkSize bytes.

    The allocation and reading of a subsequent chunk will only be attempted when chunkHandler succeeds.

    This method will not use the file descriptor’s seek pointer which means there is no danger of reading from the same FileRegion in multiple threads.

    Declaration

    Swift

    public func readChunked(fileRegion: FileRegion,
                            chunkSize: Int = NonBlockingFileIO.defaultChunkSize,
                            allocator: ByteBufferAllocator,
                            eventLoop: EventLoop,
                            chunkHandler: @escaping (ByteBuffer) -> EventLoopFuture<Void>) -> EventLoopFuture<Void>

    Parameters

    fileRegion

    The file region to read.

    chunkSize

    The size of the individual chunks to deliver.

    allocator

    A ByteBufferAllocator used to allocate space for the chunks.

    eventLoop

    The EventLoop to call chunkHandler on.

    chunkHandler

    Called for every chunk read. The next chunk will be read upon successful completion of the returned EventLoopFuture. If the returned EventLoopFuture fails, the overall operation is aborted.

    Return Value

    An EventLoopFuture which is the result of the overall operation. If either the reading of fileHandle or chunkHandler fails, the EventLoopFuture will fail too. If the reading of fileHandle as well as chunkHandler always succeeded, the EventLoopFuture will succeed too.

  • Read byteCount bytes in chunks of chunkSize bytes from fileHandle in NonBlockingFileIO‘s private thread pool which is separate from any EventLoop thread.

    chunkHandler will be called on eventLoop for every chunk that was read. Assuming byteCount is greater than zero and there are enough bytes available chunkHandler will be called 1 + |_ byteCount / chunkSize _| times, delivering chunkSize bytes each time. If less than byteCount bytes can be read from descriptor, chunkHandler will be called less often with the last invocation possibly being of less than chunkSize bytes.

    The allocation and reading of a subsequent chunk will only be attempted when chunkHandler succeeds.

    Note

    readChunked(fileRegion:chunkSize:allocator:eventLoop:chunkHandler:) should be preferred as it uses FileRegion object instead of raw NIOFileHandles. In case you do want to use raw NIOFileHandles, please consider using readChunked(fileHandle:fromOffset:chunkSize:allocator:eventLoop:chunkHandler:) because it doesn’t use the file descriptor’s seek pointer (which may be shared with other file descriptors and even across processes.)

    Declaration

    Swift

    public func readChunked(fileHandle: NIOFileHandle,
                            byteCount: Int,
                            chunkSize: Int = NonBlockingFileIO.defaultChunkSize,
                            allocator: ByteBufferAllocator,
                            eventLoop: EventLoop, chunkHandler: @escaping (ByteBuffer) -> EventLoopFuture<Void>) -> EventLoopFuture<Void>

    Parameters

    fileHandle

    The NIOFileHandle to read from.

    byteCount

    The number of bytes to read from fileHandle.

    chunkSize

    The size of the individual chunks to deliver.

    allocator

    A ByteBufferAllocator used to allocate space for the chunks.

    eventLoop

    The EventLoop to call chunkHandler on.

    chunkHandler

    Called for every chunk read. The next chunk will be read upon successful completion of the returned EventLoopFuture. If the returned EventLoopFuture fails, the overall operation is aborted.

    Return Value

    An EventLoopFuture which is the result of the overall operation. If either the reading of fileHandle or chunkHandler fails, the EventLoopFuture will fail too. If the reading of fileHandle as well as chunkHandler always succeeded, the EventLoopFuture will succeed too.

  • Read byteCount bytes from offset fileOffset in chunks of chunkSize bytes from fileHandle in NonBlockingFileIO‘s private thread pool which is separate from any EventLoop thread.

    chunkHandler will be called on eventLoop for every chunk that was read. Assuming byteCount is greater than zero and there are enough bytes available chunkHandler will be called 1 + |_ byteCount / chunkSize _| times, delivering chunkSize bytes each time. If less than byteCount bytes can be read from descriptor, chunkHandler will be called less often with the last invocation possibly being of less than chunkSize bytes.

    The allocation and reading of a subsequent chunk will only be attempted when chunkHandler succeeds.

    This method will not use the file descriptor’s seek pointer which means there is no danger of reading from the same NIOFileHandle in multiple threads.

    Note

    readChunked(fileRegion:chunkSize:allocator:eventLoop:chunkHandler:) should be preferred as it uses FileRegion object instead of raw NIOFileHandles.

    Declaration

    Swift

    public func readChunked(fileHandle: NIOFileHandle,
                            fromOffset fileOffset: Int64,
                            byteCount: Int,
                            chunkSize: Int = NonBlockingFileIO.defaultChunkSize,
                            allocator: ByteBufferAllocator,
                            eventLoop: EventLoop,
                            chunkHandler: @escaping (ByteBuffer) -> EventLoopFuture<Void>) -> EventLoopFuture<Void>

    Parameters

    fileHandle

    The NIOFileHandle to read from.

    byteCount

    The number of bytes to read from fileHandle.

    chunkSize

    The size of the individual chunks to deliver.

    allocator

    A ByteBufferAllocator used to allocate space for the chunks.

    eventLoop

    The EventLoop to call chunkHandler on.

    chunkHandler

    Called for every chunk read. The next chunk will be read upon successful completion of the returned EventLoopFuture. If the returned EventLoopFuture fails, the overall operation is aborted.

    Return Value

    An EventLoopFuture which is the result of the overall operation. If either the reading of fileHandle or chunkHandler fails, the EventLoopFuture will fail too. If the reading of fileHandle as well as chunkHandler always succeeded, the EventLoopFuture will succeed too.

  • Read a FileRegion in NonBlockingFileIO‘s private thread pool which is separate from any EventLoop thread.

    The returned ByteBuffer will not have less than fileRegion.readableBytes unless we hit end-of-file in which case the ByteBuffer will contain the bytes available to read.

    This method will not use the file descriptor’s seek pointer which means there is no danger of reading from the same FileRegion in multiple threads.

    Note

    Only use this function for small enough FileRegions as it will need to allocate enough memory to hold fileRegion.readableBytes bytes.

    Note

    In most cases you should prefer one of the readChunked functions.

    Declaration

    Swift

    public func read(fileRegion: FileRegion, allocator: ByteBufferAllocator, eventLoop: EventLoop) -> EventLoopFuture<ByteBuffer>

    Parameters

    fileRegion

    The file region to read.

    allocator

    A ByteBufferAllocator used to allocate space for the returned ByteBuffer.

    eventLoop

    The EventLoop to create the returned EventLoopFuture from.

    Return Value

    An EventLoopFuture which delivers a ByteBuffer if the read was successful or a failure on error.

  • Read byteCount bytes from fileHandle in NonBlockingFileIO‘s private thread pool which is separate from any EventLoop thread.

    The returned ByteBuffer will not have less than byteCount bytes unless we hit end-of-file in which case the ByteBuffer will contain the bytes available to read.

    Note

    Only use this function for small enough byteCounts as it will need to allocate enough memory to hold byteCount bytes.

    Note

    read(fileRegion:allocator:eventLoop:) should be preferred as it uses FileRegion object instead of raw NIOFileHandles. In case you do want to use raw NIOFileHandles, please consider using read(fileHandle:fromOffset:byteCount:allocator:eventLoop:) because it doesn’t use the file descriptor’s seek pointer (which may be shared with other file descriptors and even across processes.)

    Declaration

    Swift

    public func read(fileHandle: NIOFileHandle,
                     byteCount: Int,
                     allocator: ByteBufferAllocator,
                     eventLoop: EventLoop) -> EventLoopFuture<ByteBuffer>

    Parameters

    fileHandle

    The NIOFileHandle to read.

    byteCount

    The number of bytes to read from fileHandle.

    allocator

    A ByteBufferAllocator used to allocate space for the returned ByteBuffer.

    eventLoop

    The EventLoop to create the returned EventLoopFuture from.

    Return Value

    An EventLoopFuture which delivers a ByteBuffer if the read was successful or a failure on error.

  • Read byteCount bytes starting at fileOffset from fileHandle in NonBlockingFileIO‘s private thread pool which is separate from any EventLoop thread.

    The returned ByteBuffer will not have less than byteCount bytes unless we hit end-of-file in which case the ByteBuffer will contain the bytes available to read.

    This method will not use the file descriptor’s seek pointer which means there is no danger of reading from the same fileHandle in multiple threads.

    Note

    Only use this function for small enough byteCounts as it will need to allocate enough memory to hold byteCount bytes.

    Note

    read(fileRegion:allocator:eventLoop:) should be preferred as it uses FileRegion object instead of raw NIOFileHandles.

    Declaration

    Swift

    public func read(fileHandle: NIOFileHandle,
                     fromOffset fileOffset: Int64,
                     byteCount: Int,
                     allocator: ByteBufferAllocator,
                     eventLoop: EventLoop) -> EventLoopFuture<ByteBuffer>

    Parameters

    fileHandle

    The NIOFileHandle to read.

    fileOffset

    The offset to read from.

    byteCount

    The number of bytes to read from fileHandle.

    allocator

    A ByteBufferAllocator used to allocate space for the returned ByteBuffer.

    eventLoop

    The EventLoop to create the returned EventLoopFuture from.

    Return Value

    An EventLoopFuture which delivers a ByteBuffer if the read was successful or a failure on error.

  • Changes the file size of fileHandle to size.

    If size is smaller than the current file size, the remaining bytes will be truncated and are lost. If size is larger than the current file size, the gap will be filled with zero bytes.

    Declaration

    Swift

    public func changeFileSize(fileHandle: NIOFileHandle,
                               size: Int64,
                               eventLoop: EventLoop) -> EventLoopFuture<()>

    Parameters

    fileHandle

    The NIOFileHandle to write to.

    size

    The new file size in bytes to write.

    eventLoop

    The EventLoop to create the returned EventLoopFuture from.

    Return Value

    An EventLoopFuture which is fulfilled if the write was successful or fails on error.

  • Returns the length of the file associated with fileHandle.

    Declaration

    Swift

    public func readFileSize(fileHandle: NIOFileHandle,
                             eventLoop: EventLoop) -> EventLoopFuture<Int64>

    Parameters

    fileHandle

    The NIOFileHandle to read from.

    eventLoop

    The EventLoop to create the returned EventLoopFuture from.

    Return Value

    An EventLoopFuture which is fulfilled if the write was successful or fails on error.

  • Write buffer to fileHandle in NonBlockingFileIO‘s private thread pool which is separate from any EventLoop thread.

    Declaration

    Swift

    public func write(fileHandle: NIOFileHandle,
                      buffer: ByteBuffer,
                      eventLoop: EventLoop) -> EventLoopFuture<()>

    Parameters

    fileHandle

    The NIOFileHandle to write to.

    buffer

    The ByteBuffer to write.

    eventLoop

    The EventLoop to create the returned EventLoopFuture from.

    Return Value

    An EventLoopFuture which is fulfilled if the write was successful or fails on error.

  • Write buffer starting from toOffset to fileHandle in NonBlockingFileIO‘s private thread pool which is separate from any EventLoop thread.

    Declaration

    Swift

    public func write(fileHandle: NIOFileHandle,
                      toOffset: Int64,
                      buffer: ByteBuffer,
                      eventLoop: EventLoop) -> EventLoopFuture<()>

    Parameters

    fileHandle

    The NIOFileHandle to write to.

    toOffset

    The file offset to write to.

    buffer

    The ByteBuffer to write.

    eventLoop

    The EventLoop to create the returned EventLoopFuture from.

    Return Value

    An EventLoopFuture which is fulfilled if the write was successful or fails on error.

  • Open the file at path for reading on a private thread pool which is separate from any EventLoop thread.

    This function will return (a future) of the NIOFileHandle associated with the file opened and a FileRegion comprising of the whole file. The caller must close the returned NIOFileHandle when it’s no longer needed.

    Note

    The reason this returns the NIOFileHandle and the FileRegion is that both the opening of a file as well as the querying of its size are blocking.

    Declaration

    Swift

    public func openFile(path: String, eventLoop: EventLoop) -> EventLoopFuture<(NIOFileHandle, FileRegion)>

    Parameters

    path

    The path of the file to be opened for reading.

    eventLoop

    The EventLoop on which the returned EventLoopFuture will fire.

    Return Value

    An EventLoopFuture containing the NIOFileHandle and the FileRegion comprising the whole file.

  • Open the file at path with specified access mode and POSIX flags on a private thread pool which is separate from any EventLoop thread.

    This function will return (a future) of the NIOFileHandle associated with the file opened. The caller must close the returned NIOFileHandle when it’s no longer needed.

    Declaration

    Swift

    public func openFile(path: String, mode: NIOFileHandle.Mode, flags: NIOFileHandle.Flags = .default, eventLoop: EventLoop) -> EventLoopFuture<NIOFileHandle>

    Parameters

    path

    The path of the file to be opened for writing.

    mode

    File access mode.

    flags

    Additional POSIX flags.

    eventLoop

    The EventLoop on which the returned EventLoopFuture will fire.

    Return Value

    An EventLoopFuture containing the NIOFileHandle.