#include <core/storage/fileio/file_download_cache.hpp>

Public Member Functions
	~file_download_cache ()
	deletes all downloaded temporary files

std::string	get_file (const std::string &url)

void	release_cache (const std::string &url)

void	clear ()

Static Public Member Functions
static file_download_cache &	get_instance ()

Detailed Description

Provides URL download and caching capabilities. file_download_cache::get_instance() provides a singleton instance of the file_download_cache. Using the file_download_cache simply involves calling the file_download_cache::get_file function which take as its first argument, a general URL (s3, https, http, file, etc) and returns a local file name which can be used to access the file downloaded from the URL.

The file_download_cache will cache all the temporary files and avoid re-downloading identical URLs. file_download_cache::release_cache can be used to force a file to be uncached.

get_file() is safe to use concurrently. release_cache() has to be used carefully since there are race condition concerns if the downloaded file is still being used by another thread.

For s3 files, cache will be updated based on last modification time.

Definition at line 34 of file file_download_cache.hpp.

Member Function Documentation

◆ clear()

void turi::file_download_cache::clear ( )

Release all cached files

◆ get_file()

std::string turi::file_download_cache::get_file ( const std::string & url )

downloads the URL (it can be s3, https, http, file, or even local file) and returns a local file name at which the contents at the URL can be read from.

This function can be safely run in parallel. Though if the same file is requested in two threads simultaneously, the file may be downloaded twice.

May throw exceptions if the URL cannot be downloaded.

◆ get_instance()

static file_download_cache& turi::file_download_cache::get_instance ( )

static

Obtains the global singleton instance of the file download cache

◆ release_cache()

void turi::file_download_cache::release_cache ( const std::string & url )

Releases the cached copy of the contents of a given URL.

This function can be safely run in parallel, but there is the risk that the file may will be used / referenced by another thread. The caller must be careful to guarantee that the local file can be deleted.

The documentation for this class was generated from the following file:

core/storage/fileio/file_download_cache.hpp