Block size for consolidation

Get or set the block size for consolidating HTTP range requests.

consolidateBlockSize(block.size = NULL)

Arguments

block.size

Integer specifying the block size in bytes. Larger sizes reduce the number of requests at the cost of increasing the size of each request.

Value

If block.size=NULL, the current block size is returned.

If block.size is provided, it is used to set the block size, and the previous value is returned invisibly.

Details

Each file is split up into blocks of size approximately equal to consolidateBlockSize(). When performing a range request, all ranges in the same block will be retrieved. This consolidates near-adjacent ranges into a single request, reducing the number of requests at the cost of increasing the size of each request.

All ranges associated with a block will be cached in memory, even those that were not directly requested. Subsequent function calls can then quickly retrieve ranges from this cache instead of making a new HTTP request. Downloading and caching the entire block also ensures that the same bytes will never be requested from the server twice in the same session.

Setting the block size to a value of 1 indicates that no consolidation is to be performed, i.e., each range is requested as-is.

Author

Aaron Lun

Examples

consolidateBlockSize()
#> [1] 10000
old <- consolidateBlockSize(500)
consolidateBlockSize()
#> [1] 500
consolidateBlockSize(old)