Bulk Read Contents from S3

This function facilitates the bulk reading of multiple files from the project's designated S3 folder. By using cloud_s3_ls, you can obtain a dataframe detailing the contents of the S3 folder. Applying cloud_s3_read_bulk to this dataframe allows you to read all listed files into a named list. The function will, by default, infer the appropriate reading method based on each file's extension. However, if a specific reading function is provided via the fun parameter, it will be applied uniformly to all files, which may not be suitable for diverse file types.

Usage

cloud_s3_read_bulk(content, fun = NULL, ..., quiet = FALSE, root = NULL)

Arguments

content

(data.frame) Output of cloud_s3_ls()

fun

A custom reading function. If NULL (default), the appropriate reading function will be inferred based on the file's extension.

...

Additional arguments to pass to the reading function fun.

quiet

All caution messages may be turned off by setting this parameter to TRUE.

root

S3 path of the project root. This serves as the reference point for all relative paths. When left as NULL, the root is automatically derived from the cloudfs.s3 field of the project's DESCRIPTION file.

Value

A named list where each element corresponds to the content of a file from S3. The names of the list elements are derived from the file names.

Examples

if (FALSE) { # interactive()
# provided there's a folder called "data" in the root of the project's main
# S3 folder, and it contains csv files
data_lst <- 
  cloud_s3_ls("data") |>  
  filter(type == "csv")  |>  
  cloud_s3_read_bulk()
  
}