List Relevant Files In AWS S3 Directory

Lists relevant files in an AWS S3 directory.

Usage

s3_files_list_filter_relevant(
  s3dir,
  pattern = NULL,
  page_size = 1000,
  max_items = 1000,
  presign = FALSE,
  expiry_sec = 43200
)

Arguments

s3dir

S3 directory.

pattern

Pattern to further filter the returned file type tibble.

page_size

The size of each page to get in the AWS service call (def: 1000).

max_items

The total number of items to return in the command’s output (def: 1000).

presign

Include presigned URLs (def: FALSE).

expiry_sec

Number of seconds the presigned URL will be valid for (if generated) (def: 43200 (12hrs)).

Value

A tibble with path, date, file size, file type, and presigned URL if requested.

Examples

if (FALSE) {
s3dir <- "s3://umccr-primary-data-prod/cancer_report_tables"
s3_files_list_filter_relevant(s3dir = s3dir, presign = TRUE)
}