dracarys S3 Download — dr_s3_download • dracarys

Download only S3 files that can be processed by dracarys.

Usage

dr_s3_download(
  s3dir,
  outdir,
  max_objects = 100,
  pattern = NULL,
  regexes = DR_FILE_REGEX,
  dryrun = FALSE
)

Arguments

s3dir: S3 directory.
outdir: Path to output directory.
max_objects: Maximum objects returned.
pattern: Pattern to further filter the returned file type tibble.
regexes: Tibble with regex and function name.
dryrun: If TRUE, just list the files that will be downloaded (don't download them).

Examples

if (FALSE) { # \dontrun{
p1 <- "s3://org.umccr.data.oncoanalyser/analysis_data/SBJ05373/sash"
p2 <- "20240707becde493/L2401018_L2401017/SBJ05373_MDX240220"
s3dir <- file.path(p1, p2)
regexes <- tibble::tribble(
  ~regex, ~fun,
  "multiqc_data\\.json$", "MultiqcJsonFile",
  "pcgr.*\\.json\\.gz$", "pcgrjson"
)
outdir <- sub("s3:/", "~/s3", s3dir)
dr_s3_download(s3dir = s3dir, outdir = outdir, max_objects = 500, regexes = regexes, dryrun = F)
} # }