Merge pull request #32 from SASPAC/dev

The BasePlus package [ver. 1.35.1]

README.md

@@ -52,7 +52,7 @@ libname NEW "%workPath()/new";
 ```
 and more.
 
-SHA256 digest for the latest version of `BasePlus`: F*3407AD8068C7528E129034144F9A44CFDF14B7DC34334C64C2F1D67351D1E01E
+SHA256 digest for the latest version of `BasePlus`: F*BCD89EDF856762EB8E441BC53933774483258453D1F7D74185F8A1861E414B0E
 
 [**Documentation for BasePlus**](./baseplus.md "Documentation for BasePlus")
 

baseplus.md

@@ -45,6 +45,9 @@
   * [`%RainCloudPlot()` macro](#raincloudplot-macro)
   * [`%zipLibrary()` macro](#ziplibrary-macro)
   * [`%unzipLibrary()` macro](#unziplibrary-macro)
+  * [`%zipArch()` macro](#ziparch-macro)
+  * [`%unzipArch()` macro](#unziparch-macro)
+  * [`%downloadFilesTo()` macro](#downloadfilesto-macro)
   * [`%LDSN()` macro](#ldsn-macro)
   * [`%LDsNm()` macro](#ldsnm-macro)
   * [`%LVarNm()` macro](#lvarnm-macro)
@@ -78,7 +81,7 @@
   
 ---
 
-# The BasePlus package [ver. 1.32.0] <a name="baseplus-package"></a> ###############################################
+# The BasePlus package [ver. 1.35.1] <a name="baseplus-package"></a> ###############################################
 
 The **BasePlus** package implements useful
 functions and functionalities I miss in the BASE SAS.
@@ -100,7 +103,8 @@ Kudos to all who inspired me to generate this package:
 *Michal Ludwicki*,
 *Quentin McMullen*,
 *Kurt Bremser*,
-*Leonid Batkhan*.
+*Leonid Batkhan*,
+*Louise Hadden*.
 
 ---
 
@@ -354,81 +358,139 @@ run;
 ;
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
+**EXAMPLE 25** Zipping and unzipping directories:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas
+options dlCreateDir;
+libname arch1 "%workPath()/testArch1";
+libname arch2 "%workPath()/testArch2";
+
+filename arch1 "%workPath()/testArch1";
+
+data _null_;
+  file arch1(test1.txt);
+  put "text for test file 1";
+data _null_;
+  file arch1(test2.txt);
+  put "text for test file 2";
+data _null_;
+  file arch1(test3.txt);
+  put "text for test file 3";
+run;
+
+data arch1.class(index=(name));
+  set sashelp.class;
+run;
+data arch1.cars(index=(model));
+  set sashelp.cars;
+run;
+
+%zipArch(
+  archName2.zip
+, pathRef = arch1
+, target = %workPath()/testArch2
+, list = 1 
+, overwrite = 1
+)
+
+%unzipArch(
+  archName2.zip 
+, path = %workPath()/testArch2
+, target = %workPath()/testArch2
+, clean=1
+, list=1
+);
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+**EXAMPLE 26** Downloading data from the internet to a local dirrectory:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas
+%downloadFilesTo(~/directoryA)
+datalines4;
+https://www.lexjansen.com/wuss/2023/WUSS-2023-Paper-189.pdf
+https://www.lexjansen.com/wuss/2023/WUSS-2023-Paper-189.zip
+;;;;
+run;
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+
 ---
 
 Package contains: 
-1.  macro     bppipe
-2.  macro     deduplistc
-3.  macro     deduplistp
-4.  macro     deduplists
-5.  macro     deduplistx
-6.  macro     dirsandfiles
-7.  macro     functionexists
-8.  macro     getvars
-9.  macro     intslist
-10. macro     ldsn
-11. macro     ldsnm
-12. macro     lvarnm
-13. macro     lvarnmlab
-14. macro     qdeduplistx
-15. macro     qgetvars
-16. macro     qzipevalf
-17. macro     raincloudplot
-18. macro     repeattxt
-19. macro     splitdsintoblocks
-20. macro     splitdsintoparts
-21. macro     symdelglobal
-22. macro     unziplibrary
-23. macro     zipevalf
-24. macro     ziplibrary
-25. format    bool
-26. format    boolz
-27. format    ceil
-28. format    floor
-29. format    int
-30. function  arrfill
-31. function  arrfillc
-32. function  arrmissfill
-33. function  arrmissfillc
-34. function  arrmisstoleft
-35. function  arrmisstoleftc
-36. function  arrmisstoright
-37. function  arrmisstorightc
-38. function  bracketsc
-39. function  bracketsn
-40. function  catxfc
-41. function  catxfi
-42. function  catxfj
-43. function  catxfn
-44. function  deldataset
-45. function  semicolonc
-46. function  semicolonn
-47. format    brackets
-48. format    semicolon
-49. proto     qsortincbyprocproto
-50. function  frommissingtonumberbs
-51. function  fromnumbertomissing
-52. function  quicksort4notmiss
-53. function  quicksorthash
-54. function  quicksorthashsddv
-55. function  quicksortlight
-56. macro     date
-57. macro     datetime
-58. macro     filepath
-59. macro     finddswithvarval
-60. macro     fmt
-61. macro     gettitle
-62. macro     infmt
-63. macro     letters
-64. macro     libpath
-65. macro     minclude
-66. macro     monthshift
-67. macro     replist
-68. macro     time
-69. macro     today
-70. macro     translate
-71. macro     tranwrd
-72. macro     workpath
+1.   macro      bppipe 
+2.   macro      deduplistc 
+3.   macro      deduplistp 
+4.   macro      deduplists 
+5.   macro      deduplistx 
+6.   macro      dirsandfiles 
+7.   macro      functionexists 
+8.   macro      getvars 
+9.   macro      intslist 
+10.  macro      ldsn 
+11.  macro      ldsnm 
+12.  macro      lvarnm 
+13.  macro      lvarnmlab 
+14.  macro      qdeduplistx 
+15.  macro      qgetvars 
+16.  macro      qzipevalf 
+17.  macro      raincloudplot 
+18.  macro      repeattxt 
+19.  macro      splitdsintoblocks 
+20.  macro      splitdsintoparts 
+21.  macro      symdelglobal 
+22.  macro      unziparch 
+23.  macro      unziplibrary 
+24.  macro      ziparch 
+25.  macro      zipevalf 
+26.  macro      ziplibrary 
+27.  format     bool 
+28.  format     boolz 
+29.  format     ceil 
+30.  format     floor 
+31.  format     int 
+32.  functions  arrfill 
+33.  functions  arrfillc 
+34.  functions  arrmissfill 
+35.  functions  arrmissfillc 
+36.  functions  arrmisstoleft 
+37.  functions  arrmisstoleftc 
+38.  functions  arrmisstoright 
+39.  functions  arrmisstorightc 
+40.  functions  bracketsc 
+41.  functions  bracketsn 
+42.  functions  catxfc 
+43.  functions  catxfi 
+44.  functions  catxfj 
+45.  functions  catxfn 
+46.  functions  deldataset 
+47.  functions  semicolonc 
+48.  functions  semicolonn 
+49.  format     brackets 
+50.  format     semicolon 
+51.  proto      qsortincbyprocproto 
+52.  functions  frommissingtonumberbs 
+53.  functions  fromnumbertomissing 
+54.  functions  quicksort4notmiss 
+55.  functions  quicksorthash 
+56.  functions  quicksorthashsddv 
+57.  functions  quicksortlight 
+58.  macro      date 
+59.  macro      datetime 
+60.  macro      downloadfilesto 
+61.  macro      filepath 
+62.  macro      finddswithvarval 
+63.  macro      fmt 
+64.  macro      gettitle 
+65.  macro      infmt 
+66.  macro      letters 
+67.  macro      libpath 
+68.  macro      minclude 
+69.  macro      monthshift 
+70.  macro      replist 
+71.  macro      time 
+72.  macro      today 
+73.  macro      translate 
+74.  macro      tranwrd 
+75.  macro      workpath
 
 
 
@@ -436,10 +498,10 @@ Package contains additional content, run:  %loadPackageAddCnt(BasePlus)  to load
 or look for the baseplus_AdditionalContent directory in the Packages fileref
 localization (only if additional content was deployed during the installation process).
 
-* SAS package generated by generatePackage, version 20231024 *
+* SAS package generated by generatePackage, version 20231107 *
 
 The SHA256 hash digest for package BasePlus: 
-`F*3407AD8068C7528E129034144F9A44CFDF14B7DC34334C64C2F1D67351D1E01E` 
+`F*BCD89EDF856762EB8E441BC53933774483258453D1F7D74185F8A1861E414B0E` 
 
 ---
 # Content description ############################################################################################
@@ -3771,6 +3833,356 @@ run;
 
 ---
 
+## >>> `%zipArch()` macro: <<< <a name="ziparch-macro"></a> #######################  
+
+The zipArch() macro allows to ZIP content of a directory. 
+Macro is OS independent, the `XCMD` option is not required.
+
+Content of zipped archive can be listed in the log.
+
+Errors of decompression and are reported. 
+
+Macro **does not** include hidden files.
+
+See examples below for the details.
+
+### SYNTAX: ###################################################################
+
+The basic syntax is the following, the `<...>` means optional parameters:
+~~~~~~~~~~~~~~~~~~~~~~~sas
+%zipArch(
+    archName
+   ,path = 
+  <,pathRef=>
+  <,target=>
+  <,targetRef=>
+  <,list=>
+  <,overwrite=>
+  <,dropList=> 
+)
+~~~~~~~~~~~~~~~~~~~~~~~
+
+**Arguments description**:
+
+1. `archName`      - *Required*, name of the archive to be generated.
+                     Name should be full, i.e., with the extension!
+
+2. `path=`         - *Required/Optional*, location of a directory to ZIP.
+                     The path should be provided unquoted.
+                     Has priority over the `pathRef` parameter.
+
+* `pathRef=`       - *Required/Optional*, fileref to location of a directory to ZIP.
+                     The reference **has** to be pointing to single directory.
+                     If provided with `path` - the `path` takes 
+                     priority over the `pathRef` parameter.
+
+* `target=`        - *Optional*, a path pointing to target location where
+                     the archive will be generated.
+                     The path should be provided unquoted.
+                     Default value is `WORK` location.
+                     Has priority over the `targetRef` parameter.
+
+* `targetRef=`     - *Optional*, fileref to a path pointing to target location 
+                     where the archive will be generated.
+                     The reference **has** to be pointing to single directory.
+                     If provided with `target` - the `target` takes 
+                     priority over the `targetRef` parameter.
+
+* `list = 0`       - *Optional*, default value is `0`,
+                     indicates if zip content should be listed in the log. 
+                     `1` means *yes*, `0` means *no*. 
+
+* `overwrite = 0`  - *Optional*, default value is `0`,
+                     indicates if existing archive file should be overwritten. 
+                     `1` means *yes*, `0` means *no*. 
+
+* `overwrite = 1`  - *Technical*, default value is `1`,
+                     indicates if the "to-be-zipped-files-list" 
+                     data set should be deleted. 
+                     `1` means *yes*, `0` means *no*.
+
+---
+
+### EXAMPLES AND USECASES: ####################################################
+
+**EXAMPLE 1.** Zip a directory . Example requires the `basePlus` package.
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas
+
+options dlCreateDir;
+libname arch1 "%workPath()/testArch1";
+libname arch2 "%workPath()/testArch2";
+
+filename arch1 "%workPath()/testArch1";
+
+data _null_;
+  file arch1(test1.txt);
+  put "text for test file 1";
+data _null_;
+  file arch1(test2.txt);
+  put "text for test file 2";
+data _null_;
+  file arch1(test3.txt);
+  put "text for test file 3";
+run;
+
+data arch1.class(index=(name));
+  set sashelp.class;
+run;
+data arch1.cars(index=(model));
+  set sashelp.cars;
+run;
+
+
+
+%zipArch(
+  archName1.zip
+, path = %workPath()/testArch1
+, list = 1 
+, overwrite = 1
+
+)
+
+%zipArch(
+  archName2.zip
+, pathRef = arch1
+, target = %workPath()/testArch2
+, list = 1 
+, overwrite = 1
+)
+
+
+%unzipArch(
+  archName2.zip 
+, path = %workPath()/testArch2
+, target = %workPath()/testArch2
+, clean=1
+, list=1
+);
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+---
+
+## >>> `%unzipArch()` macro: <<< <a name="unziparch-macro"></a> #######################  
+
+The unzipArch() macro allows to unzip content of a ZIP archive. 
+Macro is OS independent, the `XCMD` option is not required.
+
+The `dlCreateDir` option is used under the hood.
+
+Content of unzipped archive can be listed in the log.
+
+Source files can be deleted after decompression.
+Errors of decompression and are reported. If any occur
+the deletion is suspended.
+
+See examples below for the details.
+
+### SYNTAX: ###################################################################
+
+The basic syntax is the following, the `<...>` means optional parameters:
+~~~~~~~~~~~~~~~~~~~~~~~sas
+%unzipArch(
+    archName
+  <,path=>
+  <,pathRef=>
+  <,target=>
+  <,targetRef=>
+  <,list=> 
+  <,clean=>
+)
+~~~~~~~~~~~~~~~~~~~~~~~
+
+**Arguments description**:
+
+1. `archName`      - *Required*, name of the ZIP archive to be extracted.
+                     Name should be full, i.e., with the extension!
+
+* `path=`          - *Optional*, a path pointing to zipped file location.
+                     The path should be provided unquoted.
+                     Default value is `WORK` location.
+
+* `pathRef=`       - *Optional*, a fileref to path pointing to zipped file location.
+                     The `path`, if not null, has precedense over the `pathRef`.
+
+* `target=`        - *Optional*, a path pointing to target location where
+                     files will be extracted.
+                     The path should be provided unquoted.
+                     Default value is `WORK` location.
+
+* `target=`        - *Optional*, a fileref to path pointing to target location where
+                     files will be extracted.
+                     The `target`, if not null, has precedense over the `targetRef`.
+
+* `list = 0`       - *Optional*, default value is `0`,
+                     indicates if zip content should be listed in the log. 
+                     `1` means *yes*, `0` means *no*. 
+
+* `clean = 0`      - *Optional*, default value is `0`,
+                     indicates if zip file should be deleted after unzipping. 
+                     `1` means *yes*, `0` means *no*. 
+
+---
+
+### EXAMPLES AND USECASES: ####################################################
+
+**EXAMPLE 1.** Unzip compressed archive. Example requires the `basePlus` package.
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas
+
+filename arch ZIP "%workPath()/testArch.zip";
+
+data _null_;
+  file arch(abc/test1.txt);
+  put "text for test file 1";
+data _null_;
+  file arch(abc/subdir/test2.txt);
+  put "text for test file 2";
+data _null_;
+  file arch(abc/subdir/test3.txt);
+  put "text for test file 3";
+run;
+
+%unzipArch(
+  testArch.zip 
+, path = %workPath()
+, target = %workPath()
+, list=1
+);
+
+
+
+filename pR "%workPath()";
+
+%unzipArch(
+  testArch.zip 
+, pathRef = pR
+, targetRef = pR
+, clean=1
+);
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+---
+
+## >>> `%downloadFilesTo()` macro: <<< <a name="downloadfilesto-macro"></a> #######################  
+
+The downloadFilesTo() macro copy files (in binary mode
+using `filename()` function with options `lrecl=1 recfm=n`) 
+from list provided by user to a directory indicated 
+in the macro call.
+
+Macro can be executed in two possible ways:
+1) by providing list of files to download in a `datalines4`(`cards4`) list 
+   directly after macro call:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas
+  %downloadFilesTo(</path/to/target/directory>)
+  datalines4;
+  <link to file1>
+  <link to file2>
+   ...
+  <link to fileN>
+  ;;;;
+  run;
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+2) by create a dataset with a list of links and use of `DS=` and `DSvar=` parameters.
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas
+  %downloadFilesTo(</path/to/target/directory>
+  , DS=<dataset with list>
+  , DSvar=<variable with list>
+  )
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+See examples below for the details.
+
+The `%downloadFilesTo()` macro **does not** execute as a pure macro code.
+
+Temporary dataset `work.______locationInfoData` is generated during processing.
+
+### SYNTAX: ###################################################################
+
+The basic syntax is the following, the `<...>` means optional parameters:
+~~~~~~~~~~~~~~~~~~~~~~~sas
+%downloadFilesTo(
+    target
+  <,DS=>
+  <,DSvar=link>
+  <,inDev=URL>
+  <,outDev=DISK>
+  <,inOptions=>
+  <,outOptions=>
+)
+~~~~~~~~~~~~~~~~~~~~~~~
+
+**Arguments description**:
+
+1. `target  `     - *Required*, a path to target directory.
+                    If empty the `WORK` location is used.
+
+*. `DS= `         - *Optional*, name of data set with list 
+                    of files to download. 
+
+*. `DSvar= `      - *Optional*, name of variable in data set 
+                    with list of files to download. 
+
+
+*. `inDev=`       - *Optional*, type of device used by the 
+                    `filename()` function to access incoming files.
+                    Default value is `URL`.
+
+*. `outDev=`      - *Optional*, type of device used by the 
+                    `filename()` function to access outgoing files.
+                    Default value is `DISK`.
+
+*. `inOptions=`   - *Optional*, list of additional options for the 
+                    `filename()` function to access incoming files.
+                    Default value is empty.
+
+*. `outOptions=`  - *Optional*, list of additional options for the 
+                    `filename()` function to access outgoing files.
+                    Default value is empty.
+
+
+---
+
+### EXAMPLES AND USECASES: ####################################################
+
+**EXAMPLE 1.** Download data from web with diect list and then copy between directories:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas
+resetline;
+%downloadFilesTo(~/directoryA)
+datalines4;
+https://www.lexjansen.com/wuss/2023/WUSS-2023-Paper-189.pdf
+https://www.lexjansen.com/wuss/2023/WUSS-2023-Paper-189.zip
+;;;;
+run;
+
+%downloadFilesTo(~/directoryB,inDev=DISK)
+datalines4;
+~/directoryA/WUSS-2023-Paper-189.pdf
+~/directoryA/WUSS-2023-Paper-189.zip
+;;;;
+run;
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+
+**EXAMPLE 2.** Download data from web using data set with list:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas
+resetline;
+data listOfFiles;
+infile cards;
+input files :$1024.;
+cards4;
+https://www.lexjansen.com/wuss/2023/WUSS-2023-Paper-201.pdf
+https://www.lexjansen.com/wuss/2023/WUSS-2023-Paper-109.pdf
+;;;;
+run;
+
+%downloadFilesTo(R:\directoryC, DS=listOfFiles, DSvar=files)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+---
+
 ## >>> `%LDSN()` macro: <<< <a name="ldsn-macro"></a> #######################  
 
 The LDSN (Long DataSet Names) macro function