diff --git a/README.md b/README.md index 07ee1cf..419088a 100644 --- a/README.md +++ b/README.md @@ -111,9 +111,9 @@ SHA256 digest for MacroCore: A23C29529F3CE7D0C8BEE9545C5D22D5B5594907547374A5135 [Documentation for MacroCore](https://core.sasjs.io "Documentation for MacroCore") -- **DFA** (Dynamic Function Arrays)\[0.3\], contains set of macros and FCMP functions which implement: a dynamically allocated array, a stack, a fifo queue, an ordered stack, and a priority queue, run `%helpPackage(DFA,createDFArray)` to find examples. +- **DFA** (Dynamic Function Arrays)\[0.4\], contains set of macros and FCMP functions which implement: a dynamically allocated array, a stack, a fifo queue, an ordered stack, and a priority queue, run `%helpPackage(DFA,createDFArray)` to find examples. -SHA256 digest for DFA: 1FC8D030D576C33F1B5DEB27E17534946209BC148D57A1357CA025ED1E69AEB8 +SHA256 digest for DFA: 5B41657A8A3BEAD185B13D072BC54902AB81FEAD32330DA8ABA90CF777D4D76E [Documentation for DFA](https://github.com/yabwon/SAS_PACKAGES/blob/main/packages/dfa.md "Documentation for DFA") diff --git a/packages/README.md b/packages/README.md index 77e6e32..2fc3fdd 100644 --- a/packages/README.md +++ b/packages/README.md @@ -32,7 +32,7 @@ SHA256 digest for MacroCore: A23C29529F3CE7D0C8BEE9545C5D22D5B5594907547374A5135 --- -- **DFA** (Dynamic Function Arrays)\[0.3\], contains set of macros and FCMP functions which implement: a dynamically allocated array, a stack, a fifo queue, an ordered stack, and a priority queue, run `%helpPackage(DFA,createDFArray)` to find examples. +- **DFA** (Dynamic Function Arrays)\[0.4\], contains set of macros and FCMP functions which implement: a dynamically allocated array, a stack, a fifo queue, an ordered stack, and a priority queue, run `%helpPackage(DFA,createDFArray)` to find examples. ```sas %createDFArray(ArrDynamic, resizefactor=17); @@ -59,7 +59,7 @@ data _null_; end; run; ``` -SHA256 digest for DFA: 1FC8D030D576C33F1B5DEB27E17534946209BC148D57A1357CA025ED1E69AEB8 +SHA256 digest for DFA: 5B41657A8A3BEAD185B13D072BC54902AB81FEAD32330DA8ABA90CF777D4D76E [Documentation for DFA](https://github.com/yabwon/SAS_PACKAGES/blob/main/packages/dfa.md "Documentation for DFA") diff --git a/packages/SHA256_for_packages.txt b/packages/SHA256_for_packages.txt index da870e5..5666159 100644 --- a/packages/SHA256_for_packages.txt +++ b/packages/SHA256_for_packages.txt @@ -1,3 +1,6 @@ +/* 20201229 */ +DFA: 5B41657A8A3BEAD185B13D072BC54902AB81FEAD32330DA8ABA90CF777D4D76E + /* 20201207 */ macroArray: AC3AD58AFBBE459616743DC6346330BD8DD33FBA8CDD595423F181B67D0475BC BasePlus: 9EA40F72191D1916189F043315CA519F6E42CEB05C186F7653AE464D21D21CFB diff --git a/packages/dfa.md b/packages/dfa.md index b01f9dd..37b076b 100644 --- a/packages/dfa.md +++ b/packages/dfa.md @@ -6,6 +6,11 @@ * [`%createDHOrdStack()` macro](#createdhordstack-macro) * [`%createDHPrtQueue()` macro](#createdhprtqueue-macro) * [`%createDHStack()` macro](#createdhstack-macro) + * [`bit64orPROTOdfa()` proto function](#bit64orprotodfa-proto-function) + * [`bit64andPROTOdfa()` proto function](#bit64andprotodfa-proto-function) + * [`bit64orDFA()` subroutine](#bit64ordfa-function) + * [`bit64andDFA()` subroutine](#bit64anddfa-function) + * [`%createDFBitmap()` macro](#createdfbitmap-macro) * [`generateArrays` exec](#createdhprtqueue-exec) * [`generateArrays` clean](#createdhprtqueue-clean) @@ -13,14 +18,15 @@ --- -# The DFA package [ver. 0.3] ############################################### +# The DFA package [ver. 0.4] ############################################### The **DFA** (a.k.a. *Dynamic Function Array*) package implements: - dynamic numeric and character arrays, - dynamic stacks (filo), - dynamic queues (fifo), - dynamic ordered stacks, - - priority queues. + - priority queues, + - bitmap. The package provides a set of *macros*, which allows to *generate* `call routines` @@ -32,19 +38,24 @@ See particular macro help for further details. --- Package contains: - 1. macro createdfarray - 2. macro createdharray - 3. macro createdhfifo - 4. macro createdhordstack - 5. macro createdhprtqueue - 6. macro createdhstack - 7. exec generatearrays - 8. clean generatearrays +1. macro createdfarray +2. macro createdharray +3. macro createdhfifo +4. macro createdhordstack +5. macro createdhprtqueue +6. macro createdhstack +7. proto bit64andprotodfa +8. proto bit64orprotodfa +9. functions bit64anddfa +10. functions bit64ordfa +11. macro createdfbitmap +12. exec generatearrays +13. clean generatearrays -*SAS package generated by generatePackage, version 20201115* +*SAS package generated by generatePackage, version 20201209* The SHA256 hash digest for package BasePlus: -`1FC8D030D576C33F1B5DEB27E17534946209BC148D57A1357CA025ED1E69AEB8` +`5B41657A8A3BEAD185B13D072BC54902AB81FEAD32330DA8ABA90CF777D4D76E` --- # Content description ############################################################################################ @@ -1089,6 +1100,408 @@ The `value` argument is **outarg**, i.e. can be changed by the function. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --- + +## >>> `bit64orPROTOdfa()` proto function: <<< ####################### + +The **bit64orPROTOdfa()** is external *C* function, +this is the implementation of the *bitwise OR* operation +on doubles. A double is returned. + +**Caution!** For SAS numeric values *only* operations on first 53 bits are valid! + +The function is used **internally** by functions in the *DFA* package. + +### SYNTAX: ################################################################### + +The basic syntax is the following: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +bit64orPROTOdfa(i, j) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `i` - A double numeric argument. + +2. `j` - A double numeric argument. + +--- + +## >>> `bit64andPROTOdfa()` proto function: <<< ####################### + +The **bit64andPROTOdfa()** is external *C* function, +this is the implementation of the *bitwise AND* operation +on doubles. A double is returned. + +**Caution!** For SAS numeric values *only* operations on first 53 bits are valid! + +The function is used **internally** by functions in the *DFA* package. + +### SYNTAX: ################################################################### + +The basic syntax is the following: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +bit64andPROTOdfa(i, j) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `i` - A double numeric argument. + +2. `j` - A double numeric argument. + +--- + +## >>> `bit64orDFA()` subroutine: <<< ####################### + +The **bit64orDFA()** function is an alternative to +the 32 bit bitwise `BOR()` function working on SAS numerics. +Allows to work on *up to* 53 bits of SAS numeric value. + +The `bit64orDFA()` is an *internal* function of the `DFA` package. + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +bit64orDFA(a, b) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `a` - Argument is a SAS numeric values. + +2. `B` - Argument is a SAS numeric values. + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Basic test of `bit64orDFA()` and `bit64andDFA()` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + options ls = max ps = max; + %let M = 53 ; %* 53 is maximum valid value; + data _null_; + array bitmask [ 0: &M] _temporary_ ; + do P = 1 to &M ; + bitmask[P] = 2**(P-1) ; + put bitmask[P] = binary54. @; + put bitmask[P] = best32.; + end ; + bitmask[0] = bitmask[&M.] ; + put bitmask[0] = best32. /; + + a=0; + put a = binary54.; + do P = 1 to &M ; + a = BIT64ORDFA (a, bitmask[P]) ; + put a = binary54.; + end; + put; + + b = 0; + put b = binary54./; + do P = 1 to &M ; + b + (BIT64ANDDFA (a, bitmask[P]) ne .) ; + put b = best32.; + end; + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- + +## >>> `bit64andDFA()` subroutine: <<< ####################### + +The **bit64andDFA()** function is an alternative to +the 32 bit bitwise `BAND()` function working on SAS numerics. +Allows to work on *up to* 53 bits of SAS numeric value. + +The `bit64andDFA()` is an *internal* function of the `DFA` package. + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +bit64andDFA(a, b) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `a` - Argument is a SAS numeric values. + +2. `B` - Argument is a SAS numeric values. + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Basic test of `bit64orDFA()` and `bit64andDFA()` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + options ls = max ps = max; + %let M = 53 ; %* 53 is maximum valid value; + data _null_; + array bitmask [ 0: &M] _temporary_ ; + do P = 1 to &M ; + bitmask[P] = 2**(P-1) ; + put bitmask[P] = binary54. @; + put bitmask[P] = best32.; + end ; + bitmask[0] = bitmask[&M.] ; + put bitmask[0] = best32. /; + + a=0; + put a = binary54.; + do P = 1 to &M ; + a = BIT64ORDFA (a, bitmask[P]) ; + put a = binary54.; + end; + put; + + b = 0; + put b = binary54./; + do P = 1 to &M ; + b + (BIT64ANDDFA (a, bitmask[P]) ne .) ; + put b = best32.; + end; + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- + +## >>> `%createDFBitmap()` macro: <<< ####################### + +The `%createDFBitmap()` macro allows to generate +a `dynamic function bitmap` which is a FCMP based +approach to create *dynamically* allocated **numeric** +bitmnap. + +*Note:* Arrays provided by the macro are *one dimensional* arrays. + +The idea of a SAS bitmap is based on: +1. SGF Paper 3101-2019 + titeled **Re-Mapping A Bitmap** by *Paul M. Dorfman* and *Lessia S. Shajenko* + [https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2019/3101-2019.pdf](https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2019/3101-2019.pdf) +2. SUGI Paper 8-26 + titeled **Table Look-Up by Direct Addressing: Key-Indexing -- Bitmapping -- Hashing** by *Paul M. Dorfman* + [https://support.sas.com/resources/papers/proceedings/proceedings/sugi26/p008-26.pdf](https://support.sas.com/resources/papers/proceedings/proceedings/sugi26/p008-26.pdf) + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +%createDFBitmap( + bitmapName + <,debug=0> + <,outlib=work.DFAfcmp.package> + <,type=32> +) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `bitmapName` - *Required*, creates a FCMP call subroutine which is also + a bitmap name. In the data step it is used in form of + a call subroutine, e.g. `call bitmapName("Allocate", 3000)`. + Has to satisfy FCMP function naming requirements, but with + maximum of 24 characters. + +* `debug=` - *Optional*, the default value is `0`. + If set to `1` then it turns on a debugging mode. + +* `outlib=` - *Optional*, the default value is `work.DFAfcmp.package`. + It points the default location for new generated dynamic + function arrays compiled by FCMP. + *Hint!* Keep it as it is. + +* `type=` - *Optional*, the default value is `32`. Sets the type of + bitwise operations executed internaly on the bitmap. + The only valid values are `32` or `52`, + With 32 the `BOR()` and `BAND()` SAS functions are used + and with 52 the `bit64orDFA()` and `bit64and DFA()` FCMP + functions are used. + +**Created function arguments description**: + +A function generated by the macro is: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +call &bitmapName.(IO, position, value) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +and accepts the following list of arguments and values: + +1. `IO` - is a *character* steering argument, possible + values and behaviour they call are the following: + - `Check`, `Test`, `T` - to get information if a bit is set to 1 (on) or not, + - `On`, `1` - to set a selected bit to 1, + - `Off`, `0` - to set a selected bit to 0, + - `C`, `Clear` - to reduce a bitmat to a single empty cell, + - `A`, `Allocate` - to reserve space for a bitmap and set all bits to 0, + - `A0`, `Allocate0` - to reserve space for a bitmap and set all bits to 0, + - `A1`, `Allocate1` - to reserve space for a bitmap and set all bits to 1, + - `D`, `Dim`, `Dimension` - to returns minimal and maximal index of the bitmap. + +2. `position` - is a *numeric* argument and depends on the `IO` value. + Behaves in the following way: + - for `On`, `Off`, `1`, `0`/ `Check`, `Test`, `T` it is a bitmap index, + - for `C`, `Clear` is ignored, + - for `A`, `Allocate` sets the value of the minposition, i.e. the minimal position of the bitmap index, + - for `D`, `Dimension` it returns value of the minposition, + +.3 `value` - is a *numeric* argument and depends on the `IO` value. + Behaves in the following way: + - for `Check`, `Test`, `T` it holds value retrieved from a bitmap on a given position, + - for `On`, `Off`, `1`, `0`, `C`, `Clear` is ignored, + - for `A`, `Allocate` it sets the value of the maxposition, i.e. maximal position of the array index, + - for `D`, `Dimension` it returns value of the maxposition + +The `position` and the `value` arguments are **outargs**, i.e. can be changed by the function. + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Bitmap of type 32: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFBitmap(MyBitmap32,type=32,debug=1); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example1; + call MyBitmap32("Allocate", -10, 100); + L = 0; H = 0; + call MyBitmap32("Dim", L, H); + put L= H=; + + * populate array with data ; + do i = L to H by 2; + put i @; + call MyBitmap32("1", i, .); + end; + put; + + * get values from the array ; + Value = .; + do i = L to H; + call MyBitmap32("T", i, Value); + put i= Value=; + end; + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**EXAMPLE 2.** Bitmap of type 52: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFBitmap(MyBitmap52,type=52,debug=1); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example2; + call MyBitmap52("Allocate", -10, 100); + L = 0; H = 0; + call MyBitmap52("Dim", L, H); + put L= H=; + + * populate array with data ; + do i = L to H by 2; + put i @; + call MyBitmap52("1", i, .); + end; + put; + + * get values from the array ; + Value = .; + do i = L to H; + call MyBitmap52("T", i, Value); + put i= Value=; + end; + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**EXAMPLE 3.** Execution time test for type 52: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFBitmap(MyBigBitmap52,type=52,debug=0); + options FULLSTIMER APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example3; + call MyBigBitmap52("Allocate", -10, 2000000001); + L = 0; H = 0; + call MyBigBitmap52("Dim", L, H); + put L= H=; + + * populate bitmap with data ; + t = time(); + do i = L to H by 17; + call MyBigBitmap52("1", i, .); + x + 1; + end; + t = time() - t; + put "populate:" t= x=; + + * get values from the bitmap ; + t = time(); + Value = .; + do i = L to H; + call MyBigBitmap52("T", i, Value); + x + (-Value); + end; + t = time() - t; + put "search:" t= x=; + run; + +%* +L=-10 H=2000000001 +populate:t=55.902999878 x=117647060 +search:t=654.12900019 x=0 +NOTE: The data set WORK.EXAMPLE3 has 1 observations and 6 variables. +NOTE: DATA statement used (Total process time): + real time 11:50.42 + user cpu time 11:46.40 + system cpu time 0.45 seconds + memory 301791.12k + OS Memory 326332.00k +; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**EXAMPLE 4.** Execution time test for type 32: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFBitmap(MyBigBitmap32,type=32,debug=0); + options FULLSTIMER APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example4; + call MyBigBitmap32("Allocate", -10, 2000000001); + L = 0; H = 0; + call MyBigBitmap32("Dim", L, H); + put L= H=; + + * populate bitmap with data ; + t = time(); + do i = L to H by 17; + call MyBigBitmap32("1", i, .); + x + 1; + end; + t = time() - t; + put "populate:" t= x=; + + * get values from the bitmap ; + t = time(); + Value = .; + do i = L to H; + call MyBigBitmap32("T", i, Value); + x + (-Value); + end; + t = time() - t; + put "populate:" t= x=; + run; + +%* +L=-10 H=2000000001 +populate:t=50.417999983 x=117647060 +populate:t=611.13600016 x=0 +NOTE: The data set WORK.EXAMPLE4 has 1 observations and 6 variables. +NOTE: DATA statement used (Total process time): + real time 11:02.07 + user cpu time 10:59.07 + system cpu time 1.46 seconds + memory 489583.90k + OS Memory 513876.00k +; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- + ## >>> `generateArrays` exec: <<< ####################### The generateArrays exec file provides a **list of automatically generated examples** of functions @@ -1112,6 +1525,8 @@ The list of provided examples is the following: - `PrtQueueNBC` - Dynamic Hash-based Character Priority Queue with *New on Bottom* (length 256 bytes) - `PrtQueueNTN` - Dynamic Hash-based Numeric Priority Queue with *New on Top* - `PrtQueueNBN` - Dynamic Hash-based Numeric Priority Queue with *New on Bottom* +- `Bitmap32` - Dynamic Function Bitmap on 32 bit +- `Bitmap52` - Dynamic Function Bitmap on 52 bit The `outlib=` option is set to `work.DFAfcmp.package`. The `cmplib=` option is updated automatically. diff --git a/packages/dfa.zip b/packages/dfa.zip index 82ec660..e1c8e6d 100644 Binary files a/packages/dfa.zip and b/packages/dfa.zip differ