diff --git a/README.md b/README.md index 253133f..961a16a 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ a dynamically allocated array, a stack, a fifo queue, an ordered stack, and a pr Run `%helpPackage(DFA,createDFArray)` to find examples. -SHA256 digest for the latest version of `DFA`: F*09EA5201360922A91A9EEE72F4567792E9CFDFB591BA33419E2BF2B31D9B7C62 +SHA256 digest for the latest version of `DFA`: F*012375D87F66EB3A7BF5DDD0CC5AEE28851733EE33CC63231DF9045BEB958168 [**Documentation for DFA**](./dfa.md "Documentation for DFA") diff --git a/dfa.md b/dfa.md index 0ab0381..8a7d4f6 100644 --- a/dfa.md +++ b/dfa.md @@ -18,7 +18,7 @@ --- -# The DFA package [ver. 0.5.6] ############################################### +# The DFA package [ver. 0.5.7] ############################################### The **DFA** (a.k.a. *Dynamic Function Array*) package implements: - dynamic numeric and character arrays, @@ -52,10 +52,10 @@ Package contains: 12. exec generatearrays 13. clean generatearrays -*SAS package generated by generatePackage, version 20230905* +*SAS package generated by generatePackage, version 20231111* The SHA256 hash digest for package DFA: -`F*09EA5201360922A91A9EEE72F4567792E9CFDFB591BA33419E2BF2B31D9B7C62` +`F*012375D87F66EB3A7BF5DDD0CC5AEE28851733EE33CC63231DF9045BEB958168` --- # Content description ############################################################################################ diff --git a/dfa.zip b/dfa.zip index c189a62..657adca 100644 Binary files a/dfa.zip and b/dfa.zip differ diff --git a/hist/0.5.7/dfa.md b/hist/0.5.7/dfa.md new file mode 100644 index 0000000..8a7d4f6 --- /dev/null +++ b/hist/0.5.7/dfa.md @@ -0,0 +1,1597 @@ +- [The DFA package](#dfa-package) +- [Content description](#content-description) + * [`%createDFArray()` macro](#createdfarray-macro) + * [`%createDHArray()` macro](#createdharray-macro) + * [`%createDHFifo()` macro](#createdhfifo-macro) + * [`%createDHOrdStack()` macro](#createdhordstack-macro) + * [`%createDHPrtQueue()` macro](#createdhprtqueue-macro) + * [`%createDHStack()` macro](#createdhstack-macro) + * [`bit64orPROTOdfa()` proto function](#bit64orprotodfa-proto-function) + * [`bit64andPROTOdfa()` proto function](#bit64andprotodfa-proto-function) + * [`bit64orDFA()` subroutine](#bit64ordfa-function) + * [`bit64andDFA()` subroutine](#bit64anddfa-function) + * [`%createDFBitmap()` macro](#createdfbitmap-macro) + * [`generateArrays` exec](#createdhprtqueue-exec) + * [`generateArrays` clean](#createdhprtqueue-clean) + + * [License](#license) + +--- + +# The DFA package [ver. 0.5.7] ############################################### + +The **DFA** (a.k.a. *Dynamic Function Array*) package implements: + - dynamic numeric and character arrays, + - dynamic stacks (filo), + - dynamic queues (fifo), + - dynamic ordered stacks, + - priority queues, + - bitmap. + +The package provides a set of *macros*, +which allows to *generate* `call routines` +simulating data structures mentioned above. + +Few exemplary functions are also generated. +See particular macro help for further details. + +--- + +Package contains: +1. macro createdfarray +2. macro createdharray +3. macro createdhfifo +4. macro createdhordstack +5. macro createdhprtqueue +6. macro createdhstack +7. proto bit64andprotodfa +8. proto bit64orprotodfa +9. functions bit64anddfa +10. functions bit64ordfa +11. macro createdfbitmap +12. exec generatearrays +13. clean generatearrays + +*SAS package generated by generatePackage, version 20231111* + +The SHA256 hash digest for package DFA: +`F*012375D87F66EB3A7BF5DDD0CC5AEE28851733EE33CC63231DF9045BEB958168` + +--- +# Content description ############################################################################################ + +## >>> `%createDFArray()` macro: <<< ####################### + +The `%createDFArray()` macro allows to generate +a `dynamic function array` which is a FCMP based +approach to create *dynamically* allocated **numeric** +array with possible values searching and `WHICHN()` +function emulation. + +*Note:* Arrays provided by the macro are *one dimensional* arrays. + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +%createDFArray( + arrayName + <,debug=0> + <,simple=0> + <,resizeFactor=0> + <,outlib=work.DFAfcmp.package> + <,hashexp=13> + <,header=1> +) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `arrayName` - *Required*, creates a FCMP call subroutine which is also + an array name. In the data step it is used in form of + a call subroutine, e.g. `call arrayName("Allocate", -3, 3)`. + Has to satisfy FCMP function naming requirements, but with + maximum of 24 characters. + +* `debug=` - *Optional*, the default value is `0`. + If set to `1` then it turns on a debugging mode. + +* `simple=` - *Optional*, the default value is `0`. A *simple* dynamic + function array is one which is not searchable and does not + allows to use `which()` functionality. + If set to `1` then it disables `SEARCH` and `WHICH` functionality. + See examples below for details. + +* `resizeFactor=` - *Optional*, the default value is `0`. If set to `0` then + the dynamic array size is not changeable(mutable) after initial + size allocation. + If set not to `0` then arrays dimensions are mutable after allocation, + i.e. even if an array is allocated for ten elements (like `A[1:10]`) + you can do `A[17] = 42` and it will resize itself dynamically. + *Hint!* Set to, e.g. 4999, for faster allocation process. + See examples below for details. + +* `outlib=` - *Optional*, the default value is `work.DFAfcmp.package`. + It points the default location for new generated dynamic + function arrays compiled by FCMP. + *Hint!* Keep it as it is. + +* `hashexp=` - *Optional*, the default value is `13`. It is the default `hashexp=` + value for internal hash table used by the function. + +* `header=` - *Optional*, the default value is `1`. Indicates if + the `proc fcmp outlib = &outlib.;` header is added to + the executed code. If not 1 then no header is added. + +**Created function arguments description**: + +A function generated by the macro is: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +call &arrayName.(IO, position, value) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +and accepts the following list of arguments and values: + +1. `IO` - is a *character* steering argument, possible + values and behaviour they call are the following: + - `O`, `Output`, `R`, `Return` - to get the data from an array, + - `I`, `Input` - to insert the data into an array, + - `+`, `Add` - to increment given position by a value, + - `C`, `Clear` - to reduce an array to a single empty cell, + - `A`, `Allocate` - to reserve space for array width and set starting values, + - `D`, `Dimension` - to return minimal and maximal index of an array, + - `F`, `Find`, `Exist` - to find out if a given value exist in an array, + - `W`, `Which` - to search the first position of data in array, `WHICHN()` function emulator, + - `Sum` - to return the sum of non-missing elements of an array, + - `Nonmiss` - to return the number of non-missing elements of an array, + - `Avg`, `Mean`, `Average` - to return the average of non-missing elements of an array, + - `Min`, `Minimum` - to return the minimum of non-missing elements of an array, + - `Max`, `Maximum` - to return the maximum of non-missing elements of an array. + +2. `position` - is a *numeric* argument and depends on the `IO` value. + Behaves in the following way: + - for `O`, `Output`, `R`, `Return`/ `I`, `Input`/ `+`, `Add` it takes + an arrays index from (into) which data is get (put), + - for `C`, `Clear` is ignored, + - for `A`, `Allocate` sets the value of the minposition, i.e. the minimal position of the array index, + - for `D`, `Dimension` it returns value of the minposition, + - for `Sum`, `Nonmiss`, `Avg`, `Mean`, `Average`, `Min`, `Minimum`, `Max`, and `Maximum` is ignored, + - for `F`, `Find`, `Exist` it returns number of occurrences of a given value in an array, + - for `W`, `Which` it returns position the first occurrence of a given value in an array. + +.3 `value` - is a *numeric* argument and depends on the `IO` value. + Behaves in the following way: + - for `O`, `Output`, `R`, `Return` it holds value retrieved from an array on a given position, + - for `I`, `Input` it holds the value inserted into an array on a given position, + - for `+`, `Add` it holds the value that is used to increment an array value at a given position, + - for `C`, `Clear` is ignored, + - for `A`, `Allocate` it sets the value of the maxposition, i.e. maximal position of the array index, + - for `D`, `Dimension` it returns value of the maxposition + - for `Sum`, `Nonmiss`, `Avg`, `Mean`, `Average`, `Min`, `Minimum`, `Max`, and `Maximum` it returns + the calculated summary value, + - for `F`, `Find`, `Exist`, `W`, `Which` is the value to be searched in an array. + +The `position` and the `value` arguments are **outargs**, i.e. can be changed by the function. + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Dynamic, Searchable, and Immutable array: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFArray(ArrDSI); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example1; + call ArrDSI("Allocate", 1, 10); + L = 0; H = 0; + call ArrDSI("Dim", L, H); + put L= H=; + + * populate array with data ; + do i = L to H; + call ArrDSI("Input", i, i**2); + end; + + * searchability allows to find number of occurrences of value in the array ; + F = .; + call ArrDSI("Find", F, 16); + put "Value 16 occurs " F "times"; + call ArrDSI("Find", F, 17); + put "Value 17 occurs " F "times"; + + * increase value of cell 4 by 1, and verify at WHICH position is 17 (by searchability); + call ArrDSI("+", 4, 1); + call ArrDSI("Which", F, 17); + put "Value 17 occurred for the first time at position " F; + + * get values from the array ; + Value = .; + do i = L to H; + call ArrDSI("Output", i, Value); + put i= Value=; + end; + + * some basic statistics ; + call ArrDSI("Sum", ., STAT); put "sum = " STAT; + call ArrDSI("Avg", ., STAT); put "avg = " STAT; + call ArrDSI("Min", ., STAT); put "min = " STAT; + call ArrDSI("Max", ., STAT); put "max = " STAT; + call ArrDSI("Cnt", ., STAT); put "cnt = " STAT; + + * immutability does _not_ allow to increase dimensions automatically; + * this line returns an error ; + call ArrDSI("Input", 42, -1); + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +**EXAMPLE 2.** Dynamic, Searchable, and Mutable array: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFArray(ArrDSM, resizefactor=17); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example2; + call ArrDSM("Allocate", -2, 2); + + do i = -2 to 2; + call ArrDSM("Input", i, 2**i); + end; + + L = .; H = .; + call ArrDSM("Dim", L, H); + put L= H=; + + * mutability allows to increase dimensions automatically + * create index 3 and -3; + call ArrDSM("+", 3, 8); + call ArrDSM("+",-3, 0.125); + call ArrDSM("Dim", L, H); + put L= H=; + + Value = .; + do i = L to H; + call ArrDSM("O", i, Value); + put i= Value=; + end; + + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +**EXAMPLE 3.** Dynamic, non-searchable (a.k.a. SiMPle), and Immutable array: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFArray(ArrDSMPLI, simple=1); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example3; + call ArrDSMPLI("Allocate", -2, 2); + + do i = -2 to 2; + call ArrDSMPLI("Input", i, 2**i); + end; + + * non-searchable array (a.k.a. simple) does not allow ; + * to find number of occurrences of value in the array ; + * and verify what is the first position of a value ; + * this lines return a warning ; + call ArrDSMPLI("Exist", i, 1); + call ArrDSMPLI("Which", i, 1); + run; + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +**EXAMPLE 4.** Dynamic, non-searchable (a.k.a. SiMPle), and Mutable array: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFArray(ArrDSMPLM, simple=1, resizefactor=42); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example4; + call ArrDSMPLM("Allocate", 1, 1); + + * mutability allows to increase dimensions automatically ; + do i = -12 to 12; + call ArrDSMPLM("Input", i, i*2); + end; + + * non-searchable array (a.k.a. simple) does not allow ; + * to find number of occurrences of value in the array ; + * and verify what is the first position of a value ; + * this lines return a warning ; + i = .; + call ArrDSMPLM("Exist", i, -24); + put "Exist " i=; + call ArrDSMPLM("Which", i, 24); + put "Which " i=; + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- +## >>> `%createDHArray()` macro: <<< ####################### + +The `%createDHArray()` macro allows to generate +a `dynamic hash array` which is a FCMP based approach +to create *dynamically* allocated **numeric** +or **character** array + +*Note:* Arrays provided by the macro are *one dimensional* arrays. + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +%createDHArray( + arrayName + <,debug=0> + <,type=8> + <,outlib=work.DFAfcmp.package> + <,hashexp=13> + <,header=1> +) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `arrayName` - *Required*, creates a FCMP call subroutine which is also + an array name. In the data step it is used in form of + a call subroutine, e.g. `call arrayName("Allocate", -3, 3)`. + Has to satisfy FCMP function naming requirements, but with + maximum of 24 characters. + +* `debug=` - *Optional*, the default value is `0`. + If set to `1` then it turns on a debugging mode. + +* `type=` - *Optional*, the default value is `8`. + Indicates what *type* (numeric/character) and *length* + are data portion of generated array. Should be in line + with the LENGTH statement, e.g. `8`, `$ 30`, etc. + Determines if the `value` argument is numeric or character. + +* `outlib=` - *Optional*, the default value is `work.DFAfcmp.package`. + It points the default location for new generated dynamic + function arrays compiled by FCMP. + *Hint!* Keep it as it is. + +* `hashexp=` - *Optional*, the default value is `13`. It is the default `hashexp=` + value for internal hash table used by the function. + +* `header=` - *Optional*, the default value is `1`. Indicates if + the `proc fcmp outlib = &outlib.;` header is added to + the executed code. If not 1 then no header is added. + +**Created function arguments description**: + +A function generated by the macro is: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +call &arrayName.(IO, position, value) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +and accepts the following list of arguments and values: + +1. `IO` - is a *character* steering argument, possible + values and behaviour they call are the following: + - `O`, `Output`, `R`, `Return` - to get the data from an array, + - `I`, `Input` - to insert the data into an array, + - `C`, `Clear` - to reduce an array to a single empty cell, + - `L`, `Low`, `Lower`, `Lbound` - to return minimal position of index, + - `H`, `High`, `Higher`, `Hbound` - to return maximal position of index. + +2. `position` - is a *numeric* argument and depends on the `IO` value. + Behaves in the following way: + - for `O`, `Output`, `R`, `Return`/ `I`, and `Input` it is an array + index from (into) which data is get (put), + - for `C` it is ignored, + - for `L`, `Low`, `Lower`, and `Lbound` it returns the first position of an index, + - for `H`, `High`, `Higher`, and `Hbound` it returns the last position of an index, + - otherwise is not modified. + +3. `value` - is a *numeric* or *character* argument (determined by the `type=`) + and depends on the `IO` value. Behaves in the following way: + - for `O`, `Output`, `R`, and `Return` it holds value retrieved from an array from a given position, + - for `I`, `Input` it holds the value inserted into an array into a given position, + - for `C` is ignored, + - for `L`, `Low`, `Lower`, and `Lbound` returns first value of index, + - for `H`, `High`, `Higher`, and `Hbound` returns last value of index, + - otherwise is not modified. + +The `position` and the `value` arguments are **outargs**, i.e. can be changed by the function. + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Dynamic, Hash-based, and Character array: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDHArray(ArrDHC, type = $ 12); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + %let zeros = 6; *[to test bigger sizes]; + data Example1; + + t = time(); + do _I_ = -1e&zeros. to 1e&zeros.; + _X_ = put(_I_*10, z12.); + call ArrDHC("Input", _I_, _X_); + end; + t = time() - t; + put t= / _X_= /; + + * get the size info ; + LB = 0; HB = 0; + drop LB HB; + call ArrDHC('Lbound', LB, _X_); + call ArrDHC('Hbound', HB, _X_); + put LB= HB= /; + + t = time(); + do _I_ = HB + 1 to LB - 1 by -1; + call ArrDHC('Output', _I_, _X_); + output; + end; + t = time() - t; + put t= / _X_= /; + + * clear for further reuse ; + call ArrDHC('C', ., ''); + + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +**EXAMPLE 2.** Dynamic, Hash-based, and Numeric array: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDHArray(ArrDHN); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example2; + + do i = -2 to 2; + call ArrDHN("Input", i, 2**i); + end; + + do i = -2 to 2; + call ArrDHN("+", i, -10); + end; + + v = .; + do i = -2 to 2; + call ArrDHN("Output", i, v); + put i= v=; + end; + + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- +## >>> `%createDHFifo()` macro: <<< ####################### + +The `%createDHFifo()` macro allows to generate +a `dynamic hash fifo` which is a FCMP based approach +to create dynamically allocated numeric or character +"first in first out" [queue](https://en.wikipedia.org/wiki/FIFO_(computing_and_electronics)) + +Interesting reading about implementing a fifo via hash table +can be found in *chapter 10.4* of the: +*"Data Management Solutions Using SAS Hash Table Operations: + A Business Intelligence Case Study"* book +by Paul Dorfman and Don Henderson. + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +%createDHFifo( + fifoName + <,debug=0> + <,type=8> + <,outlib=work.DFAfcmp.package> + <,hashexp=13> + <,header=1> +) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `fifoName` - *Required*, creates a FCMP call subroutine which is also + a fifo name. In the data step it is used in form of + a call subroutine, e.g. `call fifoName("Enqueue", 3)`. + Has to satisfy FCMP function naming requirements, but with + maximum of 24 characters. + +* `debug=` - *Optional*, the default value is `0`. + If set to `1` then it turns on a debugging mode. + +* `type=` - *Optional*, the default value is `8`. + Indicates what *type* (numeric/character) and *length* + are data portion of generated array. Should be in line + with the LENGTH statement, e.g. `8`, `$ 30`, etc. + Determines if the `value` argument is numeric or character. + +* `outlib=` - *Optional*, the default value is `work.DFAfcmp.package`. + It points the default location for new generated dynamic + function arrays compiled by FCMP. + *Hint!* Keep it as it is. + +* `hashexp=` - *Optional*, the default value is `13`. It is the default `hashexp=` + value for internal hash table used by the function. + +* `header=` - *Optional*, the default value is `1`. Indicates if + the `proc fcmp outlib = &outlib.;` header is added to + the executed code. If not 1 then no header is added. + +**Created function arguments description**: + +A function generated by the macro is: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +call &fifoName.(IO, value) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +and accepts the following list of arguments and values: + +1. `IO` - is a *character* steering argument, possible + values and behaviour they call are the following: + - `O`, `Output`, `D`, `Dequeue`, `R`, `Return` - to get the data from a fifo (and remove it from the fifo) + - `I`, `Input`, `E`, `Enqueue`, and `Insert` - to insert the data into a fifo + - `C`, `Clear` - to reduce a fifo to an empty one + - `P`, `Peek`, `T`, and `Tail` - to peek the data from a fifo (and NOT remove it from the fifo) + - `H`, `Head` - to peek the data from a fifo head (and NOT remove it from the fifo) + - `Sum` - returns sum of nonmissing numeric elements of a stack + - `Avg`, `Mean`, `Average` - returns average of nonmissing numeric elements of a stack + - `Nonmiss`, `Cnt` - returns number of nonmissing elements of a stack + - `Height` - returns height a stack + +2. `value` - is a *numeric* or *character* argument (determined by the `type=`) + and depends on the `IO` value. Behaves in the following way: + - for `O`, `Output`, `D`, `Dequeue`, `R`, `Return` it holds the value popped from the fifo, + - for `I`, `Input`, `E`, `Enqueue`, `Insert` it holds the value to be pushed into the fifo, + - for `C`, `Clear` it is ignored, + - for `P`, `Peek` holds the value peeked from the fifo, + - for `Sum`, `Nonmiss`, `Cnt`, `Avg`, `Mean`, `Average`, and `Height` it returns calculated summary value. + +The `value` argument is **outarg**, i.e. can be changed by the function. + + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Dynamic, Hash-based, and Character fifo: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDHFifo(FifoDHC, type = $ 12); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + + %let zeros = 6; *[to test bigger sizes]; + data Example1; + + t = time(); drop t; + do _I_ = 1 to 1e&zeros.; + _X_ = put(_I_*10, z12.); + call FifoDHC("Enqueue", _X_); + end; + t = time() - t; + + call FifoDHC("Height", _X_); + put t= / _X_=; + + t = time(); + do _I_ = 1 to 1e&zeros. + 3; + call FifoDHC('Dequeue', _X_); + output; + end; + t = time() - t; + + call FifoDHC("Height", _X_); + put t= / _X_=; + + %* clear for further reuse *; + call FifoDHC('Clear', ''); + + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +**EXAMPLE 2.** Dynamic, Hash-based, and Numeric fifo: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDHFifo(FifoDHN); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example2; + + do _I_ = 1,.,2,.,3,.,4,.,5,.,6; + call FifoDHN("E", _I_); + end; + + call FifoDHN("Sum", _I_); + put "Sum " _I_=; + + call FifoDHN("Avg", _I_); + put "Avg " _I_=; + + call FifoDHN("Cnt", _I_); + put "Cnt " _I_=; + + call FifoDHN("Height", _I_); + put "Height " _I_=; + + call FifoDHN("Tail", _I_); + put "Tail of fifo is " _I_=; + + call FifoDHN("Height", _I_); + put "Height after Tail " _I_=; + + call FifoDHN("Head", _I_); + put "Head of fifo is " _I_=; + + call FifoDHN("Height", _I_); + put "Height after Head" _I_=; + + _X_ = 0; + do _I_ = 1 to _I_; + call FifoDHN('D', _X_); + output; + end; + + call FifoDHN("Height", _I_); + put "Height " _I_=; + + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- +## >>> `%createDHOrdStack()` macro: <<< ####################### + +The `%createDHOrdStack()` macro allows to generate +a `dynamic ORDERED hash stack` which is a FCMP based approach +to create dynamically allocated numeric or character +*ordered* [stack](https://en.wikipedia.org/wiki/Stack_(abstract_data_type)) + +Interesting reading about implementing a stack via hash table +can be found in *chapter 10.4* of the: +*"Data Management Solutions Using SAS Hash Table Operations: + A Business Intelligence Case Study"* book +by Paul Dorfman and Don Henderson. + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +%createDHOrdStack( + fifoName + <,debug=0> + <,type=8> + <,order=A> + <,outlib=work.DFAfcmp.package> + <,hashexp=13> + <,header=1> +) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `stackName` - *Required*, creates a FCMP call subroutine which is also + a stack name. In the data step it is used in form of + a call subroutine, e.g. `call stackName("Push", 3)`. + Has to satisfy FCMP function naming requirements, but with + maximum of 24 characters. + +* `debug=` - *Optional*, the default value is `0`. + If set to `1` then it turns on a debugging mode. + +* `type=` - *Optional*, the default value is `8`. + Indicates what *type* (numeric/character) and *length* + are data portion of generated array. Should be in line + with the LENGTH statement, e.g. `8`, `$ 30`, etc. + Determines if the `value` argument is numeric or character. + +* `order=` - *Optional*, the default value is `A`. + Indicates a method of ordering of the stack, + allowed values are: `A` for ascending and `D` for descending. + +* `outlib=` - *Optional*, the default value is `work.DFAfcmp.package`. + It points the default location for new generated dynamic + function arrays compiled by FCMP. + *Hint!* Keep it as it is. + +* `hashexp=` - *Optional*, the default value is `13`. It is the default `hashexp=` + value for internal hash table used by the function. + +* `header=` - *Optional*, the default value is `1`. Indicates if + the `proc fcmp outlib = &outlib.;` header is added to + the executed code. If not 1 then no header is added. + +**Created function arguments description**: + +A function generated by the macro is: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +call &stackName.(IO, value) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +and accepts the following list of arguments and values: + +1. `IO` - is a *character* steering argument, possible + values and behaviour they call are the following: + - `O`, `Output`, `Pop`, `G`, `Get`, `R`, `Return` - to get the data from a stack (and remove it from the top), + - `I`, `Input`, `Push`, `Put`, `Insert` - to insert the data into a stack, + - `C`, `Clear` - to reduce a stack to an empty one, + - `P`, `Peek` - to peek the data from a stack (and NOT remove it from the top), + - `Sum` - returns sum of non-missing numeric elements of a stack, + - `Avg`, `Mean`, `Average` - returns average of non-missing numeric elements of a stack, + - `Nonmiss`, `Cnt`, `Nnm` - returns number of non-missing elements of a stack, + - `Height` - returns height a stack, + - `Min`, `Minimum` - returns minimum of non-missing elements of a stack, + - `Max`, `Maximum` - returns maximum of non-missing elements of a stack. + +2. `value` - is a *numeric* or *character* argument (determined by the `type=`) + and depends on the `IO` value. Behaves in the following way: + - for `O`, `Output`, `Pop`, `G`, `Get`, `R`, `Return` it holds a value popped from a stack, + - for `I`, `Input`, `Push`, `Put`, `Insert` it holds a value to be pushed into a stack, + - for `C`, `Clear` it is ignored, + - for `P`, `Peek` it holds a value peeked from a stack, + - for `Sum`, `Nonmiss`, `Cnt`, `Avg`, `Mean`, `Average`, `Height`, `Min`, `Minimum`, `Max`, and `Maximum` + it returns calculated summary value, + +The `value` argument is **outarg**, i.e. can be changed by the function. + + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Dynamic, Hash-based, and Character Descending Ordered stack: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDHOrdStack(DescStackC, type = $ 12, order=D); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example1; + + do _X_ = "A","B"," ","C","A"," ","B","C"; + call DescStackC("Push", _X_); + end; + + length S $ 12; + call DescStackC('Height', S); + put 'Height ' S; + + do until(strip(S) = "0"); + call DescStackC('Get', _X_); + call DescStackC('Height', S); + put S= _X_=; + output; + end; + + %* clear for further reuse *; + call DescStackC('Clear',''); + + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +**EXAMPLE 2.** Dynamic, Hash-based, and Numeric Ascending Ordered stack: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDHOrdStack(DescStackN, order=A); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example2; + + call missing(Sum, Avg, Min, Max, Cnt, Hgt, Peek); + do _X_ = 1,6,2,.,5,3,4; + call DescStackN("Put", _X_); + call DescStackN('Sum', Sum); + call DescStackN('Avg', Avg); + call DescStackN('Min', Min); + call DescStackN('Max', Max); + call DescStackN('Cnt', Cnt); + call DescStackN('Height', Hgt); + put (_ALL_) (=); + end; + + call DescStackN('Peek', Peek); + put Peek=; + + do _I_ = 1 to Hgt; + call DescStackN('Output', _X_); + keep _X_; + if _X_ > .z then output; + end; + + call DescStackN('Peek', Peek); + put Peek=; + + %* clear for further reuse *; + call DescStackN('Clear',.); + + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- +## >>> `%createDHPrtQueue()` macro: <<< ####################### + + +The `%createDHPrtQueue()` macro allows to generate +a `dynamic PRIORITY hash queue` which is a FCMP based approach +to create dynamically allocated numeric or character +[priority queue](https://en.wikipedia.org/wiki/Priority_queue) + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +%createDHPrtQueue( + fifoName + <,debug=0> + <,type=8> + <,newOnTop=+> + <,outlib=work.DFAfcmp.package> + <,hashexp=13> + <,header=1> +) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `queueName` - *Required*, creates a FCMP call subroutine which is also + a queue name. In the data step it is used in form of + a call subroutine, e.g. `call queueName("Bottom", 1, 3)`. + Has to satisfy FCMP function naming requirements, but with + maximum of 24 characters. + +* `debug=` - *Optional*, the default value is `0`. + If set to `1` then it turns on a debugging mode. + +* `type=` - *Optional*, the default value is `8`. + Indicates what *type* (numeric/character) and *length* + are data portion of generated array. Should be in line + with the LENGTH statement, e.g. `8`, `$ 30`, etc. + Determines if the `value` argument is numeric or character. + +* `newOnTop=` - *Optional*, the default value is `+`. + Indicates how to keep order in the same priority group, + allowed values are `+` or `-`. Plus(`+`) sets new elements + at the top of the group, minus(`-`) at the bottom. + +* `outlib=` - *Optional*, the default value is `work.DFAfcmp.package`. + It points the default location for new generated dynamic + function arrays compiled by FCMP. + *Hint!* Keep it as it is. + +* `hashexp=` - *Optional*, the default value is `13`. It is the default `hashexp=` + value for internal hash table used by the function. + +* `header=` - *Optional*, the default value is `1`. Indicates if + the `proc fcmp outlib = &outlib.;` header is added to + the executed code. If not 1 then no header is added. + +**Created function arguments description**: + +A function generated by the macro is: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +call &queueName.(IO, position, value) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +and accepts the following list of arguments and values: + +1. `IO` - is a *character* steering argument, possible + values and behaviour they call are the following: + - `O`, `Output`, `D`, `Dequeue`, `R`, `Return` - it pops/gets/outputs the data from the queue head (high priority), + - `B`, `Bottom` - it pops/gets/outputs the data from the queue tail (low priority), + - `I`, `Input`, `E`, `Enqueue`, `Insert` - it push/puts/inserts the data into the queue, + - `C`, `Clear` - it reduces a queue to an empty one, + - `H`, `Head` - it peeks the data from the queue head and NOT removes it, + - `T`, `Tail` - it peeks the data from the queue tail and NOT removes it, + - `Sum` - it returns sum of non-missing *numeric* elements of the queue, + - `Avg`, `Mean`, `Average` - it returns average of non-missing *numeric* elements of the queue, + - `Nonmiss`, `Cnt` - it returns number of non-missing elements of the queue, + - `Height` - it returns height of the queue. + +2. `position` - is a *numeric* argument and depends on the `IO` value. + Behaves in the following way: + - for `O`, `Output`, `D`, `Dequeue`, `R`, `Return` and `B`, `Bottom`, or `H`, `Head`, `T`, `Tail` + it holds a priority level of value popped from the queue, + - for `I`, `Input`, `E`, `Enqueue`, `Insert` it holds a priority level of value to be pushed into the queue, + - for `C` ignored, + - for *numeric* queue and `Sum`, `Nonmiss`, `Cnt`, `Avg`, `Mean`, `Average`, `Height` returns calculated summary value. + +3. `value` - is a *numeric* or *character* argument (determined by the `type=`) + and depends on the `IO` value. Behaves in the following way: + - for `O`, `Output`, `D`, `Dequeue`, `R`, `Return` and `B`, `Bottom` or `H`, `Head`, `T`, `Tail` + it holds a value popped from the queue, + - for `I`, `Input`, `E`, `Enqueue`, `Insert` it holds a value to be pushed into the queue, + - for `C` ignored, + - for *numeric* queue and `Sum`, `Nonmiss`, `Cnt`, `Avg`, `Mean`, `Average`, `Height` returns calculated summary value, + - otherwise does not modify value. + +The `position` and the `value` arguments are **outargs**, i.e. can be changed by the function. + + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Dynamic, Hash-based, and Character Priority queue: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDHPrtQueue(PriorityQueuePC, type = $ 12, newOnTop=+); + %createDHPrtQueue(PriorityQueueNC, type = $ 12, newOnTop=-); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example1; + + _I_ = .; + length _X_ _Y_ $ 3; + do _X_ = "AAA","BBB","CCC","AA","BB","CC","A","B","C"; + _I_ + 1; + call PriorityQueuePC("I", mod(_I_, 3), _X_); + call PriorityQueueNC("I", mod(_I_, 3), _X_); + end; + + Height = .; + call PriorityQueuePC('Height', Height, ''); + put Height=; + + do until(Height = 0); + call PriorityQueuePC('Dequeue', _I_, _X_); + call PriorityQueueNC("Dequeue", _I_, _Y_); + call PriorityQueueNC('Height', Height, ''); + put (_ALL_) (=); + output; + end; + + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +**EXAMPLE 2.** Dynamic, Hash-based, and Numeric Priority queue: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDHPrtQueue(PriorityQueueN); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example2; + + do _X_ = -5 to 5; + call PriorityQueueN("Enqueue", abs(_X_), _X_); + end; + + call missing(Sum, Avg, Cnt, Hgt); + call PriorityQueueN('Sum', ., Sum); + call PriorityQueueN('Avg', ., Avg); + call PriorityQueueN('Cnt', ., Cnt); + call PriorityQueueN('Height', ., Hgt); + put (_ALL_) (=); + + do _N_ = 1 to Hgt; + call PriorityQueueN("Dequeue", _X_, _Y_); + put _X_= _Y_=; + end; + + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- +## >>> `%createDHStack()` macro: <<< ####################### + +The `%createDHStack()` macro allows to generate +a `dynamic hash stack` which is a FCMP based approach +to create dynamically allocated numeric or character +[stack](https://en.wikipedia.org/wiki/Stack_(abstract_data_type)) + +Interesting reading about implementing a stack via hash table +can be found in *chapter 10.4* of the: +*"Data Management Solutions Using SAS Hash Table Operations: + A Business Intelligence Case Study"* book +by Paul Dorfman and Don Henderson. + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +%createDHStack( + fifoName + <,debug=0> + <,type=8> + <,outlib=work.DFAfcmp.package> + <,hashexp=13> + <,header=1> +) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `stackName` - *Required*, creates a FCMP call subroutine which is also + a stack name. In the data step it is used in form of + a call subroutine, e.g. `call stackName("Push", 3)`. + Has to satisfy FCMP function naming requirements, but with + maximum of 24 characters. + +* `debug=` - *Optional*, the default value is `0`. + If set to `1` then it turns on a debugging mode. + +* `type=` - *Optional*, the default value is `8`. + Indicates what *type* (numeric/character) and *length* + are data portion of generated array. Should be in line + with the LENGTH statement, e.g. `8`, `$ 30`, etc. + Determines if the `value` argument is numeric or character. + +* `outlib=` - *Optional*, the default value is `work.DFAfcmp.package`. + It points the default location for new generated dynamic + function arrays compiled by FCMP. + *Hint!* Keep it as it is. + +* `hashexp=` - *Optional*, the default value is `13`. It is the default `hashexp=` + value for internal hash table used by the function. + +* `header=` - *Optional*, the default value is `1`. Indicates if + the `proc fcmp outlib = &outlib.;` header is added to + the executed code. If not 1 then no header is added. + +**Created function arguments description**: + +A function generated by the macro is: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +call &stackName.(IO, value) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +and accepts the following list of arguments and values: + +1. `IO` - is a *character* steering argument, possible + values and behaviour they call are the following: + - `O`, `Output`, `Pop`, `G`, `Get`, `R`, `Return` - to get the data from a stack (and remove it from the top), + - `I`, `Input`, `Push`, `Put`, `Insert` - to insert the data into a stack, + - `C`, `Clear` - to reduce a stack to an empty one, + - `P`, `Peek` - to peek the data from a stack (and NOT remove it from the top), + - `Sum` - returns sum of non-missing numeric elements of a stack, + - `Avg`, `Mean`, `Average` - returns average of non-missing numeric elements of a stack, + - `Nonmiss`, `Cnt`, `Nnm` - returns number of non-missing elements of a stack, + - `Height` - returns height a stack, + +2. `value` - is a *numeric* or *character* argument (determined by the `type=`) + and depends on the `IO` value. Behaves in the following way: + - for `O`, `Output`, `Pop`, `G`, `Get`, `R`, `Return` it holds a value popped from a stack, + - for `I`, `Input`, `Push`, `Put`, `Insert` it holds a value to be pushed into a stack, + - for `C`, `Clear` it is ignored, + - for `P`, `Peek` it holds a value peeked from a stack, + - for `Sum`, `Nonmiss`, `Cnt`, `Avg`, `Mean`, `Average`, `Height` it returns calculated summary value, + +The `value` argument is **outarg**, i.e. can be changed by the function. + + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Dynamic, Hash-based, and Character stack: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDHStack(StackDHC, type = $ 12); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + + %let zeros = 6; *[to test bigger sizes]; + data Example1; + + t = time(); drop t; + do _I_ = 1 to 1e&zeros.; + _X_ = put(_I_*10, z12.); + call StackDHC("Put", _X_); + end; + t = time() - t; + + call StackDHC("Height", _X_); + put t= / _X_=; + + t = time(); + do _I_ = 1 to 1e&zeros. + 3; + call StackDHC('Pop', _X_); + output; + end; + t = time() - t; + + call StackDHC("Height", _X_); + put t= / _X_=; + + %* clear for further reuse *; + call StackDHC('Clear', ''); + + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +**EXAMPLE 2.** Dynamic, Hash-based, and Numeric stack: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDHStack(StackDHN); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example2; + + do _I_ = 1,.,2,.,3,.,4,.,5,.,6; + call StackDHN("Put", _I_); + end; + + call StackDHN("Sum", _I_); + put "Sum " _I_=; + + call StackDHN("Avg", _I_); + put "Avg " _I_=; + + call StackDHN("Cnt", _I_); + put "Cnt " _I_=; + + call StackDHN("Height", _I_); + put "Height " _I_=; + + _X_ = 0; + do _I_ = 1 to _I_; + call StackDHN('Pop', _X_); + output; + end; + + call StackDHN("Height", _I_); + put "Height " _I_=; + + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- + +## >>> `bit64orPROTOdfa()` proto function: <<< ####################### + +The **bit64orPROTOdfa()** is external *C* function, +this is the implementation of the *bitwise OR* operation +on doubles. A double is returned. + +**Caution!** For SAS numeric values *only* operations on first 53 bits are valid! + +The function is used **internally** by functions in the *DFA* package. + +### SYNTAX: ################################################################### + +The basic syntax is the following: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +bit64orPROTOdfa(i, j) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `i` - A double numeric argument. + +2. `j` - A double numeric argument. + +--- + +## >>> `bit64andPROTOdfa()` proto function: <<< ####################### + +The **bit64andPROTOdfa()** is external *C* function, +this is the implementation of the *bitwise AND* operation +on doubles. A double is returned. + +**Caution!** For SAS numeric values *only* operations on first 53 bits are valid! + +The function is used **internally** by functions in the *DFA* package. + +### SYNTAX: ################################################################### + +The basic syntax is the following: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +bit64andPROTOdfa(i, j) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `i` - A double numeric argument. + +2. `j` - A double numeric argument. + +--- + +## >>> `bit64orDFA()` subroutine: <<< ####################### + +The **bit64orDFA()** function is an alternative to +the 32 bit bitwise `BOR()` function working on SAS numerics. +Allows to work on *up to* 53 bits of SAS numeric value. + +The `bit64orDFA()` is an *internal* function of the `DFA` package. + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +bit64orDFA(a, b) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `a` - Argument is a SAS numeric values. + +2. `B` - Argument is a SAS numeric values. + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Basic test of `bit64orDFA()` and `bit64andDFA()` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + options ls = max ps = max; + %let M = 53 ; %* 53 is maximum valid value; + data _null_; + array bitmask [ 0: &M] _temporary_ ; + do P = 1 to &M ; + bitmask[P] = 2**(P-1) ; + put bitmask[P] = binary54. @; + put bitmask[P] = best32.; + end ; + bitmask[0] = bitmask[&M.] ; + put bitmask[0] = best32. /; + + a=0; + put a = binary54.; + do P = 1 to &M ; + a = BIT64ORDFA (a, bitmask[P]) ; + put a = binary54.; + end; + put; + + b = 0; + put b = binary54./; + do P = 1 to &M ; + b + (BIT64ANDDFA (a, bitmask[P]) ne .) ; + put b = best32.; + end; + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- + +## >>> `bit64andDFA()` subroutine: <<< ####################### + +The **bit64andDFA()** function is an alternative to +the 32 bit bitwise `BAND()` function working on SAS numerics. +Allows to work on *up to* 53 bits of SAS numeric value. + +The `bit64andDFA()` is an *internal* function of the `DFA` package. + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +bit64andDFA(a, b) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `a` - Argument is a SAS numeric values. + +2. `B` - Argument is a SAS numeric values. + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Basic test of `bit64orDFA()` and `bit64andDFA()` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + options ls = max ps = max; + %let M = 53 ; %* 53 is maximum valid value; + data _null_; + array bitmask [ 0: &M] _temporary_ ; + do P = 1 to &M ; + bitmask[P] = 2**(P-1) ; + put bitmask[P] = binary54. @; + put bitmask[P] = best32.; + end ; + bitmask[0] = bitmask[&M.] ; + put bitmask[0] = best32. /; + + a=0; + put a = binary54.; + do P = 1 to &M ; + a = BIT64ORDFA (a, bitmask[P]) ; + put a = binary54.; + end; + put; + + b = 0; + put b = binary54./; + do P = 1 to &M ; + b + (BIT64ANDDFA (a, bitmask[P]) ne .) ; + put b = best32.; + end; + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- + +## >>> `%createDFBitmap()` macro: <<< ####################### + +The `%createDFBitmap()` macro allows to generate +a `dynamic function bitmap` which is a FCMP based +approach to create *dynamically* allocated **numeric** +bitmnap. + +*Note:* Arrays provided by the macro are *one dimensional* arrays. + +The idea of a SAS bitmap is based on: +1. SGF Paper 3101-2019 + titeled **Re-Mapping A Bitmap** by *Paul M. Dorfman* and *Lessia S. Shajenko* + [https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2019/3101-2019.pdf](https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2019/3101-2019.pdf) +2. SUGI Paper 8-26 + titeled **Table Look-Up by Direct Addressing: Key-Indexing -- Bitmapping -- Hashing** by *Paul M. Dorfman* + [https://support.sas.com/resources/papers/proceedings/proceedings/sugi26/p008-26.pdf](https://support.sas.com/resources/papers/proceedings/proceedings/sugi26/p008-26.pdf) + +### SYNTAX: ################################################################### + +The basic syntax is the following, the `<...>` means optional parameters: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +%createDFBitmap( + bitmapName + <,debug=0> + <,outlib=work.DFAfcmp.package> + <,type=32> + <,header=1> +) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Arguments description**: + +1. `bitmapName` - *Required*, creates a FCMP call subroutine which is also + a bitmap name. In the data step it is used in form of + a call subroutine, e.g. `call bitmapName("Allocate", 3000)`. + Has to satisfy FCMP function naming requirements, but with + maximum of 24 characters. + +* `debug=` - *Optional*, the default value is `0`. + If set to `1` then it turns on a debugging mode. + +* `outlib=` - *Optional*, the default value is `work.DFAfcmp.package`. + It points the default location for new generated dynamic + function arrays compiled by FCMP. + *Hint!* Keep it as it is. + +* `type=` - *Optional*, the default value is `32`. Sets the type of + bitwise operations executed internaly on the bitmap. + The only valid values are `32` or `52`, + With 32 the `BOR()` and `BAND()` SAS functions are used + and with 52 the `bit64orDFA()` and `bit64and DFA()` FCMP + functions are used. + +* `header=` - *Optional*, the default value is `1`. Indicates if + the `proc fcmp outlib = &outlib.;` header is added to + the executed code. If not 1 then no header is added. + +**Created function arguments description**: + +A function generated by the macro is: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas +call &bitmapName.(IO, position, value) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +and accepts the following list of arguments and values: + +1. `IO` - is a *character* steering argument, possible + values and behaviour they call are the following: + - `Check`, `Test`, `T` - to get information if a bit is set to 1 (on) or not, + - `On`, `1` - to set a selected bit to 1, + - `Off`, `0` - to set a selected bit to 0, + - `C`, `Clear` - to reduce a bitmat to a single empty cell, + - `A`, `Allocate` - to reserve space for a bitmap and set all bits to 0, + - `A0`, `Allocate0` - to reserve space for a bitmap and set all bits to 0, + - `A1`, `Allocate1` - to reserve space for a bitmap and set all bits to 1, + - `D`, `Dim`, `Dimension` - to returns minimal and maximal index of the bitmap. + +2. `position` - is a *numeric* argument and depends on the `IO` value. + Behaves in the following way: + - for `On`, `Off`, `1`, `0`/ `Check`, `Test`, `T` it is a bitmap index, + - for `C`, `Clear` is ignored, + - for `A`, `Allocate` sets the value of the minposition, i.e. the minimal position of the bitmap index, + - for `D`, `Dimension` it returns value of the minposition, + +.3 `value` - is a *numeric* argument and depends on the `IO` value. + Behaves in the following way: + - for `Check`, `Test`, `T` it holds value retrieved from a bitmap on a given position, + - for `On`, `Off`, `1`, `0`, `C`, `Clear` is ignored, + - for `A`, `Allocate` it sets the value of the maxposition, i.e. maximal position of the array index, + - for `D`, `Dimension` it returns value of the maxposition + +The `position` and the `value` arguments are **outargs**, i.e. can be changed by the function. + +### EXAMPLES AND USECASES: #################################################### + +**EXAMPLE 1.** Bitmap of type 32: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFBitmap(MyBitmap32,type=32,debug=1); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example1; + call MyBitmap32("Allocate", -10, 100); + L = 0; H = 0; + call MyBitmap32("Dim", L, H); + put L= H=; + + * populate array with data ; + do i = L to H by 2; + put i @; + call MyBitmap32("1", i, .); + end; + put; + + * get values from the array ; + Value = .; + do i = L to H; + call MyBitmap32("T", i, Value); + put i= Value=; + end; + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**EXAMPLE 2.** Bitmap of type 52: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFBitmap(MyBitmap52,type=52,debug=1); + options APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example2; + call MyBitmap52("Allocate", -10, 100); + L = 0; H = 0; + call MyBitmap52("Dim", L, H); + put L= H=; + + * populate array with data ; + do i = L to H by 2; + put i @; + call MyBitmap52("1", i, .); + end; + put; + + * get values from the array ; + Value = .; + do i = L to H; + call MyBitmap52("T", i, Value); + put i= Value=; + end; + run; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**EXAMPLE 3.** Execution time test for type 52: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFBitmap(MyBigBitmap52,type=52,debug=0); + options FULLSTIMER APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example3; + call MyBigBitmap52("Allocate", -10, 2000000001); + L = 0; H = 0; + call MyBigBitmap52("Dim", L, H); + put L= H=; + + * populate bitmap with data ; + t = time(); + do i = L to H by 17; + call MyBigBitmap52("1", i, .); + x + 1; + end; + t = time() - t; + put "populate:" t= x=; + + * get values from the bitmap ; + t = time(); + Value = .; + do i = L to H; + call MyBigBitmap52("T", i, Value); + x + (-Value); + end; + t = time() - t; + put "search:" t= x=; + run; + +%* +L=-10 H=2000000001 +populate:t=55.902999878 x=117647060 +search:t=654.12900019 x=0 +NOTE: The data set WORK.EXAMPLE3 has 1 observations and 6 variables. +NOTE: DATA statement used (Total process time): + real time 11:50.42 + user cpu time 11:46.40 + system cpu time 0.45 seconds + memory 301791.12k + OS Memory 326332.00k +; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**EXAMPLE 4.** Execution time test for type 32: + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~sas + %createDFBitmap(MyBigBitmap32,type=32,debug=0); + options FULLSTIMER APPEND=(cmplib = WORK.DFAfcmp) ; + + data Example4; + call MyBigBitmap32("Allocate", -10, 2000000001); + L = 0; H = 0; + call MyBigBitmap32("Dim", L, H); + put L= H=; + + * populate bitmap with data ; + t = time(); + do i = L to H by 17; + call MyBigBitmap32("1", i, .); + x + 1; + end; + t = time() - t; + put "populate:" t= x=; + + * get values from the bitmap ; + t = time(); + Value = .; + do i = L to H; + call MyBigBitmap32("T", i, Value); + x + (-Value); + end; + t = time() - t; + put "populate:" t= x=; + run; + +%* +L=-10 H=2000000001 +populate:t=50.417999983 x=117647060 +populate:t=611.13600016 x=0 +NOTE: The data set WORK.EXAMPLE4 has 1 observations and 6 variables. +NOTE: DATA statement used (Total process time): + real time 11:02.07 + user cpu time 10:59.07 + system cpu time 1.46 seconds + memory 489583.90k + OS Memory 513876.00k +; +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +--- + +## >>> `generateArrays` exec: <<< ####################### + +The generateArrays exec file provides a **list of automatically generated examples** of functions +emulating data structures. + +The list of provided examples is the following: +- `SmpArray` - Simple Immutable Dynamic Function Array +- `SmpMtbArray` - Simple Mutable Dynamic Function Array +- `SrchArray` - Searchable Immutable Dynamic Function Array +- `SrchMtbArray` - Searchable Mutable Dynamic Function Array +- `DynArrayC` - Dynamic Hash-based Character Function Array (length 256 bytes) +- `StackC` - Dynamic Hash-based Character Stack (length 256 bytes) +- `StackN` - Dynamic Hash-based Numeric Stack +- `FifoC` - Dynamic Hash-based Character Fifo (length 256 bytes) +- `FifoN` - Dynamic Hash-based Character Fifo +- `DescStackC` - Dynamic Hash-based Character Descending Ordered Stack (length 256 bytes) +- `AscStackC` - Dynamic Hash-based Character Ascending Ordered Stack (length 256 bytes) +- `DescStackN` - Dynamic Hash-based Numeric Descending Ordered Stack +- `AscStackN` - Dynamic Hash-based Numeric Ascending Ordered Stack +- `PrtQueueNTC` - Dynamic Hash-based Character Priority Queue with *New on Top* (length 256 bytes) +- `PrtQueueNBC` - Dynamic Hash-based Character Priority Queue with *New on Bottom* (length 256 bytes) +- `PrtQueueNTN` - Dynamic Hash-based Numeric Priority Queue with *New on Top* +- `PrtQueueNBN` - Dynamic Hash-based Numeric Priority Queue with *New on Bottom* +- `Bitmap32` - Dynamic Function Bitmap on 32 bit +- `Bitmap52` - Dynamic Function Bitmap on 52 bit + +The `outlib=` option is set to `work.DFAfcmp.package`. The `cmplib=` option is updated automatically. + +--- +## >>> `generateArrays` clean: <<< ####################### + +The generateArrays clean file clears the list of automatically generated examples of functions +emulating data structures provided in the `generatearrays.sas` exec file. + +The `cmplib=` option is updated automatically. + +--- + +## License #################################################################### + +Copyright (c) 2019 Bartosz Jablonski + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +--- diff --git a/hist/0.5.7/dfa.zip b/hist/0.5.7/dfa.zip new file mode 100644 index 0000000..657adca Binary files /dev/null and b/hist/0.5.7/dfa.zip differ