Merge pull request #744 from sasjs/sasjs-server-csrf-cookie

fix(server): csrf cookie is created explicitly

.github/dependabot.yml

@@ -1,7 +1,7 @@
 version: 2
 updates:
-  - package-ecosystem: npm
-    directory: '/'
+  - package-ecosystem: "npm"
+    directory: "/"
     schedule:
-      interval: monthly
-    open-pull-requests-limit: 2
+      interval: "monthly"
+    open-pull-requests-limit: 1

README.md

@@ -2,7 +2,6 @@
 
 [![npm package][npm-image]][npm-url]
 [![Github Workflow][githubworkflow-image]][githubworkflow-url]
-[![Dependency Status][dependency-image]][dependency-url]
 [![npm](https://img.shields.io/npm/dt/@sasjs/adapter)]()
 ![Snyk Vulnerabilities for npm package](https://img.shields.io/snyk/vulnerabilities/npm/@sasjs/adapter)
 [![License](https://img.shields.io/apm/l/atomic-design-ui.svg)](/LICENSE)
@@ -16,7 +15,6 @@
 [githubworkflow-image]:https://github.com/sasjs/adapter/actions/workflows/build.yml/badge.svg
 [githubworkflow-url]:https://github.com/sasjs/adapter/blob/main/.github/workflows/build.yml
 [dependency-image]:https://david-dm.org/sasjs/adapter.svg
-[dependency-url]:https://github.com/sasjs/adapter/blob/main/package.json
 
 SASjs is a open-source framework for building Web Apps on SAS® platforms. You can use as much or as little of it as you like. This repository contains the JS adapter, the part that handles the to/from SAS communication on the client side. There are 3 ways to install it:
 
@@ -32,7 +30,7 @@ For more information on building web apps with SAS, check out [sasjs.io](https:/
 
 ## None of this makes sense. How do I build an app with it?
 
-Ok ok. Deploy this [example.html](https://raw.githubusercontent.com/sasjs/adapter/master/example.html) file to your web server, and update `servertype` to `SAS9` or `SASVIYA` depending on your backend.
+Ok ok. Deploy this [example.html](https://raw.githubusercontent.com/sasjs/adapter/master/example.html) file to your web server, and update `servertype` to `SAS9`, `SASVIYA`, or `SASJS` depending on your backend.
 
 The backend part can be deployed as follows:
 
@@ -52,7 +50,7 @@ parmcards4;
   %webout(OBJ,areas)
   %webout(CLOSE)
 ;;;;
-%mp_createwebservice(path=&appLoc/common,name=getdata)
+%mx_createwebservice(path=&appLoc/common,name=getdata)
 ```
 
 You now have a simple web app with a backend service!
@@ -96,10 +94,10 @@ const sasJs = new SASjs({your config})
 More on the config later.
 
 ### SAS Logon
-All authentication from the adapter is done against SASLogon.  There are two approaches that can be taken, which are configured using the `LoginMechanism` attribute of the sasJs config object (above):
+All authentication from the adapter is done against SASLogon.  There are two approaches that can be taken, which are configured using the `loginMechanism` attribute of the sasJs config object (above):
 
-* `LoginMechanism:'Redirected'` - this approach enables authentication through a SASLogon window, supporting complex authentication flows (such as 2FA) and avoids the need to handle passwords in the application itself.  The styling of the window can be modified using CSS.
-* `LoginMechanism:'Default'` - this approach requires that the username and password are captured, and used within the `.login()` method.  This can be helpful for development, or automated testing.
+* `loginMechanism:'Redirected'` - this approach enables authentication through a SASLogon window, supporting complex authentication flows (such as 2FA) and avoids the need to handle passwords in the application itself.  The styling of the window can be modified using CSS.
+* `loginMechanism:'Default'` - this approach requires that the username and password are captured, and used within the `.login()` method.  This can be helpful for development, or automated testing.
 
 Sample code for logging in with the `Default` approach:
 
@@ -127,7 +125,11 @@ sasJs.request("/path/to/my/service", dataObject)
   })
 ```
 
-We supply the path to the SAS service, and a data object.  The data object can be null (for services with no input), or can contain one or more tables in the following format:
+We supply the path to the SAS service, and a data object.  
+
+If the path starts with a `/` then it should be a full path to the service.  If there is no leading `/` then it is relative to the `appLoc`. 
+
+The data object can be null (for services with no input), or can contain one or more "tables" in the following format:
 
 ```javascript
 let dataObject={
@@ -143,7 +145,9 @@ let dataObject={
 };
 ```
 
-There are optional parameters such as a config object and a callback login function.
+These tables (`tablewith2cols1row` and `tablewith1col2rows`) will be created in SAS WORK after running `%webout(FETCH)` in your SAS service.
+
+The `request()` method also has optional parameters such as a config object and a callback login function.
 
 The response object will contain returned tables and columns.  Table names are always lowercase, and column names uppercase.
 
@@ -221,7 +225,7 @@ The SAS side is handled by a number of macros in the [macro core](https://github
 The following snippet shows the process of SAS tables arriving / leaving:
 
 ```sas
-/* fetch all input tables sent from frontend - they arrive as work tables */
+/* convert frontend input tables from into SASWORK datasets */
 %webout(FETCH)
 
 /* some sas code */
@@ -250,6 +254,8 @@ Where an entire column is made up of special missing numerics, there would be no
 %webout(OBJ,a,missing=STRING,showmeta=YES)
 ```
 
+The `%webout()` macro itself is just a wrapper for the [mp_jsonout](https://core.sasjs.io/mp__jsonout_8sas.html) macro.
+
 ## Configuration
 
 Configuration on the client side involves passing an object on startup, which can also be passed with each request.  Technical documentation on the SASjsConfig class is available [here](https://adapter.sasjs.io/classes/types.sasjsconfig.html).  The main config items are:
@@ -258,7 +264,7 @@ Configuration on the client side involves passing an object on startup, which ca
 * `serverType` - either `SAS9`, `SASVIYA` or `SASJS`.  The `SASJS` server type is for use with [sasjs/server](https://github.com/sasjs/server).
 * `serverUrl` - the location (including http protocol and port) of the SAS Server. Can be omitted, eg if serving directly from the SAS Web Server, or in streaming mode.
 * `debug` - if `true` then SAS Logs and extra debug information is returned.
-* `LoginMechanism` - either `Default` or `Redirected`.  See [SAS Logon](#sas-logon) section.
+* `loginMechanism` - either `Default` or `Redirected`.  See [SAS Logon](#sas-logon) section.
 * `useComputeApi` - Only relevant when the serverType is `SASVIYA`. If `true` the [Compute API](#using-the-compute-api) is used.  If `false` the [JES API](#using-the-jes-api) is used.  If `null` or `undefined` the [Web](#using-jes-web-app) approach is used.
 * `contextName` - Compute context on which the requests will be called.  If missing or not provided, defaults to `Job Execution Compute context`.
 * `requestHistoryLimit` - Request history limit. Increasing this limit may affect browser performance, especially with debug (logs) enabled.  Default is 10.
@@ -314,7 +320,7 @@ For more information and examples specific to this adapter you can check out the
 
 For more information on building web apps in general, check out these [resources](https://sasjs.io/training/resources/) or contact the [author](https://www.linkedin.com/in/allanbowe/) directly.
 
-If you are a SAS 9 or SAS Viya customer you can also request a copy of [Data Controller](https://datacontroller.io) - free for up to 5 users, this tool makes use of all parts of the SASjs framework.
+As a SAS customer you can also request a copy of [Data Controller](https://datacontroller.io) - free for up to 5 users, this tool makes use of all parts of the SASjs framework.
 
 
 ## Star Gazing

package-lock.json

@@ -12973,7 +12973,7 @@
     },
     "node_modules/npm/node_modules/rimraf/node_modules/brace-expansion": {
       "version": "1.1.11",
-      "dev": true,
+      "extraneous": true,
       "inBundle": true,
       "license": "MIT",
       "dependencies": {
@@ -12983,7 +12983,7 @@
     },
     "node_modules/npm/node_modules/rimraf/node_modules/glob": {
       "version": "7.2.3",
-      "dev": true,
+      "extraneous": true,
       "inBundle": true,
       "license": "ISC",
       "dependencies": {
@@ -13003,7 +13003,7 @@
     },
     "node_modules/npm/node_modules/rimraf/node_modules/minimatch": {
       "version": "3.1.2",
-      "dev": true,
+      "extraneous": true,
       "inBundle": true,
       "license": "ISC",
       "dependencies": {
@@ -26856,7 +26856,7 @@
             "brace-expansion": {
               "version": "1.1.11",
               "bundled": true,
-              "dev": true,
+              "extraneous": true,
               "requires": {
                 "balanced-match": "^1.0.0",
                 "concat-map": "0.0.1"
@@ -26865,7 +26865,7 @@
             "glob": {
               "version": "7.2.3",
               "bundled": true,
-              "dev": true,
+              "extraneous": true,
               "requires": {
                 "fs.realpath": "^1.0.0",
                 "inflight": "^1.0.4",
@@ -26878,7 +26878,7 @@
             "minimatch": {
               "version": "3.1.2",
               "bundled": true,
-              "dev": true,
+              "extraneous": true,
               "requires": {
                 "brace-expansion": "^1.1.7"
               }

sasjs-tests/src/App.tsx

@@ -11,12 +11,13 @@ import { fileUploadTests } from './testSuites/FileUpload'
 const App = (): ReactElement<{}> => {
   const { adapter, config } = useContext(AppContext)
   const [testSuites, setTestSuites] = useState<TestSuite[]>([])
+  const appLoc = config.sasJsConfig.appLoc
 
   useEffect(() => {
     if (adapter) {
       const testSuites = [
         basicTests(adapter, config.userName, config.password),
-        sendArrTests(adapter),
+        sendArrTests(adapter, appLoc),
         sendObjTests(adapter),
         specialCaseTests(adapter),
         sasjsRequestTests(adapter),
@@ -24,12 +25,12 @@ const App = (): ReactElement<{}> => {
       ]
 
       if (adapter.getSasjsConfig().serverType === 'SASVIYA') {
-        testSuites.push(computeTests(adapter))
+        testSuites.push(computeTests(adapter, appLoc))
       }
 
       setTestSuites(testSuites)
     }
-  }, [adapter, config])
+  }, [adapter, config, appLoc])
 
   return (
     <div className="app">

sasjs-tests/src/testSuites/Compute.ts

@@ -3,7 +3,7 @@ import { TestSuite } from '@sasjs/test-framework'
 
 const stringData: any = { table1: [{ col1: 'first col value' }] }
 
-export const computeTests = (adapter: SASjs): TestSuite => ({
+export const computeTests = (adapter: SASjs, appLoc: string): TestSuite => ({
   name: 'Compute',
   tests: [
     {
@@ -35,7 +35,7 @@ export const computeTests = (adapter: SASjs): TestSuite => ({
       description: 'Should start a compute job and return the session',
       test: () => {
         const data: any = { table1: [{ col1: 'first col value' }] }
-        return adapter.startComputeJob('/Public/app/common/sendArr', data)
+        return adapter.startComputeJob(`${appLoc}/common/sendArr`, data)
       },
       assertion: (res: any) => {
         const expectedProperties = ['id', 'applicationName', 'attributes']

sasjs-tests/src/testSuites/RequestData.ts

@@ -45,14 +45,14 @@ const getLargeObjectData = () => {
   return data
 }
 
-export const sendArrTests = (adapter: SASjs): TestSuite => ({
+export const sendArrTests = (adapter: SASjs, appLoc: string): TestSuite => ({
   name: 'sendArr',
   tests: [
     {
       title: 'Absolute paths',
       description: 'Should work with absolute paths to SAS jobs',
       test: () => {
-        return adapter.request('/Public/app/common/sendArr', stringData)
+        return adapter.request(`${appLoc}/common/sendArr`, stringData)
       },
       assertion: (res: any) => {
         return res.table1[0][0] === stringData.table1[0].col1

src/SASjs.ts

@@ -1,4 +1,4 @@
-import { compareTimestamps, asyncForEach } from './utils'
+import { compareTimestamps, asyncForEach, validateInput } from './utils'
 import {
   SASjsConfig,
   UploadFile,
@@ -686,7 +686,7 @@ export default class SASjs {
       ...config
     }
 
-    const validationResult = this.validateInput(data)
+    const validationResult = validateInput(data)
 
     // status is true if the data passes validation checks above
     if (validationResult.status) {
@@ -748,74 +748,6 @@ export default class SASjs {
     }
   }
 
-  /**
-   * This function validates the input data structure and table naming convention
-   *
-   * @param data A json object that contains one or more tables, it can also be null
-   * @returns An object which contains two attributes: 1) status: boolean, 2) msg: string
-   */
-  private validateInput(data: { [key: string]: any } | null): {
-    status: boolean
-    msg: string
-  } {
-    if (data === null) return { status: true, msg: '' }
-
-    const isSasFormatsTable = (key: string) =>
-      key.match(/^\$.*/) && Object.keys(data).includes(key.replace(/^\$/, ''))
-
-    for (const key in data) {
-      if (!key.match(/^[a-zA-Z_]/) && !isSasFormatsTable(key)) {
-        return {
-          status: false,
-          msg: 'First letter of table should be alphabet or underscore.'
-        }
-      }
-
-      if (!key.match(/^[a-zA-Z_][a-zA-Z0-9_]*$/) && !isSasFormatsTable(key)) {
-        return { status: false, msg: 'Table name should be alphanumeric.' }
-      }
-
-      if (key.length > 32) {
-        return {
-          status: false,
-          msg: 'Maximum length for table name could be 32 characters.'
-        }
-      }
-
-      if (this.getType(data[key]) !== 'Array' && !isSasFormatsTable(key)) {
-        return {
-          status: false,
-          msg: 'Parameter data contains invalid table structure.'
-        }
-      }
-
-      for (let i = 0; i < data[key].length; i++) {
-        if (this.getType(data[key][i]) !== 'object') {
-          return {
-            status: false,
-            msg: `Table ${key} contains invalid structure.`
-          }
-        }
-      }
-    }
-
-    return { status: true, msg: '' }
-  }
-
-  /**
-   * this function returns the type of variable
-   *
-   * @param data it could be anything, like string, array, object etc.
-   * @returns a string which tells the type of input parameter
-   */
-  private getType(data: any): string {
-    if (Array.isArray(data)) {
-      return 'Array'
-    } else {
-      return typeof data
-    }
-  }
-
   /**
    * Creates the folders and services at the given location `appLoc` on the given server `serverUrl`.
    * @param serviceJson - the JSON specifying the folders and services to be created.

src/auth/AuthManager.ts

@@ -23,7 +23,7 @@ export class AuthManager {
         ? '/SASLogon/logout?'
         : this.serverType === ServerType.SasViya
         ? '/SASLogon/logout.do?'
-        : '/SASjsApi/auth/logout'
+        : '/SASLogon/logout'
   }
 
   /**
@@ -223,9 +223,17 @@ export class AuthManager {
 
   private async getNewLoginForm() {
     if (this.serverType === ServerType.Sasjs) {
-      // server will be sending CSRF cookie,
+      // server will be sending CSRF token in response,
+      // need to save in cookie so that,
       // http client will use it automatically
-      return this.requestClient.get('/', undefined)
+      return this.requestClient.get('/', undefined).then(({ result }) => {
+        const cookie =
+          /<script>document.cookie = '(XSRF-TOKEN=.*; Max-Age=86400; SameSite=Strict; Path=\/;)'<\/script>/.exec(
+            result as string
+          )?.[1]
+
+        if (cookie) document.cookie = cookie
+      })
     }
 
     const { result: formResponse } = await this.requestClient.get<string>(
@@ -334,19 +342,9 @@ export class AuthManager {
 
   /**
    * Logs out of the configured SAS server.
-   * @param accessToken - an optional access token is required for SASjs server type.
+   *
    */
   public async logOut() {
-    if (this.serverType === ServerType.Sasjs) {
-      return this.requestClient
-        .delete(this.logoutUrl)
-        .catch(() => true)
-        .finally(() => {
-          this.requestClient.clearLocalStorageTokens()
-          return true
-        })
-    }
-
     this.requestClient.clearCsrfTokens()
 
     return this.requestClient.get(this.logoutUrl, undefined).then(() => true)

src/auth/refreshTokensForSasjs.ts

@@ -28,7 +28,7 @@ export async function refreshTokensForSasjs(
       }
     })
     .catch((err) => {
-      throw prefixMessage(err, 'Error while refreshing tokens')
+      throw prefixMessage(err, 'Error while refreshing tokens: ')
     })
 
   return authResponse

src/auth/refreshTokensForViya.ts

@@ -42,7 +42,7 @@ export async function refreshTokensForViya(
     )
     .then((res) => res.result)
     .catch((err) => {
-      throw prefixMessage(err, 'Error while refreshing tokens')
+      throw prefixMessage(err, 'Error while refreshing tokens: ')
     })
 
   return authResponse

src/auth/spec/refreshTokensForSasjs.spec.ts

@@ -27,17 +27,20 @@ describe('refreshTokensForSasjs', () => {
 
   it('should handle errors while refreshing tokens', async () => {
     setupMocks()
+
     const refresh_token = generateToken(30)
+    const tokenError = 'unable to verify the first certificate'
+
     jest
       .spyOn(requestClient, 'post')
-      .mockImplementation(() => Promise.reject('Token Error'))
+      .mockImplementation(() => Promise.reject(tokenError))
 
     const error = await refreshTokensForSasjs(
       requestClient,
       refresh_token
     ).catch((e: any) => e)
 
-    expect(error).toContain('Error while refreshing tokens')
+    expect(error).toEqual(`Error while refreshing tokens: ${tokenError}`)
   })
 })
 

src/auth/spec/refreshTokensForViya.spec.ts

@@ -46,17 +46,20 @@ describe('refreshTokensForViya', () => {
 
   it('should handle errors while refreshing tokens', async () => {
     setupMocks()
+
     const access_token = generateToken(30)
     const refresh_token = generateToken(30)
+    const tokenError = 'unable to verify the first certificate'
     const authConfig: AuthConfig = {
       access_token,
       refresh_token,
       client: 'cl13nt',
       secret: 's3cr3t'
     }
+
     jest
       .spyOn(requestClient, 'post')
-      .mockImplementation(() => Promise.reject('Token Error'))
+      .mockImplementation(() => Promise.reject(tokenError))
 
     const error = await refreshTokensForViya(
       requestClient,
@@ -65,7 +68,7 @@ describe('refreshTokensForViya', () => {
       authConfig.refresh_token
     ).catch((e: any) => e)
 
-    expect(error).toContain('Error while refreshing tokens')
+    expect(error).toEqual(`Error while refreshing tokens: ${tokenError}`)
   })
 })
 

src/file/generateTableUploadForm.ts

@@ -1,5 +1,5 @@
 import * as NodeFormData from 'form-data'
-import { convertToCSV } from '../utils/convertToCsv'
+import { convertToCSV, isFormatsTable } from '../utils/convertToCsv'
 import { splitChunks } from '../utils/splitChunks'
 
 export const generateTableUploadForm = (
@@ -13,7 +13,8 @@ export const generateTableUploadForm = (
   for (const tableName in data) {
     tableCounter++
 
-    sasjsTables.push(tableName)
+    // Formats table should not be sent as part of 'sasjs_tables'
+    if (!isFormatsTable(tableName)) sasjsTables.push(tableName)
 
     const csv = convertToCSV(data, tableName)
 

src/request/RequestClient.ts

@@ -19,7 +19,7 @@ import {
   parseSourceCode,
   createAxiosInstance
 } from '../utils'
-import { InvalidCsrfError } from '../types/errors/InvalidCsrfError'
+import { InvalidSASjsCsrfError } from '../types/errors/InvalidSASjsCsrfError'
 
 export interface HttpClient {
   get<T>(
@@ -499,12 +499,20 @@ export class RequestClient implements HttpClient {
       throw e
     }
 
-    if (e instanceof InvalidCsrfError) {
-      // Fetching root will inject CSRF token in cookie
+    if (e instanceof InvalidSASjsCsrfError) {
+      // Fetching root and creating CSRF cookie
       await this.httpClient
         .get('/', {
           withCredentials: true
         })
+        .then((response) => {
+          const cookie =
+            /<script>document.cookie = '(XSRF-TOKEN=.*; Max-Age=86400; SameSite=Strict; Path=\/;)'<\/script>/.exec(
+              response.data
+            )?.[1]
+
+          if (cookie) document.cookie = cookie
+        })
         .catch((err) => {
           throw prefixMessage(err, 'Error while re-fetching CSRF token.')
         })
@@ -611,8 +619,11 @@ export const throwIfError = (response: AxiosResponse) => {
         throw new LoginRequiredError(response.data)
       }
 
-      if (response.data.toLowerCase() === 'invalid csrf token!') {
-        throw new InvalidCsrfError()
+      if (
+        typeof response.data === 'string' &&
+        response.data.toLowerCase() === 'invalid csrf token!'
+      ) {
+        throw new InvalidSASjsCsrfError()
       }
       break
     case 401:
@@ -703,7 +714,14 @@ const parseError = (data: string) => {
   } catch (_) {}
 
   try {
+    // There are some edge cases in which the SAS mp_abort macro
+    // (https://core.sasjs.io/mp__abort_8sas.html) is unable to
+    // provide a clean exit.  In this case the JSON response will
+    // be wrapped in >>weboutBEGIN<< and >>weboutEND<< strings.
+    // Therefore, if the first string exists, we won't throw an
+    // error just yet (the parser may yet throw one instead)
     const hasError =
+      !data?.match(/>>weboutBEGIN<</) &&
       !!data?.match(/Stored Process Error/i) &&
       !!data?.match(/This request completed with errors./i)
     if (hasError) {

src/test/utils/formatDataForRequest.spec.ts

@@ -11,13 +11,16 @@ describe('formatDataForRequest', () => {
         { var1: 'string', var2: 232, nullvar: '_' },
         { var1: 'string', var2: 232, nullvar: 0 },
         { var1: 'string', var2: 232, nullvar: 'z' },
-        { var1: 'string', var2: 232, nullvar: null }
+        { var1: 'string', var2: 232, nullvar: null },
+        { var1: 'string', var2: 232, nullvar: '.A' },
+        { var1: 'string', var2: 232, nullvar: '._' },
+        { var1: 'string', var2: 232, nullvar: '.' }
       ],
       [`$${testTable}`]: { formats: { var1: '$char12.', nullvar: 'best.' } }
     }
 
     const expectedOutput = {
-      sasjs1data: `var1:$char12. var2:best. nullvar:best.\r\nstring,232,.a\r\nstring,232,.b\r\nstring,232,._\r\nstring,232,0\r\nstring,232,.z\r\nstring,232,.`,
+      sasjs1data: `var1:$char12. var2:best. nullvar:best.\r\nstring,232,.a\r\nstring,232,.b\r\nstring,232,._\r\nstring,232,0\r\nstring,232,.z\r\nstring,232,.\r\nstring,232,.a\r\nstring,232,._\r\nstring,232,.`,
       sasjs_tables: testTable
     }
 

src/types/errors/InvalidCsrfError.ts

@@ -1,9 +0,0 @@
-export class InvalidCsrfError extends Error {
-  constructor() {
-    const message = 'Invalid CSRF token!'
-
-    super(`Auth error: ${message}`)
-    this.name = 'InvalidCsrfError'
-    Object.setPrototypeOf(this, InvalidCsrfError.prototype)
-  }
-}

src/types/errors/InvalidSASjsCsrfError.ts

@@ -0,0 +1,9 @@
+export class InvalidSASjsCsrfError extends Error {
+  constructor() {
+    const message = 'Invalid CSRF token!'
+
+    super(`Auth error: ${message}`)
+    this.name = 'InvalidSASjsCsrfError'
+    Object.setPrototypeOf(this, InvalidSASjsCsrfError.prototype)
+  }
+}

src/utils/convertToCsv.ts

@@ -1,4 +1,5 @@
 import { isSpecialMissing } from '@sasjs/utils/input/validators'
+import { prefixMessage } from '@sasjs/utils/error'
 
 /**
  * Converts the given JSON object array to a CSV string.
@@ -9,7 +10,10 @@ export const convertToCSV = (
   tableName: string
 ) => {
   if (!data[tableName]) {
-    throw new Error('No table provided to be converted to CSV')
+    throw prefixMessage(
+      'No table provided to be converted to CSV.',
+      'Error while converting to CSV. '
+    )
   }
 
   const table = data[tableName]
@@ -137,7 +141,9 @@ export const convertToCSV = (
           )
         }
 
-        return `.${value.toLowerCase()}`
+        const dot = value.includes('.') ? '' : '.'
+
+        return `${dot}${value.toLowerCase()}`
       }
 
       // if there any present, it should have preceding (") for escaping
@@ -170,6 +176,12 @@ export const convertToCSV = (
   return finalCSV
 }
 
+/**
+ * Checks if table is table of formats (table name should start from '$' character).
+ * @param tableName - table name.
+ */
+export const isFormatsTable = (tableName: string) => /^\$.*/.test(tableName)
+
 const getByteSize = (str: string) => {
   let byteSize = str.length
   for (let i = str.length - 1; i >= 0; i--) {

src/utils/formatDataForRequest.ts

@@ -1,4 +1,4 @@
-import { convertToCSV } from './convertToCsv'
+import { convertToCSV, isFormatsTable } from './convertToCsv'
 import { splitChunks } from './splitChunks'
 
 export const formatDataForRequest = (data: any) => {
@@ -8,7 +8,7 @@ export const formatDataForRequest = (data: any) => {
 
   for (const tableName in data) {
     if (
-      tableName.match(/^\$.*/) &&
+      isFormatsTable(tableName) &&
       Object.keys(data).includes(tableName.replace(/^\$/, ''))
     ) {
       continue
@@ -16,7 +16,8 @@ export const formatDataForRequest = (data: any) => {
 
     tableCounter++
 
-    sasjsTables.push(tableName)
+    // Formats table should not be sent as part of 'sasjs_tables'
+    if (!isFormatsTable(tableName)) sasjsTables.push(tableName)
 
     const csv = convertToCSV(data, tableName)
 

src/utils/index.ts

@@ -1,20 +1,21 @@
+export * from './appendExtraResponseAttributes'
 export * from './asyncForEach'
 export * from './compareTimestamps'
 export * from './convertToCsv'
 export * from './createAxiosInstance'
 export * from './delay'
+export * from './fetchLogByChunks'
+export * from './getValidJson'
 export * from './isNode'
 export * from './isRelativePath'
 export * from './isUri'
 export * from './isUrl'
 export * from './needsRetry'
 export * from './parseGeneratedCode'
-export * from './parseSourceCode'
 export * from './parseSasViyaLog'
+export * from './parseSourceCode'
+export * from './parseViyaDebugResponse'
+export * from './parseWeboutResponse'
 export * from './serialize'
 export * from './splitChunks'
-export * from './parseWeboutResponse'
-export * from './fetchLogByChunks'
-export * from './getValidJson'
-export * from './parseViyaDebugResponse'
-export * from './appendExtraResponseAttributes'
+export * from './validateInput'

src/utils/spec/convertToCsv.spec.ts

@@ -1,4 +1,4 @@
-import { convertToCSV } from './convertToCsv'
+import { convertToCSV, isFormatsTable } from '../convertToCsv'
 
 describe('convertToCsv', () => {
   const tableName = 'testTable'
@@ -216,7 +216,9 @@ describe('convertToCsv', () => {
     const data = { [tableName]: [{ var1: 'string' }] }
 
     expect(() => convertToCSV(data, 'wrongTableName')).toThrow(
-      new Error('No table provided to be converted to CSV')
+      new Error(
+        'Error while converting to CSV. No table provided to be converted to CSV.'
+      )
     )
   })
 
@@ -226,3 +228,15 @@ describe('convertToCsv', () => {
     expect(convertToCSV(data, tableName)).toEqual('')
   })
 })
+
+describe('isFormatsTable', () => {
+  const tableName = 'sometable'
+
+  it('should return true if table name match pattern of formats table', () => {
+    expect(isFormatsTable(`$${tableName}`)).toEqual(true)
+  })
+
+  it('should return false if table  name does not match pattern of formats table', () => {
+    expect(isFormatsTable(tableName)).toEqual(false)
+  })
+})

src/utils/spec/validateInput.spec.ts

@@ -0,0 +1,84 @@
+import {
+  validateInput,
+  INVALID_TABLE_STRUCTURE,
+  MORE_INFO
+} from '../validateInput'
+
+const tableArray = [{ col1: 'first col value' }]
+const stringData: any = { table1: tableArray }
+
+describe('validateInput', () => {
+  it('should not return an error message if input data valid', () => {
+    const validationResult = validateInput(stringData)
+    expect(validationResult).toEqual({
+      status: true,
+      msg: ''
+    })
+  })
+
+  it('should not return an error message if input data is null', () => {
+    const validationResult = validateInput(null)
+    expect(validationResult).toEqual({
+      status: true,
+      msg: ''
+    })
+  })
+
+  it('should return an error message if input data is an array', () => {
+    const validationResult = validateInput(tableArray)
+    expect(validationResult).toEqual({
+      status: false,
+      msg: INVALID_TABLE_STRUCTURE
+    })
+  })
+
+  it('should return an error message if first letter of table is neither alphabet nor underscore', () => {
+    const validationResult = validateInput({ '1stTable': tableArray })
+    expect(validationResult).toEqual({
+      status: false,
+      msg: 'First letter of table should be alphabet or underscore.'
+    })
+  })
+
+  it('should return an error message if table name contains a character other than alphanumeric or underscore', () => {
+    const validationResult = validateInput({ 'table!': tableArray })
+    expect(validationResult).toEqual({
+      status: false,
+      msg: 'Table name should be alphanumeric.'
+    })
+  })
+
+  it('should return an error message if length of table name contains exceeds 32', () => {
+    const validationResult = validateInput({
+      xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx: tableArray
+    })
+    expect(validationResult).toEqual({
+      status: false,
+      msg: 'Maximum length for table name could be 32 characters.'
+    })
+  })
+
+  it('should return an error message if table does not have array of objects', () => {
+    const validationResult = validateInput({ table: stringData })
+    expect(validationResult).toEqual({
+      status: false,
+      msg: INVALID_TABLE_STRUCTURE
+    })
+  })
+
+  it('should return an error message if a table array has an item other than object', () => {
+    const validationResult = validateInput({ table1: ['invalid'] })
+    expect(validationResult).toEqual({
+      status: false,
+      msg: `Table table1 contains invalid structure. ${MORE_INFO}`
+    })
+  })
+
+  it('should return an error message if a row in a table contains an column with undefined value', () => {
+    const validationResult = validateInput({ table1: [{ column: undefined }] })
+    expect(validationResult).toEqual({
+      status: false,
+      msg: `A row in table table1 contains invalid value. Can't assign undefined to column.`
+    })
+  })
+})

src/utils/validateInput.ts

@@ -0,0 +1,90 @@
+export const MORE_INFO =
+  'For more info see https://sasjs.io/sasjs-adapter/#request-response'
+export const INVALID_TABLE_STRUCTURE = `Parameter data contains invalid table structure. ${MORE_INFO}`
+
+/**
+ * This function validates the input data structure and table naming convention
+ *
+ * @param data A json object that contains one or more tables, it can also be null
+ * @returns An object which contains two attributes: 1) status: boolean, 2) msg: string
+ */
+export const validateInput = (
+  data: { [key: string]: any } | null
+): {
+  status: boolean
+  msg: string
+} => {
+  if (data === null) return { status: true, msg: '' }
+
+  if (getType(data) !== 'object') {
+    return {
+      status: false,
+      msg: INVALID_TABLE_STRUCTURE
+    }
+  }
+
+  const isSasFormatsTable = (key: string) =>
+    key.match(/^\$.*/) && Object.keys(data).includes(key.replace(/^\$/, ''))
+
+  for (const key in data) {
+    if (!key.match(/^[a-zA-Z_]/) && !isSasFormatsTable(key)) {
+      return {
+        status: false,
+        msg: 'First letter of table should be alphabet or underscore.'
+      }
+    }
+
+    if (!key.match(/^[a-zA-Z_][a-zA-Z0-9_]*$/) && !isSasFormatsTable(key)) {
+      return { status: false, msg: 'Table name should be alphanumeric.' }
+    }
+
+    if (key.length > 32) {
+      return {
+        status: false,
+        msg: 'Maximum length for table name could be 32 characters.'
+      }
+    }
+
+    if (getType(data[key]) !== 'Array' && !isSasFormatsTable(key)) {
+      return {
+        status: false,
+        msg: INVALID_TABLE_STRUCTURE
+      }
+    }
+
+    for (const item of data[key]) {
+      if (getType(item) !== 'object') {
+        return {
+          status: false,
+          msg: `Table ${key} contains invalid structure. ${MORE_INFO}`
+        }
+      } else {
+        const attributes = Object.keys(item)
+        for (const attribute of attributes) {
+          if (item[attribute] === undefined) {
+            return {
+              status: false,
+              msg: `A row in table ${key} contains invalid value. Can't assign undefined to ${attribute}.`
+            }
+          }
+        }
+      }
+    }
+  }
+
+  return { status: true, msg: '' }
+}
+
+/**
+ * this function returns the type of variable
+ *
+ * @param data it could be anything, like string, array, object etc.
+ * @returns a string which tells the type of input parameter
+ */
+const getType = (data: any): string => {
+  if (Array.isArray(data)) {
+    return 'Array'
+  } else {
+    return typeof data
+  }
+}