sasjs/adapter
1
0
Fork 0
mirror of https://github.com/sasjs/adapter.git synced 2025-12-11 01:14:36 +00:00
Code Issues Projects Releases Wiki Activity

Merge branch 'master' into job-refresh-tokens

Browse Source
This commit is contained in:
Krishna Acondy
2021-06-25 09:15:33 +01:00
committed by GitHub
parent 2b16be3aef 0b755b7304
commit eae8694a29
7 changed files with 75 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
PULL_REQUEST_TEMPLATE.md
View File

@@ -12,6 +12,8 @@ What code changes have been made to achieve the intent.
## Checks
No PR (that involves a non-trivial code change) should be merged, unless all four of the items below are confirmed! If an urgent fix is needed - use a tar file.
- [ ] Code is formatted correctly (`npm run lint:fix`).
- [ ] All unit tests are passing (`npm test`).
- [ ] All `sasjs-cli` unit tests are passing (`npm test`).

34
README.md
View File

@@ -172,35 +172,37 @@ Configuration on the client side involves passing an object on startup, which ca
* `serverType` - either `SAS9` or `SASVIYA`.
* `serverUrl` - the location (including http protocol and port) of the SAS Server. Can be omitted, eg if serving directly from the SAS Web Server, or in streaming mode.
* `debug` - if `true` then SAS Logs and extra debug information is returned.
* `useComputeApi` - if `true` and the serverType is `SASVIYA` then the REST APIs will be called directly (rather than using the JES web service).
* `contextName` - if missing or blank, and `useComputeApi` is `true` and `serverType` is `SASVIYA` then the JES API will be used.
* `useComputeApi` - Only relevant when the serverType is `SASVIYA`. If `true` the [Compute API](#using-the-compute-api) is used. If `false` the [JES API](#using-the-jes-api) is used. If `null` or `undefined` the [Web](#using-jes-web-app) approach is used.
* `contextName` - Compute context on which the requests will be called. If missing or not provided, defaults to `Job Execution Compute context`.
The adapter supports a number of approaches for interfacing with Viya (`serverType` is `SASVIYA`). For maximum performance, be sure to [configure your compute context](https://sasjs.io/guide-viya/#shared-account-and-server-re-use) with `reuseServerProcesses` as `true` and a system account in `runServerAs`. This functionality is available since Viya 3.5. This configuration is supported when [creating contexts using the CLI](https://sasjs.io/sasjs-cli-context/#sasjs-context-create).
### Using JES Web App
In this setup, all requests are routed through the JES web app, at `YOURSERVER/SASJobExecution`. This is the most reliable method, and also the slowest. One request is made to the JES app, and remaining requests (getting job uri, session spawning, passing parameters, running the program, fetching the log) are made on the SAS server by the JES app.
```
{
appLoc:"/Your/Path",
serverType:"SASVIYA"
}
```
### Using the JES API
Here we are running Jobs using the Job Execution Service except this time we are making the requests directly using the REST API instead of through the JES Web App. This is helpful when we need to call web services outside of a browser (eg with the SASjs CLI or other commandline tools). To save one network request, the adapter prefetches the JOB URIs and passes them in the `__job` parameter.
In this setup, all requests are routed through the JES web app, at `YOURSERVER/SASJobExecution?_program=/your/program`. This is the most reliable method, and also the slowest. One request is made to the JES app, and remaining requests (getting job uri, session spawning, passing parameters, running the program, fetching the log) are handled by the SAS server inside the JES app.
```
{
appLoc:"/Your/Path",
serverType:"SASVIYA",
useComputeApi: true
contextName: 'yourComputeContext'
}
```
### Using the JES API
Here we are running Jobs using the Job Execution Service except this time we are making the requests directly using the REST API instead of through the JES Web App. This is helpful when we need to call web services outside of a browser (eg with the SASjs CLI or other commandline tools). To save one network request, the adapter prefetches the JOB URIs and passes them in the `__job` parameter. Depending on your network bandwidth, it may or may not be faster than the JES Web approach.
```
{
appLoc:"/Your/Path",
serverType:"SASVIYA",
useComputeApi: false,
contextName: 'yourComputeContext'
}
```
### Using the Compute API
This approach is by far the fastest, as a result of the optimisations we have built into the adapter. With this configuration, in the first sasjs request, we take a URI map of the services in the target folder, and create a session manager - which spawns an extra session. The next time a request is made, the adapter will use the 'hot' session. Sessions are deleted after every use, which actually makes this _less_ resource intensive than a typical JES web app, in which all sessions are kept alive by default for 15 minutes.
This approach is by far the fastest, as a result of the optimisations we have built into the adapter. With this configuration, in the first sasjs request, we take a URI map of the services in the target folder, and create a session manager. This manager will spawn a additional session every time a request is made. Subsequent requests will use the existing 'hot' session, if it exists. Sessions are always deleted after every use, which actually makes this _less_ resource intensive than a typical JES web app, in which all sessions are kept alive by default for 15 minutes.
```
{
@@ -225,4 +227,4 @@ If you are a SAS 9 or SAS Viya customer you can also request a copy of [Data Con
If you find this library useful, help us grow our star graph!
![](https://starchart.cc/sasjs/adapter.svg)
![](https://starchart.cc/sasjs/adapter.svg)

32
src/FileUploader.ts
View File

@@ -2,15 +2,19 @@ import { isUrl } from './utils'
import { UploadFile } from './types/UploadFile'
import { ErrorResponse, LoginRequiredError } from './types/errors'
import { RequestClient } from './request/RequestClient'
import { ServerType } from '@sasjs/utils/types'
import SASjs from './SASjs'
import { Server } from 'https'
import { SASjsConfig } from './types'
import { config } from 'process'
export class FileUploader {
constructor(
private appLoc: string,
serverUrl: string,
private sasjsConfig: SASjsConfig,
private jobsPath: string,
private requestClient: RequestClient
) {
if (serverUrl) isUrl(serverUrl)
if (this.sasjsConfig.serverUrl) isUrl(this.sasjsConfig.serverUrl)
}
public uploadFile(sasJob: string, files: UploadFile[], params: any) {
@@ -29,8 +33,8 @@ export class FileUploader {
}
}
const program = this.appLoc
? this.appLoc.replace(/\/?$/, '/') + sasJob.replace(/^\//, '')
const program = this.sasjsConfig.appLoc
? this.sasjsConfig.appLoc.replace(/\/?$/, '/') + sasJob.replace(/^\//, '')
: sasJob
const uploadUrl = `${this.jobsPath}/?${
'_program=' + program
@@ -44,6 +48,12 @@ export class FileUploader {
const csrfToken = this.requestClient.getCsrfToken('file')
if (csrfToken) formData.append('_csrf', csrfToken.value)
if (this.sasjsConfig.debug) formData.append('_debug', '131')
if (
this.sasjsConfig.serverType === ServerType.SasViya &&
this.sasjsConfig.contextName
)
formData.append('_contextname', this.sasjsConfig.contextName)
const headers = {
'cache-control': 'no-cache',
@@ -53,9 +63,15 @@ export class FileUploader {
return this.requestClient
.post(uploadUrl, formData, undefined, 'application/json', headers)
.then((res) =>
typeof res.result === 'string' ? JSON.parse(res.result) : res.result
)
.then((res) => {
let result
result =
typeof res.result === 'string' ? JSON.parse(res.result) : res.result
return result
//TODO: append to SASjs requests
})
.catch((err: Error) => {
if (err instanceof LoginRequiredError) {
return Promise.reject(

18
src/SASjs.ts
View File

@@ -24,7 +24,7 @@ const defaultConfig: SASjsConfig = {
serverType: ServerType.SasViya,
debug: false,
contextName: 'SAS Job Execution compute context',
useComputeApi: false,
useComputeApi: null,
allowInsecureRequests: false
}
@@ -544,12 +544,7 @@ export default class SASjs {
public uploadFile(sasJob: string, files: UploadFile[], params: any) {
const fileUploader =
this.fileUploader ||
new FileUploader(
this.sasjsConfig.appLoc,
this.sasjsConfig.serverUrl,
this.jobsPath,
this.requestClient!
)
new FileUploader(this.sasjsConfig, this.jobsPath, this.requestClient!)
return fileUploader.uploadFile(sasJob, files, params)
}
@@ -595,7 +590,11 @@ export default class SASjs {
const validationResult = this.validateInput(data)
if (validationResult.status) {
if (config.serverType === ServerType.SasViya && config.contextName) {
if (
config.serverType !== ServerType.Sas9 &&
config.useComputeApi !== undefined &&
config.useComputeApi !== null
) {
if (config.useComputeApi) {
return await this.computeJobExecutor!.execute(
sasJob,
@@ -935,8 +934,7 @@ export default class SASjs {
}
this.fileUploader = new FileUploader(
this.sasjsConfig.appLoc,
this.sasjsConfig.serverUrl,
this.sasjsConfig,
this.jobsPath,
this.requestClient
)

21
src/job-execution/WebJobExecutor.ts
View File

@@ -39,15 +39,18 @@ export class WebJobExecutor extends BaseJobExecutor {
? config.appLoc.replace(/\/?$/, '/') + sasJob.replace(/^\//, '')
: sasJob
: sasJob
const jobUri =
config.serverType === ServerType.SasViya
? await this.getJobUri(sasJob)
: ''
const apiUrl = `${config.serverUrl}${this.jobsPath}/?${
jobUri.length > 0
? '__program=' + program + '&_job=' + jobUri
: '_program=' + program
}`
let apiUrl = `${config.serverUrl}${this.jobsPath}/?${'_program=' + program}`
if (config.serverType === ServerType.SasViya) {
const jobUri =
config.serverType === ServerType.SasViya
? await this.getJobUri(sasJob)
: ''
apiUrl += jobUri.length > 0 ? '&_job=' + jobUri : ''
apiUrl += config.contextName ? `&_contextname=${config.contextName}` : ''
}
let requestParams = {
...this.getRequestParams(config)

10
src/test/FileUploader.spec.ts
View File

@@ -3,7 +3,7 @@
*/
import { FileUploader } from '../FileUploader'
import { UploadFile } from '../types'
import { SASjsConfig, UploadFile } from '../types'
import { RequestClient } from '../request/RequestClient'
import axios from 'axios'
jest.mock('axios')
@@ -32,9 +32,13 @@ const prepareFilesAndParams = () => {
}
describe('FileUploader', () => {
const config: SASjsConfig = {
...new SASjsConfig(),
appLoc: '/sample/apploc'
}
const fileUploader = new FileUploader(
'/sample/apploc',
'https://sample.server.com',
config,
'/jobs/path',
new RequestClient('https://sample.server.com')
)

12
src/types/SASjsConfig.ts
View File

@@ -40,23 +40,19 @@ export class SASjsConfig {
*/
debug: boolean = true
/**
* The name of the compute context to use when calling the Viya APIs directly.
* The name of the compute context to use when calling the Viya services directly.
* Example value: 'SAS Job Execution compute context'
* If set to missing or empty, and useComputeApi is true, the adapter will use
* the JES APIs. If provided, the Job Code will be executed in pooled
* compute sessions on this named context.
*/
contextName: string = ''
/**
* Set to `false` to use the Job Execution Web Service. To enhance VIYA
* If it's `false` adapter will use the JES API as connection approach. To enhance VIYA
* performance, set to `true` and provide a `contextName` on which to run
* the code. When running on a named context, the code executes under the
* user identity. When running as a Job Execution service, the code runs
* under the identity in the JES context. If no `contextName` is provided,
* and `useComputeApi` is `true`, then the service will run as a Job, except
* under the identity in the JES context. If `useComputeApi` is `null` or `undefined`, the service will run as a Job, except
* triggered using the APIs instead of the Job Execution Web Service broker.
*/
useComputeApi = false
useComputeApi: boolean | null = null
/**
* Defaults to `false`.
* When set to `true`, the adapter will allow requests to SAS servers that use a self-signed SSL certificate.