Compare commits

..
50 Commits
Author SHA1 Message Date
semantic-release-bot 9c3a1086b6 chore(release): 0.39.7 [skip ci]
## [0.39.7](https://github.com/sasjs/server/compare/v0.39.6...v0.39.7) (2026-07-14)

### Bug Fixes

* **jsonwebtoken:** bumped version to avoid vulnerability ([1473925](https://github.com/sasjs/server/commit/1473925896db0e4472c9ef5ba64955527a1be2de))
2026-07-14 13:53:13 +00:00
Yury ShkodaandGitHub 05768890a2 Merge pull request #366 from sasjs/cpr-fix
Fix API scripts
2026-07-14 16:50:34 +03:00
YuryShkoda 386185d1a0 Merge remote-tracking branch 'origin/main' into cpr-fix
# Conflicts:
#	api/package-lock.json
#	api/package.json
#	api/public/swagger.yaml
2026-07-14 16:28:10 +03:00
Yury ShkodaandGitHub 681f0123ad Merge pull request #387 from sasjs/fix/pin-axios-version
chore: pin axios versions
2026-07-14 15:45:40 +03:00
YuryShkoda daa5274fb0 Merge remote-tracking branch 'origin/main' into fix/pin-axios-version 2026-07-14 15:29:26 +03:00
semantic-release-bot 50fa4320cd chore(release): 0.39.6 [skip ci]
## [0.39.6](https://github.com/sasjs/server/compare/v0.39.5...v0.39.6) (2026-07-14)

### Bug Fixes

* **api:** isolate drive.spec.ts's files folder from the real drive ([dec5191](https://github.com/sasjs/server/commit/dec51914911c10b66b43ebe95f77b0af63e2d03e))
* **api:** stop overwriting a failed JS/PY/R session back to completed ([4858245](https://github.com/sasjs/server/commit/4858245372fa19929b6e59c28148f5d004239c9d))
2026-07-14 09:05:49 +00:00
Allan BoweandGitHub 40b9fa8735 Merge pull request #391 from sasjs/issue-390
Issue 390
2026-07-14 10:02:31 +01:00
YuryShkoda fd31fe94ea Merge remote-tracking branch 'origin/main' into issue-390
# Conflicts:
#	api/src/controllers/internal/spec/Execution.spec.ts
2026-07-14 11:58:24 +03:00
YuryShkoda dec5191491 fix(api): isolate drive.spec.ts's files folder from the real drive
drive.spec.ts already isolates itself into a unique, timestamped
tmpFolder for getSasjsRootFolder()/getUploadsFolder(), cleaned up via
afterAll(() => deleteFolder(tmpFolder)). But getFilesFolder() - what
nearly every test in this file actually writes to - resolves through
a separate, unmocked function (getSasjsDriveFolder()/process.driveLoc),
so every run left real files/folders (e.g. 'level1', 'my/path/...')
behind in the shared api/sasjs_root/drive/files, causing later runs
to fail: folder-listing tests saw stale entries, and file-creation
tests got 409 Conflict against files a previous run already created.

Mock getFilesFolder() the same way getUploadsFolder() already is, so
it resolves inside the same isolated tmpFolder and gets cleaned up
by the existing afterAll. Verified by running the suite twice in a
row from a clean slate: both runs pass, and the real sasjs_root/drive
is never touched.
2026-07-14 11:32:01 +03:00
YuryShkoda 4858245372 fix(api): stop overwriting a failed JS/PY/R session back to completed
For JS/PY/R, processProgram sets session.state = failed (with
failureReason) itself when the interpreter process exits non-zero,
without throwing. ExecutionController.executeProgram then
unconditionally set session.state = completed right after
processProgram returned, silently overwriting that - so anything
downstream inspecting session.state (e.g. scheduleSessionDestroy's
expiresAfterMins branch) would see a crashed session mis-reported as
successful.

Guard the assignment so a failed state is never overwritten. No
effect on SAS, which sets state independently via its own spawned
process lifecycle.

Added Execution.spec.ts covering both the failure case (state stays
failed) and the success case (state still becomes completed), to
guard against regressing in either direction.
2026-07-14 11:29:30 +03:00
semantic-release-bot 2fe733d02f chore(release): 0.39.5 [skip ci]
## [0.39.5](https://github.com/sasjs/server/compare/v0.39.4...v0.39.5) (2026-07-14)

### Bug Fixes

* **api:** harden mock SAS executable and timeouts for CI ([67fce47](https://github.com/sasjs/server/commit/67fce475a3c53b3c20919902edb45dce809dffb9))
* **api:** return prompt error response instead of hanging when a SAS session fails ([f84af4a](https://github.com/sasjs/server/commit/f84af4ac0639625fee4508e764f0eed07d7b4e8a))
* fix code.spec.ts failing on a fresh checkout (CI) ([0af6b63](https://github.com/sasjs/server/commit/0af6b63ffa770bfa2b77505d74778be517f8d631))
2026-07-14 06:38:40 +00:00
Allan BoweandGitHub 63086a8a0b Merge pull request #389 from sasjs/issue-388
fix(api): return prompt error response instead of hanging when a SAS …
2026-07-14 07:35:39 +01:00
YuryShkoda 0af6b63ffa fix: fix code.spec.ts failing on a fresh checkout (CI) 2026-07-14 09:20:35 +03:00
YuryShkoda 67fce475a3 fix(api): harden mock SAS executable and timeouts for CI
- wrap the file read/write in a small retry() helper instead of
  letting an uncaught exception (e.g. a rename racing an existsSync
  check) crash the process with a non-zero exit indistinguishable
  from a genuine SAS failure
- bump the internal give-up deadline from 1500ms to 8000ms
- bump the corresponding Jest test timeouts from 15000ms to 30000ms to match
2026-07-14 08:41:35 +03:00
YuryShkoda f84af4ac06 fix(api): return prompt error response instead of hanging when a SAS session fails
The SAS-runtime poll loop in processProgram only checked for
SessionState.completed, so a session that failed (e.g. via %abort;)
left the loop spinning on delay(50) forever - the request never
resolved, even though the SAS log had already been written. Split a single completed-on-either-outcome
flag into separate completed/failed states without updating this loop.

- processProgram now throws when session.state becomes 'failed'
- Execution.ts catches that, reads the complete log (guaranteed
  complete since the session's process has already exited by then),
  and throws a SessionExecutionError carrying it
- stp.ts/code.ts surface { ..., log } in the HTTP error response
- add unit tests for the poll loop and the log-enrichment logic, plus
  a mock SAS executable + end-to-end test exercising the real
  session/process pipeline without needing a SAS install
- add docs/diagrams covering the session lifecycle and the SAS
  execution handshake mechanism, for future context
2026-07-13 13:46:36 +03:00
mulahasanovic 4f75fd290f chore: pin axios versions 2026-04-01 20:55:50 +02:00
semantic-release-bot 8b2a6155a9 chore(release): 0.39.4 [skip ci]
## [0.39.4](https://github.com/sasjs/server/compare/v0.39.3...v0.39.4) (2025-12-21)

### Bug Fixes

* **deps:** bump dependencies and resolve audit issues ([4f78202](https://github.com/sasjs/server/commit/4f782025dbcdfcbae6ca1fabb42ce1bc385e0162))
2025-12-21 01:27:37 +00:00
Trevor MoodyandGitHub a56c0b0340 Merge pull request #386 from sasjs/depBumps_20251221
fix(deps): bump dependencies and resolve audit issues
2025-12-21 01:24:31 +00:00
Trevor Moody 02fe79d4d7 chore(deps): full package-lock rebuild 2025-12-21 01:22:35 +00:00
Trevor Moody 00a107babd chore(deps): rebuilt api/package-json to sync 2025-12-21 01:15:05 +00:00
Trevor Moody 4f782025db fix(deps): bump dependencies and resolve audit issues 2025-12-21 00:27:24 +00:00
semantic-release-bot 8b5abcd661 chore(release): 0.39.3 [skip ci]
## [0.39.3](https://github.com/sasjs/server/compare/v0.39.2...v0.39.3) (2025-11-25)

### Bug Fixes

* (deps) bump @sasjs/core to 4.59.7 ([ab96653](https://github.com/sasjs/server/commit/ab966535642d08d4e8e984007b98c8fdffbe30f7))
* (deps) rerun npm i to sync ([225f381](https://github.com/sasjs/server/commit/225f381bdf8ad5aa2af8d75648df1dd5175e12e0))
2025-11-25 12:39:15 +00:00
Allan BoweandGitHub 48e8cb7b2d Merge pull request #385 from sasjs/bumpCore_20251125
fix: (deps) rerun npm i to sync
2025-11-25 12:36:18 +00:00
Trevor Moody 225f381bdf fix: (deps) rerun npm i to sync 2025-11-25 12:27:14 +00:00
Allan BoweandGitHub 3f49186e3b Merge pull request #384 from sasjs/bumpCore_20251125
fix: (deps) bump @sasjs/core to 4.59.7
2025-11-25 12:24:27 +00:00
Trevor Moody ab96653564 fix: (deps) bump @sasjs/core to 4.59.7 2025-11-25 12:18:50 +00:00
semantic-release-bot 471c28eaa2 chore(release): 0.39.2 [skip ci]
## [0.39.2](https://github.com/sasjs/server/compare/v0.39.1...v0.39.2) (2025-09-25)

### Bug Fixes

* addressing test fail ([e51b204](https://github.com/sasjs/server/commit/e51b20421adc1598ea267c79b1fb4dbc085f97b9))
* packages missmatch ([379ea60](https://github.com/sasjs/server/commit/379ea604bcb5686b5299fae6a32f759c45b275ea))
* type libs ([6d123c3](https://github.com/sasjs/server/commit/6d123c3e23628c1d703eaa13142c77f0da970a55))
* typescript errors ([631e956](https://github.com/sasjs/server/commit/631e95604b64b1a96f2abade659348618f3b00b2))
* typescript errors ([198cd79](https://github.com/sasjs/server/commit/198cd79354254511c21ac1acfbf7b6bcfdab2af7))
2025-09-25 16:57:01 +00:00
Allan BoweandGitHub 584ffe9e0e Merge pull request #383 from sasjs/npm_update_20250919
Npm update 20250919
2025-09-25 17:53:46 +01:00
M e51b20421a fix: addressing test fail 2025-09-25 13:49:32 +02:00
M 631e95604b fix: typescript errors 2025-09-25 13:40:10 +02:00
M 198cd79354 fix: typescript errors 2025-09-25 13:34:55 +02:00
M 379ea604bc fix: packages missmatch 2025-09-25 13:12:23 +02:00
M 9ffa403bcb chore: package-lock 2025-09-25 13:06:06 +02:00
M 6d123c3e23 fix: type libs 2025-09-25 13:03:47 +02:00
M dda1aadc67 chore(git): Merge branch 'main' into npm_update_20250919 2025-09-25 12:48:10 +02:00
M d47cf15cdb ci: ubuntu 22 2025-09-25 12:46:19 +02:00
Trevor Moody d0c7968d66 build: updated package dependencies for /web 2025-09-19 18:24:58 +01:00
Trevor Moody a5c99971cc build: server/api dependency update 2025-09-19 14:06:50 +01:00
semantic-release-bot c422e7f02e chore(release): 0.39.1 [skip ci]
## [0.39.1](https://github.com/sasjs/server/compare/v0.39.0...v0.39.1) (2025-03-13)

### Bug Fixes

* extra bit of sleep for file recognition ([f4768bf](https://github.com/sasjs/server/commit/f4768bffd3dbb2fe243966572ba74002024d96e1)), closes [#381](https://github.com/sasjs/server/issues/381)
2025-03-13 10:59:10 +00:00
Allan BoweandGitHub 02a993611c Merge pull request #382 from sasjs/381-add-slight-delay-to-enable-file-detection
fix: extra bit of sleep for file recognition
2025-03-13 10:56:13 +00:00
sabhas aca2fff4ac chore(workflow): run the build workflow on ubuntu 20.04 2025-03-13 15:50:23 +05:00
sabhas af1a386b13 chore(workflow): install openssl 1.1 in actions 2025-03-13 15:43:20 +05:00
Allan Bowe f5018ce1df chore: prettier fix 2025-03-12 17:33:02 +00:00
Allan Bowe 3529232f1f chore: whitespace removal 2025-03-12 17:29:02 +00:00
Allan Bowe f4768bffd3 fix: extra bit of sleep for file recognition
closes #381
2025-03-12 17:27:57 +00:00
Yury Shkoda 66232aefd2 chore(lint): bumped prettier and fixed lint issues 2023-09-21 17:58:47 +03:00
Yury Shkoda bf35791655 chore(lint): fixed Session.ts 2023-09-21 17:52:28 +03:00
Yury Shkoda 2dc11630e4 chore(api): regenerated swagger.yaml 2023-09-21 17:45:52 +03:00
Yury Shkoda 1473925896 fix(jsonwebtoken): bumped version to avoid vulnerability 2023-09-21 17:45:21 +03:00
Yury Shkoda b472f1bd61 chore(scripts): used cpr package to improve coping 2023-09-21 17:44:31 +03:00
32 changed files with 13254 additions and 24593 deletions
+1
View File
@@ -0,0 +1 @@
* text=auto eol=lf
+3 -3
View File
@@ -5,7 +5,7 @@ on:
jobs: jobs:
lint: lint:
runs-on: ubuntu-latest runs-on: ubuntu-22.04
strategy: strategy:
matrix: matrix:
@@ -28,7 +28,7 @@ jobs:
run: npm run lint-web run: npm run lint-web
build-api: build-api:
runs-on: ubuntu-latest runs-on: ubuntu-22.04
strategy: strategy:
matrix: matrix:
@@ -66,7 +66,7 @@ jobs:
CI: true CI: true
build-web: build-web:
runs-on: ubuntu-latest runs-on: ubuntu-22.04
strategy: strategy:
matrix: matrix:
+1 -1
View File
@@ -7,7 +7,7 @@ on:
jobs: jobs:
release: release:
runs-on: ubuntu-latest runs-on: ubuntu-22.04
strategy: strategy:
matrix: matrix:
+1
View File
@@ -0,0 +1 @@
ignore-scripts=true
+57
View File
@@ -1,3 +1,60 @@
## [0.39.7](https://github.com/sasjs/server/compare/v0.39.6...v0.39.7) (2026-07-14)
### Bug Fixes
* **jsonwebtoken:** bumped version to avoid vulnerability ([1473925](https://github.com/sasjs/server/commit/1473925896db0e4472c9ef5ba64955527a1be2de))
## [0.39.6](https://github.com/sasjs/server/compare/v0.39.5...v0.39.6) (2026-07-14)
### Bug Fixes
* **api:** isolate drive.spec.ts's files folder from the real drive ([dec5191](https://github.com/sasjs/server/commit/dec51914911c10b66b43ebe95f77b0af63e2d03e))
* **api:** stop overwriting a failed JS/PY/R session back to completed ([4858245](https://github.com/sasjs/server/commit/4858245372fa19929b6e59c28148f5d004239c9d))
## [0.39.5](https://github.com/sasjs/server/compare/v0.39.4...v0.39.5) (2026-07-14)
### Bug Fixes
* **api:** harden mock SAS executable and timeouts for CI ([67fce47](https://github.com/sasjs/server/commit/67fce475a3c53b3c20919902edb45dce809dffb9))
* **api:** return prompt error response instead of hanging when a SAS session fails ([f84af4a](https://github.com/sasjs/server/commit/f84af4ac0639625fee4508e764f0eed07d7b4e8a))
* fix code.spec.ts failing on a fresh checkout (CI) ([0af6b63](https://github.com/sasjs/server/commit/0af6b63ffa770bfa2b77505d74778be517f8d631))
## [0.39.4](https://github.com/sasjs/server/compare/v0.39.3...v0.39.4) (2025-12-21)
### Bug Fixes
* **deps:** bump dependencies and resolve audit issues ([4f78202](https://github.com/sasjs/server/commit/4f782025dbcdfcbae6ca1fabb42ce1bc385e0162))
## [0.39.3](https://github.com/sasjs/server/compare/v0.39.2...v0.39.3) (2025-11-25)
### Bug Fixes
* (deps) bump @sasjs/core to 4.59.7 ([ab96653](https://github.com/sasjs/server/commit/ab966535642d08d4e8e984007b98c8fdffbe30f7))
* (deps) rerun npm i to sync ([225f381](https://github.com/sasjs/server/commit/225f381bdf8ad5aa2af8d75648df1dd5175e12e0))
## [0.39.2](https://github.com/sasjs/server/compare/v0.39.1...v0.39.2) (2025-09-25)
### Bug Fixes
* addressing test fail ([e51b204](https://github.com/sasjs/server/commit/e51b20421adc1598ea267c79b1fb4dbc085f97b9))
* packages missmatch ([379ea60](https://github.com/sasjs/server/commit/379ea604bcb5686b5299fae6a32f759c45b275ea))
* type libs ([6d123c3](https://github.com/sasjs/server/commit/6d123c3e23628c1d703eaa13142c77f0da970a55))
* typescript errors ([631e956](https://github.com/sasjs/server/commit/631e95604b64b1a96f2abade659348618f3b00b2))
* typescript errors ([198cd79](https://github.com/sasjs/server/commit/198cd79354254511c21ac1acfbf7b6bcfdab2af7))
## [0.39.1](https://github.com/sasjs/server/compare/v0.39.0...v0.39.1) (2025-03-13)
### Bug Fixes
* extra bit of sleep for file recognition ([f4768bf](https://github.com/sasjs/server/commit/f4768bffd3dbb2fe243966572ba74002024d96e1)), closes [#381](https://github.com/sasjs/server/issues/381)
# [0.39.0](https://github.com/sasjs/server/compare/v0.38.0...v0.39.0) (2024-10-31) # [0.39.0](https://github.com/sasjs/server/compare/v0.38.0...v0.39.0) (2024-10-31)
@@ -0,0 +1,38 @@
# Code execution & session diagrams
Mermaid diagrams describing how `@sasjs/server` executes submitted code
(SAS/JS/PY/R) against pooled "sessions". Written for fast context-loading by
an AI coding agent: each diagram is self-contained, node/edge labels carry
the actual file:line references, and prose is kept to the minimum needed to
disambiguate the diagram.
| File | Covers |
|---|---|
| [session-lifecycle.md](session-lifecycle.md) | `SessionState` state machine; how it differs between the SAS runtime and the JS/PY/R runtimes |
| [sas-execution-handshake.md](sas-execution-handshake.md) | The SYSIN/AUTOEXEC file-swap mechanism a SAS session uses to turn one long-lived `sas` process into a single-use execution slot |
| [request-execution-flow.md](request-execution-flow.md) | End-to-end flowchart from HTTP request to response, covering session-pool acquisition and both runtime branches (SAS vs JS/PY/R) |
## Core concept
A "session" is not a request-scoped object. It is a pooled, reusable
execution slot with a filesystem folder (`session.path`) and a state
(`SessionState`). For the SAS runtime specifically, a session also owns one
real OS process, spawned at session-*creation* time and consumed by exactly
one code submission (see `sas-execution-handshake.md`). For JS/PY/R, a
session is just an ID + folder; the actual interpreter process is spawned
fresh per request inside `processProgram`.
## Key source files
- `api/src/controllers/internal/Session.ts` — session pool + lifecycle
(`SessionController`, `SASSessionController`), `SessionState` transitions.
- `api/src/controllers/internal/processProgram.ts` — writes the submitted
code into the session and drives it to completion/failure, per runtime.
- `api/src/controllers/internal/Execution.ts``ExecutionController`,
the entry point controllers call; acquires a session, calls
`processProgram`, reads back `log.log`/`webout.txt`/headers, builds the
HTTP response (or a `SessionExecutionError` on failure).
- `api/src/controllers/internal/create{SAS,JS,Python,R}Program.ts`
per-runtime code templating (wraps the user's submitted code with
boilerplate: variable injection, `_webout` redirection, etc).
- `api/src/types/Session.ts``SessionState` enum and `Session` interface.
@@ -0,0 +1,64 @@
# Request execution flow (end-to-end)
Full path from an inbound HTTP request to the HTTP response, covering
session-pool acquisition and the two structurally different runtime
branches: SAS (reuses a parked long-lived process) vs JS/PY/R (spawns a
fresh interpreter process per request).
```mermaid
flowchart TD
A["HTTP request<br/>POST /SASjsApi/code/execute<br/>or /SASjsApi/stp/execute"] --> B["Controller<br/>code.ts / stp.ts<br/>try/catch wrapper"]
B --> C["ExecutionController.executeProgram<br/>or .executeFile<br/>Execution.ts:58-91"]
C --> D{"session already provided?<br/>(e.g. file-upload flow)"}
D -- no --> E["getSessionController(runTime).getSession()<br/>Session.ts:59-69"]
D -- yes --> F["use provided session"]
E --> G{"pending session<br/>available in pool?"}
G -- yes --> H["reuse it"]
G -- no --> I["createSession()<br/>SAS: spawns a real sas process, see<br/>sas-execution-handshake.md<br/>JS/PY/R: cheap - folder + id only"]
H --> J["pre-warm: if pool has fewer<br/>than 3 pending, fire off more<br/>createSession() calls (not awaited)<br/>Session.ts:66"]
I --> J
F --> K
J --> K["session.state = running<br/>Execution.ts:96"]
K --> L["write webout.txt (empty),<br/>reqHeaders.txt<br/>Execution.ts:103-107"]
L --> M["processProgram(...)<br/>processProgram.ts:16-27"]
M --> N{"runTime?"}
N -- SAS --> O["createSASProgram():<br/>wrap user code with _webout<br/>filename, macro vars, autoexec"]
O --> P["write code.sas via .bkp + rename<br/>processProgram.ts:48-49"]
P --> Q["poll: while session.state !== completed<br/>processProgram.ts:52-58"]
Q -- "state becomes completed" --> R["resolve"]
Q -- "state becomes failed" --> S["throw Error(session.failureReason)"]
N -- "JS / PY / R" --> T["createJSProgram / createPythonProgram /<br/>createRProgram: wrap user code similarly"]
T --> U["write code file; spawn interpreter<br/>fresh via execFile; pipe stdout/stderr<br/>into a log.log write stream<br/>processProgram.ts:108-134"]
U -- "exit 0" --> R
U -- "exit non-zero" --> S
R --> V["read log.log, webout.txt,<br/>stpsrv_header.txt<br/>Execution.ts:128-131"]
V --> W["build httpHeaders + result<br/>return ExecuteReturnRaw<br/>Execution.ts:128-151"]
W --> X["Controller: res.send(result)<br/>HTTP 200"]
S --> Y["catch in Execution.ts:109-126:<br/>read whatever log exists,<br/>throw SessionExecutionError(message, log)"]
Y --> Z["Controller catch block<br/>rethrow { code: 400, status: 'failure',<br/>message, error, log }"]
Z --> AA["res.status(err.code).send(err)<br/>HTTP 400 with complete log"]
```
## Notes
- **Session pool is per-runtime.** SAS sessions and JS/PY/R sessions live in
separate controllers/pools (`process.sasSessionController` vs
`process.sessionController`, `Session.ts:239-253`) - a pending SAS session
is never handed out for a JS request or vice versa.
- **The SAS branch and the JS/PY/R branch converge** on the same
success/failure signal shape: `session.state` becomes `completed` or
`failed` either way, and both paths flow through the same log/webout
reading logic in `Execution.ts` afterward. The mechanics of *how* that
state gets set differ substantially (see `session-lifecycle.md`).
- **`includePrintOutput`** (SAS only) additionally appends `output.lst`
content to the result when debug mode is on - omitted above for brevity;
see `Execution.ts:135-142`.
- **`triggerProgram`/`triggerCode`** (fire-and-forget variants, not shown)
call the same `ExecutionController` methods without awaiting them and
immediately return `{ sessionId }`; the client polls
`GET /SASjsApi/session/{sessionId}/state` separately.
@@ -0,0 +1,81 @@
# SAS session execution handshake
How one long-lived `sas` process is turned into a single-use execution slot
via a SYSIN file-swap trick. This is the mechanism specific to
`RunTimeType.SAS`; JS/PY/R sessions spawn a fresh interpreter process per
request instead (see `request-execution-flow.md`).
```mermaid
sequenceDiagram
participant Client as HTTP client
participant Ctrl as Controller<br/>(code.ts / stp.ts)
participant Exec as ExecutionController<br/>(Execution.ts)
participant Pool as SASSessionController<br/>(Session.ts)
participant FS as Filesystem<br/>(session.path)
participant SAS as spawned "sas" process
Note over Pool,SAS: SESSION CREATION - happens ahead of any request,<br/>pool pre-warms up to 3 sessions (Session.ts:59-69)
Pool->>FS: createFile(code.sas, "") - empty dummy SYSIN (Session.ts:117-118)
Pool->>FS: readFile(sysInitCompiledPath) - compiled system-init<br/>macros, produced by `npm run compileSysInit` (Session.ts:105)
Note over Pool: throws if this file doesn't exist yet -<br/>session creation never reaches execFile below
Pool->>FS: createFile(autoexec.sas, autoExecContent) (Session.ts:108-114)
Pool->>SAS: execFile(sasLoc, -SYSIN code.sas -AUTOEXEC autoexec.sas -LOG log.log ...) (Session.ts:127-147)
activate SAS
Note right of SAS: process stays active (this box)<br/>from spawn until it finally exits below
SAS->>FS: autoexec step 1: delete code.sas, the dummy SYSIN (Session.ts:257-261)
Pool->>FS: waitForSession() polls fileExists(code.sas) (Session.ts:175-192)
FS-->>Pool: code.sas no longer exists
Pool->>Pool: session.state = pending (Session.ts:190)
SAS->>SAS: autoexec step 2: busy-wait for code.sas to reappear,<br/>up to 15 minutes (Session.ts:263-274)
Note over SAS: process is now idle, parked mid-autoexec,<br/>waiting on the filesystem.<br/>Counts as "pending" in the pool.
Note over Client,SAS: REQUEST ARRIVES - reuses the parked process above
Client->>Ctrl: POST .../execute { code, runTime: sas }
Ctrl->>Exec: executeProgram({ program, runTime: SAS, ... })
Exec->>Pool: getSession() returns the pending session (Session.ts:59-69)
Exec->>Exec: session.state = running (Execution.ts:96)
Exec->>FS: createFile(webout.txt, "") and createFile(reqHeaders.txt, ...) (Execution.ts:103-107)
Exec->>Pool: processProgram(program, session, ...) (Execution.ts:110-121)
Pool->>FS: createSASProgram() wraps user code with macro vars,<br/>_webout filename, autoexec injection (createSASProgram.ts)
Pool->>FS: write code.sas.bkp, then rename to code.sas (processProgram.ts:48-49)
Note over FS: write-then-rename, not a direct write,<br/>so SAS never reads a partial file
FS-->>SAS: code.sas exists again
SAS->>SAS: autoexec step 2 loop exits, sleeps 0.01s, autoexec ends (Session.ts:267-271)
Note over SAS: -SYSIN has pointed at code.sas from the start,<br/>so SAS now executes ITS CONTENT as the main job
SAS->>FS: writes log.log, output.lst, webout.txt while running
Pool->>Pool: processProgram poll loop:<br/>while session.state !== completed (processProgram.ts:52-58)
alt program reaches EOF normally
SAS->>SAS: exits with code 0
SAS-->>Pool: execFilePromise resolves - .then() (Session.ts:148-152)
Pool->>Pool: session.state = completed
Pool-->>Exec: processProgram() resolves
else program aborts (fatal error, license failure, hard STOP)
SAS->>SAS: exits with non-zero code
SAS-->>Pool: execFilePromise rejects - .catch() (Session.ts:153-163)
Pool->>Pool: session.state = failed<br/>session.failureReason = err.toString()
Pool-->>Exec: processProgram() throws (processProgram.ts:53-55)
end
deactivate SAS
Exec->>FS: read log.log, webout.txt, stpsrv_header.txt (Execution.ts:123, 128-131)
Exec-->>Ctrl: { httpHeaders, result }<br/>or throws SessionExecutionError{ message, log } (Execution.ts:109-126)
Ctrl-->>Client: 200 + result<br/>or 400 { status, message, error, log }
```
## Why this design
- SAS has meaningful startup cost (loading the engine, running system init
macros). Spawning a fresh `sas` process per request would pay that cost
every time. Instead, a process is spawned once and parked, ready to run
exactly one job the moment code shows up at a path it's already watching.
- The pool (`SessionController.getSession`, `Session.ts:59-69`) keeps up to
3 such parked processes ready, so most requests get an instant handoff
instead of waiting through the ~15-minute autoexec spin-wait or a cold
start.
- A session is single-use: once its process consumes the real SYSIN content
and exits (success or failure), that process is gone. The session object
itself is deleted later by `scheduleSessionDestroy` (`Session.ts:204-236`),
not reused for a second job.
+70
View File
@@ -0,0 +1,70 @@
# Session lifecycle (`SessionState`)
Enum defined in `api/src/types/Session.ts`. A `Session` is `{ id, state,
path, creationTimeStamp, deathTimeStamp, expiresAfterMins?, failureReason? }`.
Sessions live in an in-memory array on a per-runtime singleton controller
(`process.sasSessionController` for SAS, `process.sessionController` shared
by JS/PY/R — see `getSessionController()` in `Session.ts:239-253`).
```mermaid
stateDiagram-v2
[*] --> initialising: SAS runtime<br/>SASSessionController.createSession()<br/>Session.ts:77-95
[*] --> pending: JS/PY/R runtime, no process yet<br/>SessionController.createSession()<br/>Session.ts:30-57
initialising --> pending: dummy SYSIN file deleted<br/>by SAS's autoexec<br/>waitForSession(), Session.ts:175-192
initialising --> failed: spawned SAS process exits<br/>before handshake completes<br/>Session.ts:153-163, 184-188
pending --> running: session picked for a request<br/>executeProgram(), Execution.ts:94-96
running --> completed: process exits 0<br/>SAS: Session.ts:148-152 (original process)<br/>JS/PY/R: processProgram.ts:116-120 (fresh process)
running --> failed: process exits non-zero<br/>SAS: Session.ts:153-163<br/>JS/PY/R: processProgram.ts:121-131<br/>failureReason = err.toString()
completed --> [*]: deleteSession()<br/>scheduleSessionDestroy(), Session.ts:194-236
failed --> [*]: deleteSession()<br/>scheduleSessionDestroy()
note right of initialising
SAS only. The real sas
executable is already
running, spin-waiting
inside its AUTOEXEC for
real code to arrive.
See sas-execution-handshake.md
end note
note right of pending
Session sits in the pool,
reusable. Pre-warmed up to
3 pending sessions per
runtime (getSession(),
Session.ts:59-69)
end note
note right of running
SAS: no new process spawned
here, request just feeds
code to the already-running
process.
JS/PY/R: a brand new
interpreter process is
spawned right now.
end note
```
## Consumers of `state`
| Reader | Location | Watches for |
|---|---|---|
| `waitForSession` | `Session.ts:175-192` | `failed` (breaks early) or the dummy SYSIN file disappearing (implies session survived init) |
| `processProgram` (SAS branch poll loop) | `processProgram.ts:52-58` | `completed` (success exit) or `failed` (throws, carrying `session.failureReason`) |
| `scheduleSessionDestroy` | `Session.ts:204-236` | `running` (extends death timer instead of destroying) |
## Asymmetry between runtimes
- **SAS**: one OS process per session, spawned once at `createSession()`
time and reused for exactly one job (see `sas-execution-handshake.md`).
`running`/`completed`/`failed` are all driven by that *same* process's
eventual exit.
- **JS/PY/R**: a session is cheap (folder + id, no process). The interpreter
process is spawned fresh, per request, inside `processProgram.ts:115` and
its exit drives `completed`/`failed` directly in the same function - there
is no separate poll loop for these runtimes.
+6115 -14453
View File
File diff suppressed because it is too large Load Diff
+24 -25
View File
@@ -6,11 +6,9 @@
"scripts": { "scripts": {
"initial": "npm run swagger && npm run compileSysInit && npm run copySASjsCore && npm run downloadMacros", "initial": "npm run swagger && npm run compileSysInit && npm run copySASjsCore && npm run downloadMacros",
"prestart": "npm run initial", "prestart": "npm run initial",
"prebuild": "npm run initial",
"start": "NODE_ENV=development nodemon ./src/server.ts", "start": "NODE_ENV=development nodemon ./src/server.ts",
"start:prod": "node ./build/src/server.js", "start:prod": "node ./build/src/server.js",
"build": "rimraf build && tsc", "build": "npm run initial && rimraf build && tsc && npm run copy:files",
"postbuild": "npm run copy:files",
"swagger": "tsoa spec", "swagger": "tsoa spec",
"prepare": "[ -d .git ] && git config core.hooksPath ./.git-hooks || true", "prepare": "[ -d .git ] && git config core.hooksPath ./.git-hooks || true",
"test": "mkdir -p tmp && mkdir -p ../web/build && jest --silent --coverage", "test": "mkdir -p tmp && mkdir -p ../web/build && jest --silent --coverage",
@@ -18,10 +16,10 @@
"lint": "npx prettier --check \"src/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\"", "lint": "npx prettier --check \"src/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\"",
"exe": "npm run build && pkg .", "exe": "npm run build && pkg .",
"copy:files": "npm run public:copy && npm run sasjsbuild:copy && npm run sas:copy && npm run web:copy", "copy:files": "npm run public:copy && npm run sasjsbuild:copy && npm run sas:copy && npm run web:copy",
"public:copy": "cp -r ./public/ ./build/public/", "public:copy": "cpr ./public/ ./build/public/",
"sasjsbuild:copy": "cp -r ./sasjsbuild/ ./build/sasjsbuild/", "sasjsbuild:copy": "cpr ./sasjsbuild/ ./build/sasjsbuild/",
"sas:copy": "cp -r ./sas/ ./build/sas/", "sas:copy": "cpr ./sas/ ./build/sas/",
"web:copy": "rimraf web && mkdir web && cp -r ../web/build/ ./web/build/", "web:copy": "rimraf web && mkdir web && cpr ../web/build/ ./web/build/",
"compileSysInit": "ts-node ./scripts/compileSysInit.ts", "compileSysInit": "ts-node ./scripts/compileSysInit.ts",
"copySASjsCore": "ts-node ./scripts/copySASjsCore.ts", "copySASjsCore": "ts-node ./scripts/copySASjsCore.ts",
"downloadMacros": "ts-node ./scripts/downloadMacros.ts" "downloadMacros": "ts-node ./scripts/downloadMacros.ts"
@@ -48,25 +46,25 @@
}, },
"author": "4GL Ltd", "author": "4GL Ltd",
"dependencies": { "dependencies": {
"@sasjs/core": "^4.40.1", "@sasjs/core": "^4.59.10",
"@sasjs/utils": "3.2.0", "@sasjs/utils": "^3.5.6",
"bcryptjs": "^2.4.3", "bcryptjs": "^2.4.3",
"connect-mongo": "^4.6.0", "connect-mongo": "^5.1.0",
"cookie-parser": "^1.4.6", "cookie-parser": "^1.4.7",
"cors": "^2.8.5", "cors": "^2.8.5",
"express": "^4.17.1", "express": "^4.21.2",
"express-session": "^1.17.2", "express-session": "^1.18.2",
"helmet": "^5.0.2", "helmet": "^5.0.2",
"joi": "^17.4.2", "joi": "^17.4.2",
"jsonwebtoken": "^8.5.1", "jsonwebtoken": "^9.0.3",
"ldapjs": "2.3.3", "ldapjs": "2.3.3",
"mongoose": "^6.0.12", "mongoose": "^6.13.8",
"morgan": "^1.10.0", "morgan": "^1.10.1",
"multer": "^1.4.5-lts.1", "multer": "^1.4.5-lts.1",
"rate-limiter-flexible": "2.4.1", "rate-limiter-flexible": "2.4.1",
"rotating-file-stream": "^3.0.4", "rotating-file-stream": "^3.0.4",
"swagger-ui-express": "4.3.0", "swagger-ui-express": "4.3.0",
"unzipper": "^0.10.11", "unzipper": "^0.12.3",
"url": "^0.10.3" "url": "^0.10.3"
}, },
"devDependencies": { "devDependencies": {
@@ -76,32 +74,33 @@
"@types/cors": "^2.8.12", "@types/cors": "^2.8.12",
"@types/express": "^4.17.12", "@types/express": "^4.17.12",
"@types/express-session": "^1.17.4", "@types/express-session": "^1.17.4",
"@types/jest": "^26.0.24", "@types/jest": "^29.5.0",
"@types/jsonwebtoken": "^8.5.5", "@types/jsonwebtoken": "^8.5.5",
"@types/ldapjs": "^2.2.4", "@types/ldapjs": "^2.2.4",
"@types/morgan": "^1.9.3", "@types/morgan": "^1.9.3",
"@types/multer": "^1.4.7", "@types/multer": "^1.4.7",
"@types/node": "^15.12.2", "@types/node": "^20.0.0",
"@types/supertest": "^2.0.11", "@types/supertest": "^2.0.11",
"@types/swagger-ui-express": "^4.1.3", "@types/swagger-ui-express": "^4.1.3",
"@types/unzipper": "^0.10.5", "@types/unzipper": "^0.10.5",
"adm-zip": "^0.5.9", "adm-zip": "^0.5.9",
"axios": "0.27.2", "axios": "1.12.2",
"cpr": "^3.0.1",
"csrf": "^3.1.0", "csrf": "^3.1.0",
"dotenv": "^16.0.1", "dotenv": "^16.0.1",
"http-headers-validation": "^0.0.1", "http-headers-validation": "^0.0.1",
"jest": "^27.0.6", "jest": "^29.7.0",
"mongodb-memory-server": "8.11.4", "mongodb-memory-server": "8.11.4",
"nodejs-file-downloader": "4.10.2", "nodejs-file-downloader": "4.10.2",
"nodemon": "^2.0.7", "nodemon": "^3.0.0",
"pkg": "5.6.0", "pkg": "5.6.0",
"prettier": "^2.3.1", "prettier": "^3.0.3",
"rimraf": "^3.0.2", "rimraf": "^3.0.2",
"supertest": "^6.1.3", "supertest": "^6.1.3",
"ts-jest": "^27.0.3", "ts-jest": "^29.1.0",
"ts-node": "^10.0.0", "ts-node": "^10.0.0",
"tsoa": "3.14.1", "tsoa": "3.14.1",
"typescript": "^4.3.2" "typescript": "^5.0.0"
}, },
"nodemonConfig": { "nodemonConfig": {
"ignore": [ "ignore": [
+3 -2
View File
@@ -234,9 +234,10 @@ const verifyAuthCode = async (
jwt.verify(code, process.secrets.AUTH_CODE_SECRET, (err, data) => { jwt.verify(code, process.secrets.AUTH_CODE_SECRET, (err, data) => {
if (err) return resolve(undefined) if (err) return resolve(undefined)
const payload = data as InfoJWT
const clientInfo: InfoJWT = { const clientInfo: InfoJWT = {
clientId: data?.clientId, clientId: payload?.clientId,
userId: data?.userId userId: payload?.userId
} }
if (clientInfo.clientId === clientId) { if (clientInfo.clientId === clientId) {
return resolve(clientInfo) return resolve(clientInfo)
+2 -1
View File
@@ -114,7 +114,8 @@ const executeCode = async (
code: 400, code: 400,
status: 'failure', status: 'failure',
message: 'Job execution failed.', message: 'Job execution failed.',
error: typeof err === 'object' ? err.toString() : err error: typeof err === 'object' ? err.toString() : err,
log: err?.log
} }
} }
} }
+33
View File
@@ -15,6 +15,24 @@ export interface ExecutionVars {
[key: string]: string | number | undefined [key: string]: string | number | undefined
} }
// Thrown when the session itself fails (e.g. SAS exits abnormally via
// %abort;). Carries the complete log - by the time this is thrown, the
// session's process has already exited, so the log file it wrote is final,
// not a partial/truncated snapshot.
export class SessionExecutionError extends Error {
constructor(
message: string,
public log?: string
) {
super(message)
// required for `instanceof` to work when compiling to ES5, since the
// default __extends helper does not preserve the prototype chain for
// classes extending built-ins like Error
Object.setPrototypeOf(this, SessionExecutionError.prototype)
}
}
export interface ExecuteReturnRaw { export interface ExecuteReturnRaw {
httpHeaders: HTTPHeaders httpHeaders: HTTPHeaders
result: string | Buffer result: string | Buffer
@@ -88,6 +106,7 @@ export class ExecutionController {
preProgramVariables?.httpHeaders.join('\n') ?? '' preProgramVariables?.httpHeaders.join('\n') ?? ''
) )
try {
await processProgram( await processProgram(
program, program,
preProgramVariables, preProgramVariables,
@@ -100,6 +119,11 @@ export class ExecutionController {
logPath, logPath,
otherArgs otherArgs
) )
} catch (err: any) {
const log = (await fileExists(logPath)) ? await readFile(logPath) : ''
throw new SessionExecutionError(err.message, log)
}
const log = (await fileExists(logPath)) ? await readFile(logPath) : '' const log = (await fileExists(logPath)) ? await readFile(logPath) : ''
const headersContent = (await fileExists(headersPath)) const headersContent = (await fileExists(headersPath))
@@ -120,7 +144,16 @@ export class ExecutionController {
: '' : ''
// it should be deleted by scheduleSessionDestroy // it should be deleted by scheduleSessionDestroy
//
// Guarded: for JS/PY/R, processProgram sets state to `failed` itself
// (without throwing) when the interpreter process exits non-zero - if
// we unconditionally set `completed` here we'd silently overwrite that,
// and anything downstream inspecting session.state (e.g.
// scheduleSessionDestroy's expiresAfterMins branch in Session.ts) would
// see a crashed session mis-reported as successful.
if ((session.state as SessionState) !== SessionState.failed) {
session.state = SessionState.completed session.state = SessionState.completed
}
const resultParts = [] const resultParts = []
+10 -3
View File
@@ -260,9 +260,16 @@ data _null_;
rc=filename(fname,getoption('SYSIN') ); rc=filename(fname,getoption('SYSIN') );
if rc = 0 and fexist(fname) then rc=fdelete(fname); if rc = 0 and fexist(fname) then rc=fdelete(fname);
rc=filename(fname); rc=filename(fname);
/* now wait for the real SYSIN */ /* now wait for the real SYSIN (location of code.sas) */
slept=0; slept=0;fname='';
do until ( fileexist(getoption('SYSIN')) or slept>(60*15) ); do until (slept>(60*15));
rc=filename(fname,getoption('SYSIN'));
if rc = 0 and fexist(fname) then do;
putlog fname=;
rc=filename(fname);
rc=sleep(0.01,1); /* wait just a little more */
stop;
end;
slept=slept+sleep(0.01,1); slept=slept+sleep(0.01,1);
end; end;
stop; stop;
@@ -50,6 +50,10 @@ export const processProgram = async (
// we now need to poll the session status // we now need to poll the session status
while (session.state !== SessionState.completed) { while (session.state !== SessionState.completed) {
if (session.state === SessionState.failed) {
throw new Error(session.failureReason || 'SAS session failed')
}
await delay(50) await delay(50)
} }
} else { } else {
@@ -0,0 +1,117 @@
import path from 'path'
import os from 'os'
import { createFile, deleteFolder, generateTimestamp } from '@sasjs/utils'
import * as ProcessProgramModule from '../processProgram'
import { ExecutionController, SessionExecutionError } from '../Execution'
import { Session, SessionState, PreProgramVars } from '../../../types'
import { RunTimeType } from '../../../utils'
const preProgramVariables: PreProgramVars = {
username: 'testUser',
userId: 1,
displayName: 'Test User',
serverUrl: 'http://localhost:5000',
httpHeaders: []
}
describe('ExecutionController.executeProgram', () => {
let session: Session
beforeEach(() => {
const sessionId = `test-session-${generateTimestamp()}`
session = {
id: sessionId,
state: SessionState.pending,
creationTimeStamp: '0',
deathTimeStamp: '0',
path: path.join(os.tmpdir(), sessionId)
}
})
afterEach(async () => {
jest.restoreAllMocks()
await deleteFolder(session.path)
})
// Regression coverage: for JS/PY/R, processProgram sets session.state to
// `failed` itself (without throwing) on a non-zero interpreter exit.
// ExecutionController.executeProgram used to unconditionally overwrite
// that back to `completed` right after processProgram returned, silently
// losing the failure for anything downstream that inspects session.state.
describe('JS/PY/R failure path', () => {
it('does not overwrite a failed session state back to completed', async () => {
// mirrors processProgram's real JS/PY/R branch on a non-zero exit:
// it sets state/failureReason itself and resolves - it does not throw
jest
.spyOn(ProcessProgramModule, 'processProgram')
.mockImplementation(async () => {
session.state = SessionState.failed
session.failureReason = 'Error: process exited with code 1'
})
const controller = new ExecutionController()
await controller.executeProgram({
program: 'throw new Error("boom")',
preProgramVariables,
vars: {},
session,
runTime: RunTimeType.JS
})
expect(session.state).toBe(SessionState.failed)
})
it('still marks a genuinely successful session as completed', async () => {
jest
.spyOn(ProcessProgramModule, 'processProgram')
.mockImplementation(async () => {
session.state = SessionState.completed
})
const controller = new ExecutionController()
await controller.executeProgram({
program: 'console.log("hello")',
preProgramVariables,
vars: {},
session,
runTime: RunTimeType.JS
})
expect(session.state).toBe(SessionState.completed)
})
})
describe('SAS failure path', () => {
it('throws a SessionExecutionError carrying the complete log when the session fails', async () => {
const logPath = path.join(session.path, 'log.log')
const logContent =
'NOTE: SAS session\nERROR: SAS session terminated. See log for details.\n'
await createFile(logPath, logContent)
jest
.spyOn(ProcessProgramModule, 'processProgram')
.mockImplementation(async () => {
throw new Error('ERROR: SAS session terminated. See log for details.')
})
const controller = new ExecutionController()
const resultPromise = controller.executeProgram({
program: '%abort;',
preProgramVariables,
vars: {},
session,
runTime: RunTimeType.SAS
})
await expect(resultPromise).rejects.toBeInstanceOf(SessionExecutionError)
await expect(resultPromise).rejects.toMatchObject({
log: logContent,
message: expect.stringContaining('SAS session terminated')
})
})
})
})
@@ -0,0 +1,109 @@
import path from 'path'
import os from 'os'
import { createFile, deleteFolder, generateTimestamp } from '@sasjs/utils'
import { processProgram } from '../processProgram'
import { Session, SessionState, PreProgramVars } from '../../../types'
import {
RunTimeType,
getSessionsFolder,
generateUniqueFileName
} from '../../../utils'
const preProgramVariables: PreProgramVars = {
username: 'testUser',
userId: 1,
displayName: 'Test User',
serverUrl: 'http://localhost:5000',
httpHeaders: []
}
const makeSession = (): Session => {
const sessionId = generateUniqueFileName(generateTimestamp())
const sessionFolder = path.join(getSessionsFolder(), sessionId)
const creationTimeStamp = sessionId.split('-').pop() as string
const deathTimeStamp = (
parseInt(creationTimeStamp) +
15 * 60 * 1000 -
1000
).toString()
return {
id: sessionId,
state: SessionState.running,
creationTimeStamp,
deathTimeStamp,
path: sessionFolder
}
}
describe('processProgram (SAS runtime)', () => {
let session: Session
let logPath: string
let weboutPath: string
let headersPath: string
let tokenFile: string
beforeAll(() => {
const root = path.join(
os.tmpdir(),
`sasjs-processProgram-spec-${generateTimestamp()}`
)
process.sasjsRoot = root
process.driveLoc = path.join(root, 'drive')
})
beforeEach(async () => {
session = makeSession()
logPath = path.join(session.path, 'log.log')
weboutPath = path.join(session.path, 'webout.txt')
headersPath = path.join(session.path, 'stpsrv_header.txt')
tokenFile = path.join(session.path, 'reqHeaders.txt')
await createFile(weboutPath, '')
})
afterEach(async () => {
await deleteFolder(session.path)
})
it('rejects instead of hanging when the session fails (e.g. %abort;)', async () => {
setTimeout(() => {
session.state = SessionState.failed
session.failureReason =
'ERROR: SAS session terminated. See log for details.'
}, 100)
await expect(
processProgram(
'%abort;',
preProgramVariables,
{},
session,
weboutPath,
headersPath,
tokenFile,
RunTimeType.SAS,
logPath
)
).rejects.toThrow(/SAS session terminated/)
}, 3000)
it('resolves without throwing when the session completes normally', async () => {
setTimeout(() => {
session.state = SessionState.completed
}, 100)
await expect(
processProgram(
'%put hello world;',
preProgramVariables,
{},
session,
weboutPath,
headersPath,
tokenFile,
RunTimeType.SAS,
logPath
)
).resolves.toBeUndefined()
}, 3000)
})
+2 -1
View File
@@ -166,7 +166,8 @@ const execute = async (
code: 400, code: 400,
status: 'failure', status: 'failure',
message: 'Job execution failed.', message: 'Job execution failed.',
error: typeof err === 'object' ? err.toString() : err error: typeof err === 'object' ? err.toString() : err,
log: err?.log
} }
} }
} }
+5 -2
View File
@@ -106,7 +106,10 @@ const login = async (
const rateLimiter = RateLimiter.getInstance() const rateLimiter = RateLimiter.getInstance()
if (!validPass) { if (!validPass) {
const retrySecs = await rateLimiter.consume(req.ip, user?.username) const retrySecs = await rateLimiter.consume(
req.ip || 'unknown',
user?.username
)
if (retrySecs > 0) throw errors.tooManyRequests(retrySecs) if (retrySecs > 0) throw errors.tooManyRequests(retrySecs)
} }
@@ -114,7 +117,7 @@ const login = async (
if (!validPass) throw errors.invalidPassword if (!validPass) throw errors.invalidPassword
// Reset on successful authorization // Reset on successful authorization
rateLimiter.resetOnSuccess(req.ip, user.username) rateLimiter.resetOnSuccess(req.ip || 'unknown', user.username)
req.session.loggedIn = true req.session.loggedIn = true
req.session.user = { req.session.user = {
+3 -3
View File
@@ -37,10 +37,10 @@ export const authenticateAccessToken: RequestHandler = async (
if (user.isActive) { if (user.isActive) {
req.user = user req.user = user
return csrfProtection(req, res, nextFunction) return csrfProtection(req, res, nextFunction)
} else return res.sendStatus(401) } else return res.status(401).send('Unauthorized')
} }
} }
return res.sendStatus(401) return res.status(401).send('Unauthorized')
} }
await authenticateToken( await authenticateToken(
@@ -118,6 +118,6 @@ const authenticateToken = async (
return next() return next()
} }
res.sendStatus(401) res.status(401).send('Unauthorized')
} }
} }
+1 -1
View File
@@ -3,7 +3,7 @@ import { convertSecondsToHms } from '@sasjs/utils'
import { RateLimiter } from '../utils' import { RateLimiter } from '../utils'
export const bruteForceProtection: RequestHandler = async (req, res, next) => { export const bruteForceProtection: RequestHandler = async (req, res, next) => {
const ip = req.ip const ip = req.ip || 'unknown'
const username = req.body.username const username = req.body.username
const rateLimiter = RateLimiter.getInstance() const rateLimiter = RateLimiter.getInstance()
+1 -1
View File
@@ -76,7 +76,7 @@ groupSchema.post('save', function (group: IGroup, next: Function) {
}) })
// pre remove hook to remove all references of group from users // pre remove hook to remove all references of group from users
groupSchema.pre('remove', async function () { groupSchema.pre('remove', async function (this: IGroupDocument) {
const userIds = this.users const userIds = this.users
await Promise.all( await Promise.all(
userIds.map(async (userId) => { userIds.map(async (userId) => {
+132
View File
@@ -0,0 +1,132 @@
import path from 'path'
import { createFile, fileExists } from '@sasjs/utils'
import { Express } from 'express'
import mongoose, { Mongoose } from 'mongoose'
import { MongoMemoryServer } from 'mongodb-memory-server'
import request from 'supertest'
import {
UserController,
PermissionController,
PermissionType,
PermissionSettingForRoute,
PrincipalType
} from '../../../controllers/'
import {
generateAccessToken,
saveTokensInDB,
RunTimeType,
sysInitCompiledPath
} from '../../../utils'
// Real, unmocked end-to-end test of the SAS execution pipeline (session
// spawn -> autoexec handshake -> exit code -> HTTP response), using a fake
// "SAS executable" (mockSas.js) in place of a real SAS install. This is the
// regression test for issue #388: SAS code that aborts (%abort;) used to
// hang the request forever instead of returning an error response.
const mockSasPath = path.join(__dirname, 'files', 'mockSas.js')
const clientId = 'codeSpecClientID'
const user = {
displayName: 'Code Spec User',
username: 'codeSpecUsername',
password: '87654321',
isAdmin: false,
isActive: true
}
let app: Express
let accessToken: string
describe('code', () => {
let con: Mongoose
let mongoServer: MongoMemoryServer
let userController: UserController
let permissionController: PermissionController
beforeAll(async () => {
// SASSessionController.createSession() unconditionally reads this file
// (Session.ts:105) before it ever spawns the SAS process. It's normally
// produced by `npm run compileSysInit` (part of the `initial`/`build`
// scripts), which `npm test` does not run - so on a fresh checkout
// (e.g. CI, where "Run Unit Tests" happens before "Build Package") it
// doesn't exist yet. Content is irrelevant here since mockSas.js never
// interprets it; it just needs to exist and be readable.
if (!(await fileExists(sysInitCompiledPath))) {
await createFile(sysInitCompiledPath, '')
}
mongoServer = await MongoMemoryServer.create()
process.env.DB_CONNECT = mongoServer.getUri()
process.env.MODE = 'server'
process.env.RUN_TIMES = 'sas'
process.env.SAS_PATH = mockSasPath
const appPromise = (await import('../../../app')).default
app = await appPromise
con = await mongoose.connect(mongoServer.getUri())
userController = new UserController()
permissionController = new PermissionController()
const dbUser = await userController.createUser(user)
accessToken = await generateAndSaveToken(dbUser.id)
await permissionController.createPermission({
path: '/SASjsApi/code/execute',
type: PermissionType.route,
principalType: PrincipalType.user,
principalId: dbUser.id,
setting: PermissionSettingForRoute.grant
})
process.runTimes = [RunTimeType.SAS]
process.sasLoc = mockSasPath
}, 30000)
afterAll(async () => {
await con.connection.dropDatabase()
await con.connection.close()
await mongoServer.stop()
})
describe('execute', () => {
it('returns 200 with the mock log when the SAS program completes normally', async () => {
const response = await request(app)
.post('/SASjsApi/code/execute')
.auth(accessToken, { type: 'bearer' })
.send({ code: '%put hello world;', runTime: 'sas' })
.expect(200)
expect(response.text).toEqual(
expect.stringContaining('mock SAS execution')
)
}, 30000)
it('returns a prompt 400 (not a hang) with the complete log when the SAS session fails (%abort;)', async () => {
const response = await request(app)
.post('/SASjsApi/code/execute')
.auth(accessToken, { type: 'bearer' })
.send({ code: '%abort;', runTime: 'sas' })
.expect(400)
expect(response.body).toMatchObject({
status: 'failure',
message: 'Job execution failed.'
})
expect(response.body.log).toEqual(
expect.stringContaining('mock SAS execution')
)
}, 30000)
})
})
const generateAndSaveToken = async (userId: number) => {
const accessToken = generateAccessToken({
clientId,
userId
})
await saveTokensInDB(userId, clientId, accessToken, 'refreshToken')
return accessToken
}
+9
View File
@@ -28,6 +28,15 @@ jest
.spyOn(fileUtilModules, 'getUploadsFolder') .spyOn(fileUtilModules, 'getUploadsFolder')
.mockImplementation(() => path.join(tmpFolder, 'uploads')) .mockImplementation(() => path.join(tmpFolder, 'uploads'))
// getFilesFolder() resolves via getSasjsDriveFolder()/process.driveLoc, a
// separate root from getSasjsRootFolder()/process.sasjsRoot above - without
// this, every test in this file that creates content under the drive (e.g.
// 'level1', 'my/path/...') writes into the real, shared sasjs_root/drive
// instead of this run's isolated tmpFolder, leaking state into later runs.
jest
.spyOn(fileUtilModules, 'getFilesFolder')
.mockImplementation(() => path.join(tmpFolder, 'drive', 'files'))
import appPromise from '../../../app' import appPromise from '../../../app'
import { import {
UserController, UserController,
+105
View File
@@ -0,0 +1,105 @@
#!/usr/bin/env node
// Minimal fake "SAS executable" used by tests in place of a real SAS
// install. It only fulfils the CLI/filesystem handshake contract that
// Session.ts / processProgram.ts rely on:
//
// -SYSIN <path> the file used as a signal channel: it starts out
// containing a dummy (empty) placeholder, we delete it
// to signal "session ready", then wait for it to be
// rewritten with the real submitted program.
// -LOG <path> where we write a fake log so downstream fileExists()/
// readFile() calls have something to find.
//
// It does NOT interpret real SAS syntax. Exit code is the only thing that
// matters to the Node side: 0 mimics a normal SAS termination, non-zero
// mimics an abnormal one (e.g. %abort;).
const fs = require('fs')
const arg = (flag) => {
const idx = process.argv.indexOf(flag)
return idx === -1 ? undefined : process.argv[idx + 1]
}
const sysin = arg('-SYSIN')
const logPath = arg('-LOG')
// Give up waiting for the real program after this long, so an unused
// pre-warmed session (sasjs pools up to 3 ready sessions) doesn't linger
// forever and keep a test process alive. Generous rather than tight: CI
// runners are frequently slower/more contended than a local dev machine
// (shared CPU, coverage instrumentation on the Node side slowing down the
// round trip this process is waiting on), and this cost is only ever paid
// by sessions nothing is actually waiting on - the session actually used
// by a request gets its real code written far sooner than this in practice.
const GIVE_UP_AFTER_MS = 8000
const sleepSync = (ms) => {
const until = Date.now() + ms
while (Date.now() < until) {
/* busy-wait, mirroring the real autoexec's SAS-side sleep() loop */
}
}
// Any of the filesystem calls below can legitimately race against
// processProgram's own write-then-rename (e.g. existsSync sees the file,
// then a rename mid-flight makes the following read miss). Treat that as
// "not ready yet" and retry a few times, rather than letting an uncaught
// exception crash this process with a non-zero exit - which would be
// indistinguishable, to the Node side, from a genuine SAS failure.
const retry = (fn, attempts = 5, delayMs = 20) => {
for (let i = 0; i < attempts; i++) {
try {
return { ok: true, value: fn() }
} catch (err) {
if (i === attempts - 1) return { ok: false, error: err }
sleepSync(delayMs)
}
}
}
// 1. remove the dummy SYSIN, signalling "session ready" to waitForSession()
if (sysin) {
retry(() => {
if (fs.existsSync(sysin)) fs.unlinkSync(sysin)
})
}
// 2. wait for the real program to be written back to the same path
const deadline = Date.now() + GIVE_UP_AFTER_MS
while (!sysin || !fs.existsSync(sysin)) {
if (Date.now() > deadline) process.exit(0)
sleepSync(10)
}
// small settle delay, mirroring the real autoexec's sleep(0.01,1) after
// detecting the file, so a fast-moving rename isn't read mid-write
sleepSync(50)
const readResult = retry(() => fs.readFileSync(sysin, 'utf-8'))
if (!readResult.ok) {
process.stderr.write(
`mockSas.js: failed to read ${sysin}: ${readResult.error}\n`
)
process.exit(1)
}
const code = readResult.value
if (logPath) {
retry(() => fs.writeFileSync(logPath, `NOTE: mock SAS execution\n${code}\n`))
}
if (code.includes('%abort;')) {
process.stderr.write('ERROR: SAS session terminated. See log for details.\n')
process.exit(1)
}
process.exit(0)
+4 -1
View File
@@ -277,7 +277,10 @@ const performLogin = async (
.set('x-xsrf-token', csrfToken) .set('x-xsrf-token', csrfToken)
.send(credentials) .send(credentials)
return { authCookies: header['set-cookie'].join() } return {
authCookies:
(header['set-cookie'] as unknown as string[] | undefined)?.join() || ''
}
} }
const extractCSRF = (text: string) => const extractCSRF = (text: string) =>
+3 -1
View File
@@ -1,5 +1,6 @@
import jwt from 'jsonwebtoken' import jwt from 'jsonwebtoken'
import User from '../model/User' import User from '../model/User'
import { InfoJWT } from '../types/InfoJWT'
const isValidToken = async ( const isValidToken = async (
token: string, token: string,
@@ -11,7 +12,8 @@ const isValidToken = async (
jwt.verify(token, key, (err, decoded) => { jwt.verify(token, key, (err, decoded) => {
if (err) return reject(false) if (err) return reject(false)
if (decoded?.userId === userId && decoded?.clientId === clientId) { const payload = decoded as InfoJWT
if (payload?.userId === userId && payload?.clientId === clientId) {
return resolve(true) return resolve(true)
} }
+4231 -8629
View File
File diff suppressed because it is too large Load Diff
+1989 -1432
View File
File diff suppressed because it is too large Load Diff
+6 -4
View File
@@ -4,7 +4,9 @@
"private": true, "private": true,
"scripts": { "scripts": {
"start": "webpack-dev-server --config webpack.dev.ts --hot", "start": "webpack-dev-server --config webpack.dev.ts --hot",
"build": "webpack --config webpack.prod.ts" "build": "webpack --config webpack.prod.ts",
"lint": "npx prettier --check \"src/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\"",
"lint:fix": "npx prettier --write \"src/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\""
}, },
"dependencies": { "dependencies": {
"@emotion/react": "^11.4.1", "@emotion/react": "^11.4.1",
@@ -19,9 +21,8 @@
"@types/jest": "^26.0.24", "@types/jest": "^26.0.24",
"@types/node": "^12.20.28", "@types/node": "^12.20.28",
"@types/react": "^17.0.27", "@types/react": "^17.0.27",
"axios": "^0.24.0", "axios": "1.12.2",
"monaco-editor": "^0.33.0", "monaco-editor": "^0.33.0",
"monaco-editor-webpack-plugin": "^7.0.1",
"react": "^17.0.2", "react": "^17.0.2",
"react-copy-to-clipboard": "^5.1.0", "react-copy-to-clipboard": "^5.1.0",
"react-dom": "^17.0.2", "react-dom": "^17.0.2",
@@ -54,8 +55,9 @@
"eslint-webpack-plugin": "^3.1.1", "eslint-webpack-plugin": "^3.1.1",
"file-loader": "^6.2.0", "file-loader": "^6.2.0",
"html-webpack-plugin": "5.5.0", "html-webpack-plugin": "5.5.0",
"monaco-editor-webpack-plugin": "^7.0.1",
"path": "0.12.7", "path": "0.12.7",
"prettier": "^2.4.1", "prettier": "^3.0.3",
"sass": "^1.44.0", "sass": "^1.44.0",
"sass-loader": "^12.3.0", "sass-loader": "^12.3.0",
"style-loader": "^3.3.1", "style-loader": "^3.3.1",
+5 -5
View File
@@ -1,15 +1,15 @@
body { body {
margin: 0; margin: 0;
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', font-family:
'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 'Ubuntu',
sans-serif; 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', sans-serif;
-webkit-font-smoothing: antialiased; -webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale; -moz-osx-font-smoothing: grayscale;
} }
code { code {
font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New', font-family:
monospace; source-code-pro, Menlo, Monaco, Consolas, 'Courier New', monospace;
} }
.container { .container {