Create a node.js target for xterm.js

[JavaCS3]

Details

• OS version: node
• xterm.js version: 4.4.0

> require('xterm')
Thrown:
ReferenceError: window is not defined
    at Object.<anonymous> (/private/tmp/test/node_modules/xterm/lib/xterm.js:1:224)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)

Steps to reproduce

https://runkit.com/embed/q7w97d0b8wj2

Looks like webpack4 bug,
See: https://medium.com/@JakeXiao/window-is-undefined-in-umd-library-output-for-webpack4-858af1b881df

[Tyriar]

xterm.js won't work in node because it relies on window and the DOM, it needs a browser.

[JavaCS3]

@Tyriar Though xterm.js doesn't support node env currently, it should be working at importing level. The fix is quite simple, set globalObject: 'this' in webpack output will change window to this.

I wish xterm.js will support headless mode some day.

[Tyriar]

@JavaCS3 if JavaCS3@54ec873 is all that's needed to unblock you for this then we can take that, there may be other runtime problems though? I'm also not totally sure how a headless node-only version is useful. It looks like xterm-player is a browser app?

[jerch]

I think we should try to get the planned separation of the core and browser parts finished. This would solve those issues and also unclutter the logic by a clean separation of business logic (term emulation) and screen output. Once we got this done, we could even create a new build target for the coreparts only, to be used as a library in nodejs (xtermlib.js, whatsoever).

@JavaCS3 I currently use core parts of xterm.js in nodejs by pulling the source into a project. This way I can access internal interfaces and build them with TS as "native source". This is ofc not a viable general solution, I can only do that since I know which parts are relatively stable or changed from release to release. Its still a major step forward compared to what we had before (around v3.12/v4) with mixed browser and core stuff all over the place.

On the other hand - messing with the global object sounds like a bad idea to me. I am pretty sure that this will lead to follow up issues and should not be part of the official build. If that's a neat quickfix for your issue at hand (@JavaCS3) - maybe maintain a fork with it applied until we got the separation finished?

[JavaCS3]

@Tyriar It's not that urgent for JavaCS3@54ec873
I can wait until xterm.js core parts get separated or workaround in someway.

xterm-player is a browser app.

The idea of running xterm.js in headless mode is, I want to create a terminal based live sharing stuff that allow anyone to join that sharing session at anytime. Just like a video sharing, but it's a terminal based version. I want xterm.js can generate some key frames in the backend server. If anyone joins this session, the server will just send that keyframe and the following term sequence to him. That's why I want a headless mode.

[the21st]

I'd also love it if xterm supported node. I originally thought node is supported, since xterm can already run in "headless" mode: #266 , without being attached to a DOM.

I tried a simple workaround in my code, just setting global.window = undefined before require('xterm') and that fixed the ReferenceError and everything is working as expected. It's just a pain to have to set that everywhere, including in tests.

Awesome work on xterm by the way 🙂

Edit: this is what got me to think Node.js should work 🙂 https://github.com/xtermjs/xterm.js/blob/master/src/Terminal.ts#L70-L71 :

// Let it work inside Node.js for automated testing purposes.
const document = (typeof window !== 'undefined') ? window.document : null;

[jerch]

I tried a simple workaround in my code, just setting global.window = undefined before

Thats a neat trick to get it loading. Also most of the core parts should work, thats basic I/O (write and onData), basic parser stuff and reading from the buffer. But I am not sure if all browser access code is already hidden behind .open or behind proper browser service checks, I remember still getting document errors withterm.write tests months ago w'o touching .open.

As I wrote above, the untangling is on the way, but not yet finished. If someone cares enough - the final cream topping would be a renderer abstraction for nodejs w'o any browser semantics (output to parent terminal, SDL/OpenGL).

[Tyriar]

I think we should try to get the planned separation of the core and browser parts finished.

To be clear, it was never my intention in #1507 to be able to run xterm.js within node.js. Just that we would have a clean separation in code such that we would never use the DOM within "common" (previously core) and that we could write pure node.js unit tests for core components without needing to bring in something like jsdom for them.

Once we got this done, we could even create a new build target for the coreparts only, to be used as a library in nodejs (xtermlib.js, whatsoever).

If it's easy to make an extra target (I guess with a separate .d.ts?) then we could consider it after that. However, I was also starting to rethink #1507 that maybe it should separated in a more component-based way instead of layers and then use eslint to enforce layering. This might cause issues with a node-only lib, but it would allow moving components closer together (eg. everything to do with links all in the same component as opposed to split into common and browser).

[jerch]

To be clear, it was never my intention in #1507 to be able to run xterm.js within node.js.

Thats clear and also would not be possible for the browser part (unless someone steps in and reimplements 50% of the codebase by other means). But wouldn't it be a shame not to make the core parts available in nodejs? By testing them in nodejs we already have unofficial node support, it just misses a neat access interface. (Edit: To emphasize this even more - strictly speaking, we kinda guarantee them to work in nodejs, but not in the browser, as we dont run the full unit tests in the browser.)

If it's easy to make an extra target (I guess with a separate .d.ts?) then we could consider it after that.

Yepp - I think all it needs, is a stripped down interface removing all browser notions. It could either be a separate build target in a subfolder with a custom package.json, or even a separate repo, that pulls from the main repo from time to time. Not sure yet which would be easier to maintain in the long run.

I was also starting to rethink #1507 that maybe it should separated in a more component-based way instead of layers and then use eslint to enforce layering. This might cause issues with a node-only lib, but it would allow moving components closer together (eg. everything to do with links all in the same component as opposed to split into common and browser).

Isnt this giving up the degree of separation we already have achieved? What real benefits do you see in such a step? To me moving parts more close together is not helpful, if the entities still show a high complexity. For a terminal emulator it seems to be logical to separate the actual emulation (business logic) from the outer world (browser), be it for testing it separately or to be able to connect it to different outer world interfaces (non browser, which is not a primary goal of term.js). Btw Bob Martin has a nice talk about this (https://www.youtube.com/watch?v=o_TH-Y78tt4).

When I reworked the mouse event handling, I modelled it to have a core and a browser part (thats the reason for creating the CoreMouseService). Imho this level of indirection has a major advantage - it separates the emulation concerns from browser related I/O. In the case of mouse events - we can test independently for the correctness of the terminal mouse protocols, and can deal with mouse events in the browser parts.

[Tyriar]

To emphasize this even more - strictly speaking, we kinda guarantee them to work in nodejs, but not in the browser, as we dont run the full unit tests in the browser.

We should probably just use a browser runner at some point, that way we can get rid of jsdom.

Isnt this giving up the degree of separation we already have achieved? What real benefits do you see in such a step? To me moving parts more close together is not helpful, if the entities still show a high complexity. For a terminal emulator it seems to be logical to separate the actual emulation (business logic) from the outer world (browser), be it for testing it separately or to be able to connect it to different outer world interfaces (non browser, which is not a primary goal of term.js). Btw Bob Martin has a nice talk about this (https://www.youtube.com/watch?v=o_TH-Y78tt4).

When I reworked the mouse event handling, I modelled it to have a core and a browser part (thats the reason for creating the CoreMouseService). Imho this level of indirection has a major advantage - it separates the emulation concerns from browser related I/O. In the case of mouse events - we can test independently for the correctness of the terminal mouse protocols, and can deal with mouse events in the browser parts.

It would be shuffling around the code base such that we still have most of the separation, just related files are more closely located. Take the mouse services as an example, here is the current layout:

browser/
  tsconfig.json
  input/
    Mouse.test.ts
    Mouse.ts
  services/
    MouseService.ts
common/
  tsconfig.json
  services/
    CoreMouseService.test.ts
    CoreMouseService.ts

This could be changed into something like this:

components/
  mouse/
    tsconfig.json
    common/
      MouseService.test.ts
      MouseService.ts
    browser/
      MouseEventUtil.test.ts
      MouseEventUtil.ts
      BrowserMouseServiceDecorator.ts

Where BrowserMouseServiceDecorator.ts augments MouseService.ts when it's attached (or some other similar design pattern, would need some thought how this works). This close proximity and better encapsulation has huge benefits in understanding the codebase, no longer are features scattered in different folders but completely contained within a single component folder. This would also allow us to do things like specify a public API in components/mouse/mouse.d.ts and only allow other modules to import from there, additionally since the component itself is a TS project, it could define it's dependencies (need to fix through files for this now: renderservice, charsizeservice, coreservice).

What we would lose here is tsconfig.json enforcing that the common part cannot import from browser, however we could enforce this using eslint (integration tests help verify too). Depending on how it's done we may lose the ability to do a node target, but it could certainly be done in a way that we could achieve that.