eslint loader is horribly inefficient

[isaacl]

eslint-loader takes about 10s to run through 5k lines of js files. This appears to be a combination of:

• initializing eslint once per file
• using executeOnText instead of executeOnFile, which means users can't take advantage of the eslint cache feature.

Is there anything which can be done for these?

[MoOx]

initializing eslint once per file

Yeah maybe that can be optimized

using executeOnText instead of executeOnFile, which means users can't take advantage of the eslint cache feature.

Sadly, not sure it's a good idea to switch. If loaders are used before this one, executeOnFile might not be relevant since the source provided might be different that the one on the fs.
And webpack have a pretty good cache, so should not be a real issue. Maybe we can do something else.

Is there anything which can be done for these?

Totally. You work on it, make this loader better and submit a PR. That would be lovely.

[dvdzkwsk]

As an FYI, I started fiddling with it (totally new to writing webpack loaders/using the ESLint Node API), but from my preliminary investigation the true slowdown is with executeOnText (as both of you probably know). I updated the code to only initialize eslint once, rather than on each file, but the savings were negligible (< 1s shaved off of 16s for an initial build [~11s of that is eslint-loader]), so the true win would come from optimizing the linter execution.

Will keep messing with it and report back if I find anything substantial.

[isaacl]

Sorry didn't mean to sound harsh. Normally I'd be happy to pitch in but I think eslint is just too slow to run it like this. I'm going to write a standalone script that only calls eslint on changed files based on git.

[isaacl]

For anyone else interested, here's the script I wrote for faster eslint https://gist.github.com/isaacl/3451f9e048b5cc51c9e4

[thangngoc89]

@isaacl how to execute it ? Could you port it to node ?

[isaacl]

@thangngoc89 I'm not a node expert, but if you want to port you may find the version I wrote in bash easier to follow. I wrote this script to be checked into the repository it's used on, and it uses git to find the root of the repository.

In "fast mode" it runs two git commands to changed/added files and untracked files respectively. Then it calls eslint --cache with these. You can look at eslint docs to see what the options do.

c/reporting (eslintSpeedup)$ scripts/lint -fv
> eslint --ext .jsx,.js --cache --max-warnings=0 gruntfile.js webpack.config.js
  eslint:cli-engine Processing /Users/isaac.levy/code/reporting/gruntfile.js +0ms
  eslint:cli-engine Linting /Users/isaac.levy/code/reporting/gruntfile.js +2ms
  eslint:cli-engine Load plugin react +2ms
  eslint:cli-engine Processing /Users/isaac.levy/code/reporting/webpack.config.js +512ms
  eslint:cli-engine Linting /Users/isaac.levy/code/reporting/webpack.config.js +0ms
  eslint:cli-engine Linting complete in: 862ms +45ms

[thangngoc89]

@isaacl thanks a bunch. bash is even better

[MoOx]

FYI https://www.npmjs.com/package/lint-staged

[thangngoc89]

@isaacl @MoOx 👍

[kodonnell]

I'm completely new to webpack, but I've been using https://www.npmjs.com/package/eslint_d. Not sure if it's relevant, given you're probably not starting up separate node processes.

As another thought - is it possible to only lint the files that changed? Generally that's all I want while developing (i.e. to see if my changes introduced/fixed any errors) and it'd obviously be a lot faster ...

[MoOx]

Webpack will rebuild (so relint) only updated files (after the first run).

[genintho]

Here is my proposal to make the linter faster on cold build #93

It remove 30% of my webpack first built time (around 1 minute for my application).