[BUG] "sill idealTree buildDeps" takes a lot of time a project with workspaces
Problem
Is there an existing issue for this? - [X] I have searched the existing issues Current Behavior After executing `npm install` the `sill idealTree buildDeps` step takes a long time (minutes) causing npm@7 working with workspaces slower than npm@6 installing projects separately. It seems that is fetching packages sequentially instead using parallelization. Vídeo with the demo: https://user-images.githubusercontent.com/1561955/118666331-61c26d80-b7f3-11eb-8060-4cb7d28480a9.mp4 Expected Behavior `npm install` on npm@7 with workspaces takes less time than npm@6 installing manually each workspace. Steps To Reproduce 1. In a environment with a lot of dependencies and workspaces. 2. With this config: [code block] 3. Run `npm install` 4. See that the step `buildDeps` take longer than expected. Some considerations: - We're using some npm private packages on our package.json. - Workspaces are dependencies of root project as well. More info: On Travis, builds using npm@6 and installing workspaces manually takes 90s. Using npm@7 it takes 170s using a single npm install. Environment - OS: macOS 11.3.1 - Node: 16.1.0 - npm: 7.17.0
Unverified for your environment
Select your OS to check compatibility.
1 Fix
Optimize npm install performance for workspaces in npm@7
The 'sill idealTree buildDeps' step in npm@7 processes dependencies sequentially instead of in parallel, leading to increased installation times, especially in projects with multiple workspaces and dependencies. This behavior differs from npm@6, which handles installations more efficiently in such scenarios.
Awaiting Verification
Be the first to verify this fix
- 1
Upgrade npm to the latest version
Ensure that you are using the latest version of npm, as performance improvements and bug fixes are frequently released. This may include optimizations for workspace handling.
bashnpm install -g npm@latest - 2
Use npm ci for consistent installs
If you have a package-lock.json file, consider using 'npm ci' instead of 'npm install'. This command is optimized for CI environments and can improve installation speed by skipping certain steps.
bashnpm ci - 3
Check for circular dependencies
Inspect your workspace configuration for circular dependencies, as these can significantly slow down the dependency resolution process. Refactor any circular dependencies found.
bashnpm ls --depth=0 - 4
Enable npm parallel installation
If not already enabled, consider using the 'npm set' command to allow parallel installation of packages. This can help speed up the installation process.
bashnpm set progress=false - 5
Profile installation performance
Use the npm's built-in performance profiling to identify bottlenecks during the installation process. This can help pinpoint specific areas that need optimization.
bashnpm install --loglevel=silly
Validation
To confirm the fix worked, run 'npm install' again and measure the time taken for the 'sill idealTree buildDeps' step. Compare the time with previous runs to ensure a decrease in duration. Additionally, check for any warnings or errors during the installation.
Sign in to verify this fix
Environment
Submitted by
Alex Chen
2450 rep