Do you know the difference between npm, yarn, and pnpm?

npm

Nested node_modules structure

In the early days, npm adopted a nested node_modules structure. The “node_modules” folder usually contained the modules that the project depended on. Nested “node_modules” structures occur when multiple dependencies are used in a project and these dependencies themselves have their own dependencies.

The main feature of the nested “node_modules” structure is that dependent modules are nested within the folders of their parent modules, rather than all dependencies being placed in the “node_modules” folder in the project root directory. The advantage of this structure is that it ensures that each dependency uses the specific version it requires, thereby improving the reliability of version management.

Here’s an example showing a project directory containing nested “node_modules” structures:

my-project/
|-- node_modules/
| |-- dependency-1/
| | |-- node_modules/
| | | |-- nested-dependency-1/
| | | | |-- ...
| | |
| | |-- ...
| |
| |-- dependency-2/
| | |-- ...
| |
| |-- ...
|
|-- package.json
|-- package-lock.json (or yarn.lock)
|-- ...

The nested “node_modules” structure allows the version of each dependency to be isolated and avoids version conflicts between different dependencies. This helps ensure project stability and maintainability. The project’s “package.json” file and lock files (such as “package-lock.json” or “yarn.lock”) are responsible for managing dependencies and ensuring the correct versions are installed. Although nested “node_modules” structures can resolve version conflicts, they also increase the file size of your project.

Dependency Hell

“Dependency Hell” refers to problems that may arise in JavaScript projects, especially when using npm for dependency management. This term describes a situation where dependencies in a project become very complex and difficult to manage and resolve.

Dependency hell may cause the following problems:

1. Deep dependencies: Project dependencies have become very deep, and dependencies depend on other dependencies, forming a complex dependency chain. This makes the project’s “node_modules” directory huge and takes up a lot of disk space.

2. Version conflict: Different dependencies may depend on different versions of the same module, resulting in version conflicts. This can lead to code bugs, instability, and inconsistent behavior.

3. Maintenance Difficulty: As a project grows, it becomes very difficult to maintain complex dependencies and ensure that all dependencies are kept up to date. Manually resolving dependency issues can be time-consuming.

4. Security Issues: Complex dependencies may lead to potential security vulnerabilities in the project, because some dependencies may contain known vulnerabilities and are difficult to update in a timely manner.

To mitigate the “NPM dependency hell” problem, here are some things you can do:

1. Use lock files: Use “package-lock.json” (for npm) or “yarn.lock” (for Yarn) to ensure dependency version consistency. This will reduce the risk of version conflicts.

2. Regularly update dependencies: Regularly update dependencies to get the latest versions to ensure that the project remains up to date and secure. Tools are available to help automate this process.

3. Streamline dependencies: Review project dependencies and remove unnecessary dependencies to reduce project complexity.

4. Use tools: Use tools such as npm-check, npm audit, etc. to help identify and solve dependency problems.

5. Regularly check for vulnerabilities: Use vulnerability scanning tools to check whether there are known vulnerabilities in project dependencies, and update affected dependencies in a timely manner.

Flat node_modules structure

In order to flatten nested dependencies as much as possible and avoid excessively deep dependency trees and package redundancy, npm v3 hoists sub-dependencies and adopts a flat node_modules structure. Dependencies are kept as flat as possible, without deep nested structures. Under this structure, all dependent packages are installed directly in the node_modules folder in the project root directory, without multi-level nesting.

The main features of the flat node_modules structure include:

1. All dependencies are installed directly in the root directory: Each dependency will be installed in the node_modules folder of the project, but not in the node_modules of other dependencies. > Create nested structures within folders.

2. Reduce the size of the node_modules folder: Since dependencies are not nested at multiple levels, the size of the node_modules folder is relatively small and takes up less disk space. Less space.

3. Reduce version conflicts: A flat structure helps reduce version conflicts between different dependencies, because each dependency has its own copy and will not be affected by other dependencies.

Using a flat node_modules structure can solve the “NPM dependency hell” problem and reduce the complexity and performance issues of dependency management. This structure can be useful in some situations, especially for large projects or projects that require precise version control.

To enable a flat node_modules structure in npm, you can use npm‘s --legacy-peer-deps option, for example:

npm install --legacy-peer-deps

This will force npm to use a flat structure for installing dependencies. But this may cause some dependencies to not work properly.

Ghost dependencies

“Phantom dependencies” means that in a Node.js project, when installing dependencies through npm, some dependencies may be installed that do not appear to be explicitly listed. These dependencies are not explicitly listed in the project’s package.json file, but they are still installed automatically due to some behavior of npm when parsing the dependency tree.

Phantom dependencies can cause a number of problems, including:

1. Unclear dependencies: The project’s package.json file does not clearly list these dependencies, which may cause other developers to not know the actual dependencies of the project.

2. Version control issues: Since these dependencies are not explicitly listed, it may lead to inaccurate dependency version control of the project, increasing the risk of version conflicts.

3. Difficulty in maintenance: Phantom dependencies may make the project’s dependencies more complex, thereby making the project more difficult to maintain.

Phantom dependencies usually appear in the following situations:

• peerDependencies: If a dependency’s peerDependencies contains some dependencies that are not explicitly listed in the project, npm may automatically install these unlisted dependencies as ghost dependencies.

• Indirect dependencies: If the project’s dependencies have complex dependencies, where one dependency requires another dependency but is not explicitly listed in the project’s package.json, npm may install this dependency as a ghost dependency.

In order to solve the Phantom dependencies problem, you can take the following measures:

1. Review dependencies: Regularly review the project’s node_modules directory to see if there are any dependencies that are not explicitly listed. You can use the npm ls or yarn list command to list the project’s dependency tree.

2. Explicitly list dependencies: Explicitly list all required dependencies in the project in the dependencies or of the package.json file devDependencies to ensure dependencies are explicit.

3. Update dependencies: Regularly update the project’s dependencies to ensure that their versions are up to date, and to fix any Phantom dependencies.

4. Use npm audit: Use the npm audit command to check whether the dependencies in the project have known security vulnerabilities, including Phantom dependencies.
   For example:

{
  "dependencies": {
    "A": "^1.0.0",
    "C": "^1.0.0"
  }
}

Since B is promoted to the same level as A during installation, referencing B in the project can still work normally.

Depend on clone

“Doppelgangers” means that in a Node.js project, when installing dependencies through npm, multiple versions of the same dependency package may exist simultaneously in the node_modules directory of the project. These different versions of dependency packages usually come from direct dependencies and indirect dependencies of the project, and their versions may be inconsistent.

Dependency issues may cause the following problems:

1. Version conflict: Different versions of the same dependency package may be incompatible, causing errors or instability in the project.

2. Disk usage: Multiple versions of dependent packages will occupy disk space and increase the size of the project.

3. Instability: Dependency clones can lead to inconsistent project behavior because different code paths use different dependency versions.

Solutions to the problem of dependency on clones include:

1. Use the npm ls command: Use the npm ls command to view the dependency tree of the project and identify which dependencies exist in multiple versions.

2. Update dependencies: Try to update the project’s dependencies so that they use the latest versions, thereby reducing the possibility of version conflicts.

3. Delete duplicate dependencies: Manually delete unnecessary duplicate dependencies in the project. You can use the npm dedupe command to automatically solve some problems.

4. Lock dependency versions: Use package-lock.json or yarn.lock to lock files to ensure that the dependency versions used in the project are consistent.

5. Use npm audit: Use the npm audit command to check whether the project’s dependencies have known security vulnerabilities, including dependency clones.

6. Use npm ci: For production environment builds, you can use the npm ci command to quickly and reliably install dependencies and reduce the problem of dependency clones.

Uncertainty

For the same package.json file, you may not get the same node_modules directory structure after installing dependencies. If there are changes in package.json, you need to delete node_modules locally and reinstall it. Otherwise, the node_modules structure of the production environment and the development environment may be different, and the code cannot run normally.

yarn

Yarn is a package management tool for managing JavaScript project dependencies. It was launched after npm and aims to provide faster, reliable and secure dependency management.
Yarn also has a flat node_modules structure, which does not solve the problems of ghost dependencies and dependency clones.

Features

1. Quick installation: Yarn’s parallel installation and caching mechanism make the installation of dependent packages faster. It can download multiple dependencies in parallel, thereby improving installation efficiency. This is especially useful for large projects or projects with a lot of dependencies.

2. Version Lock: Yarn uses a yarn.lock file to ensure that the versions of dependent packages are consistent in different environments. This helps avoid dependency version conflicts between different developers or different deployment environments.

3. Reliability: Yarn’s dependency resolution algorithm is more reliable and can avoid some npm-related problems, such as dependency clones (Doppelgangers) and ghost dependencies (Phantom dependencies).

4. Offline support: Yarn allows you to install dependencies without an internet connection, provided you have cached them in a previous installation. This is useful for developers working in environments without internet connectivity.

5. Security: Yarn provides a yarn audit command to check whether the dependencies in the project have known security vulnerabilities and provide repair suggestions. This helps improve the security of your project.

6. Easy to use: Yarn’s command line interface (CLI) is similar to npm, so there is a low learning curve for developers familiar with npm. At the same time, Yarn also provides some additional functions and commands, such as yarn workspaces for managing multi-package repositories.

7. Workspace support: Yarn supports workspaces, allowing you to more easily manage dependencies between multiple related projects or package repositories. This is useful for projects using monorepo style.

8. Plug-in support: Yarn supports plug-ins, and its functionality can be extended by installing plug-ins. This allows developers to customize and extend Yarn as needed.

pnpm

Improvements with yarn and npm

1. Hard links and symbolic links: pnpm uses hard links and symbolic links to reuse instances of the same dependencies instead of duplicating dependencies for each project. This reduces disk space usage and reduces the copying of dependent packages.

2. Shared storage: pnpm introduces a global dependency storage location called “store”, which can reuse dependencies across multiple projects. This reduces network downloads and local disk usage, especially for developers with multiple projects.

3. Parallel installation and update: pnpm supports parallel installation and update of dependencies, which means that it can handle the installation and update operations of multiple dependencies at the same time faster.

4. Automatic garbage collection: pnpm has a built-in garbage collection mechanism that regularly cleans up dependencies that are no longer needed to free up disk space.

5. Optional version locking: pnpm provides an optional version locking mode, and developers can flexibly choose whether to lock dependent versions according to their needs. This makes pnpm suitable for a wider range of project needs.

6. Fast unpacking: pnpm introduces the “fast unpacking” function, which can start the project faster and reduce waiting time.

7. Compatibility: pnpm claims to be compatible with the npm and Yarn ecosystems, so switching to pnpm can be seamless in existing projects.

8. CLI commands: pnpm provides similar CLI commands to npm and Yarn, making it easy to learn and use.

Content-addressed storage

Content-Addressable Storage (CAS) is a method of storing dependent packages that hashes the content of each package and stores it in a repository with a unique hash address. This method is different from the traditional file copying method, in which each project will store a complete dependency package in the node_modules directory.

Key features and benefits of CAS include:

1. Save disk space: CAS only stores the contents of each package once, no matter how many projects depend on it. This reduces disk space usage, especially for developers with multiple projects.

2. Efficient network download: CAS allows reuse of instances of the same dependency, so there is no need to download the same dependency package multiple times, which improves the efficiency of network downloads.

3. Dependency isolation: Each project’s node_modules directory contains a reference to a specific hash address in CAS, which means that each project’s dependencies are isolated and not mutually exclusive. interference.

4. Version locking: CAS combined with hash address can ensure the consistency of dependent versions. As long as the contents of the package don’t change, the hash address doesn’t change, so version conflicts can be avoided.

5. Garbage Collection: CAS has a built-in garbage collection mechanism that regularly cleans up dependencies that are no longer needed to free up disk space.

Summary

npm, Yarn and pnpm are JavaScript package management tools for downloading, installing and managing JavaScript packages and dependencies. They have their own advantages and disadvantages, and choosing which one to use depends on project needs and personal preference.

Here are the main differences between them, as well as their pros and cons:

npm (Node Package Manager):

• Advantages:

  • npm is the package management tool officially provided by Node.js, and it is the default package manager.
  • Has extensive community support and ecosystem, including a large number of open source packages and modules.
  • Supports custom scripts for building and testing.
  • Compatible with CommonJS module specification.

• Disadvantages:

  • npm is relatively slow in terms of performance, especially when installing a large number of dependencies.
  • When installing and deleting dependencies, a large number of intermediate files and dependencies will be generated.

Yarn:

• Advantages:

  • Yarn is developed in partnership with Facebook, Google, and Exponent to improve performance and reliability.
  • Supports parallel downloads for faster installation.
  • The mechanism for locking dependency versions is more reliable and ensures consistency across different development environments.
  • There is a simplified CLI.

• Disadvantages:

  • Relative to npm, Yarn’s ecosystem is slightly smaller but already growing rapidly.

pnpm (Plug’n’Play):

• Advantages:

  • The biggest advantages of pnpm are extremely low disk footprint and faster installation since it does not create a lot of intermediate files and instead stores the packages in a global cache.
  • It has excellent support, including compatibility with the npm ecosystem.

• Disadvantages:

  • Although pnpm can significantly reduce disk usage, its ecosystem is relatively small and some dependencies may not be fully compatible.
  • Some tools and CI/CD environments may require some additional configuration to support pnpm.

When to use:

• Use npm: npm is a good choice when you need a compatible default package manager and you don’t care much about performance differences. It works fine in most cases.

• Use Yarn: Yarn is a good choice when you need faster installations, more reliable version control, and parallel downloads. It is suitable for larger projects and situations where tighter dependency management is required.

• Use pnpm: When you care about disk usage and faster installation speed, and your project has good compatibility, you can consider using pnpm. Especially in containerized environments, it can significantly reduce image size.