npm module installation mechanism

Table of Contents

1. npm module installation mechanism

2.npm implementation principle

1. Execute the preinstall of the project itself

2. Determine the first-level dependent modules

3. Get the module

4. Module flattening

5. Install the module

6. Execute the life cycle of the project itself

7. Generate or update the version description file, and the npm install process is completed.

3. Solution for offline installation

1. A solution that still has problems

Offline solution idea 1

Offline solution idea 2

Offline solution idea 3

1. npm module installation mechanism

Issue the npm install command

Query whether the specified module already exists in the node_modules directory

  • If it exists, it will not be reinstalled, even if there is already a new version in the remote repository.
  • If it does not exist:
    • npm queries the registry for the URL of the module’s compressed package
    • Download the compressed package and store it in the .npm directory in the root directory
    • Unzip the compressed package to the node_modules directory of the current project

If you want npm to force reinstall a module regardless of whether it has been installed before, you can use -f or –force

npm install –force

【Replenish】

1. npm update: update installed modules

①: Will first go to the remote warehouse to query the latest version:

The npm module repository provides a query service, registry

  • Taking npmjs.org as an example, its query service URL is https://registry.npmjs.org/.

  • Add the module name after the URL, and you will get a json object containing all version information of the module, https://registry.npmjs.org/react.

  • Add the version number or label after the module name to query the specific version information https://registry.npmjs.org/react/0.14.6 to see React version 0.14.6

  • The dist.taball attribute in the json file is the compressed package address of this version

②: Then query the local version. If the local version does not exist or the remote version is newer, it will be installed.

2. Cache directory:

The npm install and npm update commands, after downloading the compressed package from the registry, will be stored in the local cache directory. In Linux and Mac, the default is the .npm directory in the user’s home directory. In Windows, it is %AppData%/npm-cache.

$ npm config get cache
$home/.npm

//View directory
$ ls ~/.npm
# or
$ npm cache ls

Each version of each module has its own subdirectory, and the code inside is the compressed package package.tgz file and a description file.

In addition, a {cache}/{hostname}/{path}/.cache.json file will also be generated. For example, when downloading the react module from the npm official repository, the registry.npmjs.org/react/.cache.json file will be generated. This file saves all version information, as well as the most recent modification time of the module and the ETag returned by the server during the most recent request.

2.npm implementation principle

After entering the npm install command, it will go through the following stages:

1. Execute the preinstall of the project itself

If the preinstall hook is defined in the current project, it will be executed at this time.

2. Determine the first-level dependent modules

The first thing to do is to determine the first layer of the project, which is the module directly specified in the dependencies and dev-denpendencies attributes. (Assuming no npm install parameters are added at this time)

The project itself is the root node of the entire dependency tree. Each first-level dependent module is a subtree under the root node. npm will start multiple processes to gradually search for deeper-level nodes starting from each first-level dependent module.

3. Get module

Obtaining the module is a recursive process, divided into the following steps:

  • Obtain module information: Before downloading a module, first determine its version, because package.json is often a semantic version (semantic version). At this time, if the version description file (npm-shrinkwrap.json or package-lock.json) If you have the module information, you can just get it directly, if it is not obtained from the warehouse. (For example, the version of a package in packaeg.json is ^1.1.0, npm will go to the warehouse to get the latest version in the form of 1.x.x)
  • Get the module entity: In the previous step, you will get the compressed package address of the module (resolved field). NPM will use this address to check the local cache. If it is in the cache, it will be taken directly. If not, it will be downloaded from the warehouse.
  • Find the dependencies of the module: if there are dependencies, execute from the beginning; if not, stop

4. Module flattening

What you get in the previous step is a complete dependency tree, which may contain a large number of repeated modules (for example, module A depends on loadsh, and module B also depends on lodash.). Before npm3, installation was strictly based on the dependency tree structure, which resulted in module redundancy.

Starting from npm3, a dedupe process has been added by default, which will traverse all nodes and place the modules one by one under the root node, which is the first layer of node_modules. When duplicate modules are found, they will be discarded.

Duplicate module: It means that the module name is the same and the semver (semantic version number) is compatible. Each semver corresponds to a version allowed range. If the version allowed ranges of the two modules overlap, you can get a Compatible version. The version numbers do not have to be exactly the same, allowing more redundant modules to be removed during the dedupe process.

node_modules
--foo
---- lodash@version1

--bar
---- lodash@version2

Assuming that version1 and version2 are compatible versions, after dedupe it will become the following form:
node_modules
-- foo

--bar

-- lodash (the reserved version is a compatible version)



Assuming that version1 and version2 are incompatible versions, later versions remain in the dependency tree:
node_modules
--foo
-- lodash@version1

-- bar
---- lodash@version2

5. Install module

This step will update node_modules in the project and execute the life cycle function in the module (preInstall install postInstall sequence)

6. Execute the project’s own life cycle

If the current npm project has a hook defined, it will be executed at this time (install postInstall prePublish prepare sequence)

7. Generate or update the version description file, the npm install process is completed

3. Offline installation solution

After a module is installed, two copies are saved locally, one is the compressed package saved in .npm, and the other is the decompressed code in node_Modules. However, when running npm install, only node_modules will be checked, that is to say, When a module has a compressed package under .npm but is not installed in node_modules, npm will also download it from the remote repository.

It will greatly affect the installation speed, and in an environment without network, even if there is an installation package locally, it cannot be used.

1. A solution that still has problems

To solve the above problem, npm provides a –cache-min parameter to install modules from the cache directory

The –cache-min parameter specifies a time (minutes). Only modules that exceed this time will be downloaded from the registry.

npm install --cache-min 9999999 <package-name>

npm install --cache-min Infinity <package-name>

But this does not mean offline mode, it still requires network downloading

If the specified module is in the cache directory, npm will also connect to the registry and issue the etag of the specified module. The server will return status code 304, indicating that there is no need to re-download the compressed package.

If a module is already in the cache, but the version is lower than required, npm will report an error directly instead of going to the registry to download the latest version.

Offline solution idea 1

  • npm-proxy-cache
  • local-npm (usage)
  • npm-lazy

Registry agent, a registry service is started on this machine. All npm install commands will be changed through a service agent, which can fully realize cache installation and use offline.

Offline solution idea 2

npm install replaces, npm-cache replaces npm install

Offline solution idea 3

Use node_modules as cache directory

  • Freight
  • npmbox

Pack the project’s node_modules directory into a compressed package, and remove the files from this compressed package during subsequent installations.

The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge. Vue entry skill treeNode.js and npmNode installation and configuration 39638 people are learning the system