In the Angular Component Router documentation I just stumbled over a npm command I have never seen before and I don't understand what is going on:
npm install #angular/router --save
What is the meaning of #angular/router?
Is the whole string a package name? But then I dont find that package when I use the search on npmjs.com.
And also the commandline search does return no such package:
npm search #angular/router
:No match found for "#angular/router"
So is the #angular/ some kind of prefix mechanism in npm? And how does it work?
This is a new feature of NPM called 'scoped packages', which effectively allow NPM packages to be namespaced. Every user and organization on NPM has their own scope, and they are the only people who can add packages to it.
This is useful for several reasons:
It allows organizations to make it clear which packages are 'official' and which ones are not.
For example, if a package has the scope #angular, you know it was published by the Angular core team.
The package name only has to be unique to the scope it is published in, not the entire registry.
For example, the package name http is already taken in the main repository, but Angular is able to have #angular/http as well.
The reason that scoped packages don't show up in public search is because a lot of them are private packages created by organizations using NPM's paid services, and they're not comfortable opening the search up until they can be totally certain they're not going to make anything public that shouldn't be public - from a legal perspective, this is pretty understandable.
For more information, see the NPM docs and the Angular docs.
EDIT: It appears that public scoped packages now show up properly in search!
Basically there are two types of modules on npm, they are -
Global modules - these are modules that follow the naming convention that exists today. You require('foo') and there is much rejoicing. They are owned by one or more people through the npm install XYZ command.
Scoped modules - these are new modules that are "scoped" under an organization name that begins with an # the organisation's name, a slash and finally the package name, e.g. #someOrgScope/packagename. Scopes are a way of grouping related packages together, and also affect a few things about the way npm treats the package.
A scoped package is installed by referencing it by name, preceded by an #-symbol, in npm install:
npm install #myorg/mypackage
see also
http://blog.nodejitsu.com/a-summary-of-scoped-modules-in-npm/
https://docs.npmjs.com/misc/scope
# has different means according to its place where it is in the npm package name.
A package is:
A folder containing a program described by a package.json file.
A gzipped tarball containing (1).
A url that resolves to (2).
A <name>#<version> that is published on the registry with (3).
A <name>#<tag> that points to (4).
A <name> that has a “latest” tag satisfying (5).
A <git remote url> that resolves to (1).
npm install [<#scope>/]<name>
<scope> is optional. The package will be downloaded from the registry associated with the specified scope. If no registry is associated with the given scope the default registry is assumed.
Note: if you do not include the #-symbol on your scope name, npm will interpret this as a GitHub repository instead, see below. Scopes names must also be followed by a slash.
npm install [<#scope>/]<name>#<tag>
Install the version of the package that is referenced by the specified tag. If the tag does not exist in the registry data for that package, then this will fail.
Example:
npm install packagename#latest
npm install #myorg/mypackage#latest
npm install [<#scope>/]<name>#<version>
Install the specified version of the package. This will fail if the version has not been published to the registry.
Example:
npm install packagename#0.1.1
npm install #myorg/privatepackage#1.5.0
npm install [<#scope>/]<name>#<version range>
Install a version of the package matching the specified version range.
Example:
npm install packagename#">=0.1.0 <0.2.0"
npm install #myorg/privatepackage#">=0.1.0 <0.2.0"
What are scoped modules.
All npm packages have a name and these name should be unique. A scoped npm package follows the same rules as other npm package names (URL-safe characters, underscores or no leading dots). When used in package names, scopes are preceded by an # symbol and followed by a slash /, e.g.
#somescope/somepackagename
The npm scoped modules are usually grouped related npm packages together. When you sign up for an npm user account or create an organization. Each npm user/organization has their own scope, and only they and their employees can add packages in your scope. Usually you are granted a scope that matches your user or organization name. You can use this scope as a namespace for related packages.
As a npm user you don't have to worry about someone taking your package name ahead of you. Thus using scope module is also a good way to organize the npm packages for an organizations.
Advantages of using scoped packages:
Scoped packages allows organizations to manage the private packages.
The Scoped package name only has to be unique to the scope in which it is published in, not the entire npm registry.
Usually organizations choose to keep their scoped packages private and they don't show up in public search for various reasons.
Related
I am trying to use this library here: multi-rake
However, as stated in the docs, we have to run this before installing multi-rake:
CFLAGS="-Wno-narrowing" pip install cld2-cffi
So I cannot simply put cld2-cffi and multi-rake in requirements.txt because cld2-cffi needs to be installed like this beforehand. How could I overcome this problem?
According to the official documentation you have to package as local dependencies.
You can also package and deploy dependencies alongside your function.
This approach is useful if your dependency is not available via the
pip package manager or if your Cloud Functions environment's internet
access is restricted. For example, you might use a directory structure
such as the following:
You can then use code as usual from the included local dependency,
localpackage. You can use this approach to bundle any Python packages
with your deployment.
Note: You can still use a requirements.txt file to specify additional
dependencies you haven't packaged alongside your function.
Specifying dependencies in Python
I have a problem with a requirement, hope you can help me.
I have 1 package named #unicoderns/orm, that requires mysql
https://github.com/unicoderns/ORM
Now Im trying to code another OpenSource package named #unicoderns/cerberus, cerberus has the orm as dependency, but when I try to build cerberus I get this error:
node_modules/#unicoderns/orm/connection.d.ts:4:24 - error TS7016: Could not find a declaration file for module 'mysql'. '/home/chriss/Sites/cerberus/node_modules/mysql/index.js' implicitly has an 'any' type.
Try npm install #types/mysql if it exists or add a new declaration (.d.ts) file containing declare module 'mysql';
4 import * as mysql from "mysql";
Is obviously trying to find the types of mysql in Ceberus, but they dont exists, why?
Edit 1: typo
#unicoderns/orm lists #types/mysql in its devDependencies, so #types/mysql gets installed when you run npm install directly in #unicoderns/orm, but not when you run npm install in another package that depends on #unicoderns/orm (such as #unicoderns/cerberus). If #unicoderns/orm is going to have a declaration file such as connection.d.ts that references declarations from #types/mysql and is intended to be loaded by dependent packages, then the recommendation is to put #types/mysql in the regular dependencies of #unicoderns/orm to avoid the problem you saw.
With the release of npm#5, it will now write a package-lock.json unless a npm-shrinkwrap.json already exists.
I installed npm#5 globally via:
npm install npm#5 -g
And now, if a npm-shrinkwrap.json is found during:
npm install
a warning will be printed:
npm WARN read-shrinkwrap This version of npm
is compatible with lockfileVersion#1,
but npm-shrinkwrap.json was generated for lockfileVersion#0.
I'll try to do my best with it!
So my take-away is that I should replace the shrinkwrap with the package-lock.json.
Yet why is there a new format for it? What can the package-lock.json do that the npm-shrinkwrap.json cannot?
The files have exactly the same content, but there are a handful of differences in how npm handles them, most of which are noted on the docs pages for package-lock.json and npm-shrinkwrap.json:
package-lock.json is never published to npm, whereas npm-shrinkwrap is by default
package-lock.json files that are not in the top-level package are ignored, but shrinkwrap files belonging to dependencies are respected
npm-shrinkwrap.json is backwards-compatible with npm versions 2, 3, and 4, whereas package-lock.json is only recognized by npm 5+
You can convert an existing package-lock.json to an npm-shrinkwrap.json by running npm shrinkwrap.
Thus:
If you are not publishing your package to npm, the choice between these two files is of little consequence. You may wish to use package-lock.json because it is the default and its name is clearer to npm beginners; alternatively, you may wish to use npm-shrinkwrap.json for backwards compatibility with npm 2-4 if it is difficult for you to ensure everyone on your development team is on npm 5+. (Note that npm 5 was released on 25th May 2017; backwards compatibility will become less and less important the further we get from that date, as most people will eventually upgrade.)
If you are publishing your package to npm, you have a choice between:
using a package-lock.json to record exactly which versions of dependencies you installed, but allowing people installing your package to use any version of the dependencies that is compatible with the version ranges dictated by your package.json, or
using an npm-shrinkwrap.json to guarantee that everyone who installs your package gets exactly the same version of all dependencies
The official view described in the docs is that option 1 should be used for libraries (presumably in order to reduce the amount of package duplication caused when lots of a package's dependencies all depend on slightly different versions of the same secondary dependency), but that option 2 might be reasonable for executables that are going to be installed globally.
Explanation from NPM Developer:
The idea is definitely for package-lock.json to be the Latest and
Greatest in shrinkwrap technology, and npm-shrinkwrap.json to be
reserved for those precious few folks out there who care very much
about their libraries having an exact node_modules -- and for people
who want CI using npm#>=2 to install a particular tree without having
to bump its npm version.
The new lockfile ("package-lock.json") shares basically all of the
same code, the exact same format as npm-shrinkwrap (you can rename
them between one another!). It's also something the community seems to
understand: "it has a lockfile" seems to click so much faster with
people. Finally, having a new file meant that we could have relatively
low-risk backwards-compat with shrinkwrap without having to do weird
things like allow-publication mentioned in the parent post.
I think the idea was to have --save and shrinkwrap happen by default but avoid any potential issues with a shrinkwrap happening where it wasn't wanted. So, they just gave it a new file name to avoid any conflicts. Someone from npm explained it more thoroughly here:
https://www.reddit.com/r/javascript/comments/6dgnnq/npm_v500_released_save_by_default_lockfile_better/di3mjuk/
The relevant quote:
npm publishes most files in your source directory by default, and
people have been publishing shrinkwraps for years. We didn't want to
break compatibility. With --save and shrinkwrap by default, there was
a great risk of it accidentally making it in and propagating through
the registry, and basically render our ability to update deps and
dedupe... null.
So we chose a new name. And we chose a new name kind of all of a
sudden. The new lockfile shares basically all of the same code, the
exact same format
package-lock.json versions are guaranteed with only npm ci (since npm install overwrites package-lock.json if there is a conflict with package.json).
npm-shrinkwrap.json versions are guaranteed with both npm ci and npm install.
Editor's note: The question's original title was "Use npm install to install node modules stored on a local directory", which made the desire to transparently redefine the installation source less obvious. Therefore, some existing answers suggest solutions based on modifying the installation process.
I know this is a simple thing, but I'm quite new to anything in this area so after searching around and constantly finding answers that weren't really what I want I figured I'd just ask directly.
I currently have a process that is run in directory FOO that calls npm install. Directory FOO contains a package.json and a npm-shrinkwrap.json file to specify the modules (bluebird, extend, and mysql in this case but it doesn't really matter) and versions. This all works perfectly fine.
But now instead of reaching out to the internet to get the modules I want to have them stored in local directory BAR and have the process in foo use npm to install them from there. I can not store them permanently in FOO but I can in BAR for reasons outside my control. I know this is relatively simple but I can't seem to get the right set of commands down. Thanks for the help.
Note: This answer originally suggested only redefining the cache location. While that works in principle, npm still tries to contact the network for each package, causing excessive delays.
I assume your intent is to transparently change the installation source: in other words: you don't want to change your package, you simply want to call npm install as before, but have the packages be installed from your custom filesystem location, offline (without the need for an Internet connection).
There are two pieces to the puzzle:
Redefine npm's cache filesystem location (where previously downloaded packages are cached) to point to your custom location:
Note that cached packages are stored in a specific way: the package.json file is stored in subfolder package, and the zipped package as a whole as package.tgz. It's easiest to copy packages from your existing cache to your custom location, or to simply install additionally needed ones while you have an Internet connection, which caches them automatically.
For transparent use (npm install can be called as usual):
By setting the configuration item globally:
npm config set cache '/path/to/BAR'
Note that this will take effect for all npm operations, persistently.
Via an environment variable (which can be scoped to a script or even a single command):
export npm_config_cache='/path/to/BAR'
npm_config_cache='path/to/BAR' npm install
Ad-hoc use, via a command-line option:
npm install --cache /path/to/BAR
Force npm to use cached packages:
Currently, that requires a workaround via the cache-min configuration item.
A more direct feature, such as via an --offline switch has been a feature request for years - see https://github.com/npm/npm/issues/2568
The trick is to set cache-min to a very high value, so that all packages in the cache are considered fresh and served from there:
For transparent use (npm install can be called as usual):
By setting the configuration item globally:
npm config set cache-min 9999999999
Note that this will take effect for all npm operations, persistently.
Via an environment variable (which can be scoped to a script or even a single command):
export npm_config_cache_min=9999999999
npm_config_cache_min=9999999999 npm install
Ad-hoc use, via a command-line option:
npm install --cache-min 9999999999
Assuming you've set cache-min globally or through an environment variable,
running npm install should now serve the packages straight from your custom cache location.
Caveats:
This assumes that all packages your npm install needs are available in your custom location; trying to install a package that isn't in the cache will obviously fail without an Internet connection.
Conversely, if you do have Internet access but want to prevent npm from using it to fetch packages - which it still will attempt if a package is not found in the cache - you must change the registry configuration item to something invalid so as to force the online installation attempt to fail; e.g.:
export npm_config_registry=http://example.org
Note that the URL must exist to avoid delays while npm tries to connect to it; while you could set the value to something syntactically invalid (e.g., none), npm will then issue a warning on every use.
Sample bash script:
#!/usr/bin/env bash
# Set environment variables that set npm configuration items to:
# - redefine the location of the cache folder
# - make npm look in the cache only (assuming the packages are there)
# Note that by doing this inside a script the changes will only take effect
# in the script and NOT persist.
export npm_config_cache='/path/to/BAR' npm_config_cache_min=9999999999
# Now cd to your package and invoke `npm install` as usual.
cd '/path/to/project'
npm install
You might want to try npm link. You could:
download the dependency
run npm link from the dependency's directory
run npm link mycrazydependency from you project
Detail here: https://docs.npmjs.com/cli/link
If a shrink wrap file is present then package.json is ignored. What you need to do is change the URL they are being retrieved from using a find and replace operation like sed .... However I'm not sure changing the URL to a file:/// syntax is valid but give it a go.
Normally a bower.json file specifies some dependencies, but these are typically expressed so that they allow a range of versions of a bower package to be used (e.g. >=1.0, which means anything higher than version 1.0).
I have an automated process which needs to find what version of a bower package is actually installed on this system right now.
How can I find this out programmatically (just the version itself), ideally using standard Unix command line tools / the bower command?
bower info <thepackagename> does not show this - it shows information about what is currently available from the bower repository (for example, even if I do bower info apackageIdonthaveinstalled it will still show a valid JSON structure containing a version number).
cat bower_components/thepackagename/bower.json | node_modules/json/lib/json.js version works for some packages (assuming the npm package json is installed), but not all (e.g. jquery 2.2.0's bower package does not contain a bower.json).
Here's a grep command to do that:
grep "version\"\:" bower_components/thepackagename/.bower.json
Also, a command to see versions of all bower components for the project - this list can be a handy CI artefact:
grep "version\"\:" bower_components/*/.bower.json
Have you ever tried "bower list --json=0 --offline".
It would list all bower packages info.
The best approach I've now found, which seems to work for every package I've come across so far, is:
cat bower_components/thepackagename/.bower.json | node_modules/json/lib/json.js version
(note the extra . in .bower.json).
It would appear that bower stores some metadata about the installed package in .bower.json, and that includes the installed version.
The best I've come up with so far is:
bower list | grep jquery | perl -pe 's/.*jquery#(.*?) .*$/$1/'
(if, for example, the package I was interested in was jquery).
That's pretty ugly for a variety of reasons:
I have to repeat the package name (although this could probably be improved
with a better Perl script which filters lines too, I'm just being lazy).
bower list gets information about all installed packages, not just the one I'm interested in - the rest of the information is discarded.
bower list seems to require internet connectivity to check the registry, otherwise it fails.
Would be interested to see if this could be improved upon, particularly the last point.