I'm deploying a gcloud function with two mounted secrets (from google secret manager), my local dir structure is the following:
├── index.js
├── mounted-secret-config
│ ├── config.js
├── mounted-secret-credentials
│ └── googleServiceAccountCredentials.json
├── package-lock.json
└── package.json
config.js and googleServiceAccountCredentials.json are ignored so the deploy process doesn't upload them.
I deploy using this command:
gcloud functions deploy <...> --region <...> --trigger-http --runtime nodejs16 --allow-unauthenticated --gen2 --memory 256Mi --set-secrets=/workspace/mounted-secret-config/config.js=configjs:latest,/workspace/mounted-secret-credentials/googleServiceAccountCredentials.json=googleServiceAccountCredentials:latest
It works, the node app finds the files and overall works but after each deploy I see this in the gcloud logs:
2022-08-26 10:11:18.130 CEST Could not open file at path /secret_volume_0/config. The path is in a mounted secrets volume, but the exact path does not correspond to any secret specified in the mount configuration.
Warning
2022-08-26 10:11:18.182 CEST Could not open file at path /secret_volume_0/package.json. The path is in a mounted secrets volume, but the exact path does not correspond to any secret specified in the mount configuration.
Warning
And after each http request to my service i get:
2022-08-26 10:05:33.511 CEST Could not open file at path /workspace/mounted-secret-config/config. The path is in a mounted secrets volume, but the exact path does not correspond to any secret specified in the mount configuration.
Warning
2022-08-26 10:05:33.572 CEST Could not open file at path /workspace/mounted-secret-config/package.json. The path is in a mounted secrets volume, but the exact path does not correspond to any secret specified in the mount configuration.
I've no idea what's going on here, I don't even know who's logging this. /workspace/mounted-secret-config/config doesn't exist, but /workspace/mounted-secret-config/config.js (note the .js extension) does, and the app finds it or it would not even start. /workspace/mounted-secret-config/package.json this doesn't but it isn't supposed to, who's even trying to access it? And why it doesn't complain about the other mounted secret?
config.js is required with: require('./mounted-secret-config/config')
If I change it to require('./mounted-secret-config/config.js') (adding .js) one of the two warnings disappears. Is node trying to import the exact name (giving the warning) and then falling back to config.js? But what about the package.json?
This is not a real solution, more like a workaround, feel free to add a real solution if you find it.
What I did to "fix" the problem was change the format of my config file from .js to .json (easy in my case because it only contained a dictionary with few keys/values)
Everything else stayed the same, I still read the file with require. I don't know what weird thing node was doing with the .js, but it doesn't do it with .json.
Both warning disappeared (including the one about package.json).
Related
What's the difference between Mutable File System's (MFS) read:
await ipfs.files.read('/some/stuff/success.txt')
and Regular Files API's cat:
await ifps.cat(someFile)
Is it the fact that the cat method first searches your own node for the file requested, and if it can't find it there, it will attempt to find it on the broader IPFS network?
The parameter for cat is an IPFS Path, not a file path. An IPFS Path is basically all the variations on a content identifier.
cat will fetch the data from the local repo if it exists there, or ask the network if it does not.
files.read accepts an MFS path, which is a file path for a file you've already added to your IPFS repo - as well as an IPFS Path or CID object. If you pass an MFS path, a file or directory at that path must already be in the local repo already or the API will not find anything.
The documentation for files.read:
https://github.com/ipfs/js-ipfs/blob/master/docs/core-api/FILES.md#filesread
The documentation for cat:
https://github.com/ipfs/js-ipfs/blob/master/docs/core-api/FILES.md#cat
A good tutorial on the Regular Files API is here:
https://proto.school/#/regular-files-api/04
And one for MFS and file.read:
https://proto.school/#/mutable-file-system/10
And a tutorial on CIDs (content identifiers) is here:
https://proto.school/#/data-structures/04
According to https://docs.ipfs.io/guides/concepts/pinning/ , running the command ipfs add hello.txt apparently "pins" the file "hello.txt", yet why don't I see the file listed afterwards when I run the command ipfs files ls? It only lists files I added with the IPFS desktop app. Why is "hello.txt" not in the list now?
Also, I found a list of so-called "pinned" objects, by running the command ipfs pin ls, however none of the CID's that show up there correspond to "hello.txt", or even any of the previously mentioned files added using the IPFS desktop app.
How does one actually manage pinned files?
cool to see some questions about IPFS pop up here! :)
So, there are two different things:
Pins
Files/Folders (Called MFS)
They both overlap heavily, but it's best to describe that the MFS is basically a locally alterable filesystem with a mapping of 'objects' as files and folders.
You have a root ( / ) in your local IPFS client, where you can put files and folders.
For example you can add a folder recursively:
ipfs add -r --recursive /path/to/folder
You get a CID (content ID) back. This content ID represents the folder, all its files and all the file structure as a non-modifiable data structure.
This folder can be mapped to a name in your local root:
ipfs files cp /ipfs/<CID> /<foldername>
A ipfs files ls will now show this folder by name, while an ipfs pin ls --type=recursive will show the content-ID as pinned.
If you use the (Web)GUI, files will show up under the 'files' tab, while the pins show up under the 'pins' tab.
Just a side note, you don't have to pin a file or folder stored in your MFS, everything stored there will be permanently available.
If you going to change the folders, subfolders, files, etc in your MFS, the folder will get a different Content-ID and your pin will still make sure the old version is held on your client.
So if you add another file to your folder, by something like cat /path/to/file | ipfs files write --create /folder/<newfilename>, the CID of your folder will be different.
Compare ipfs files stat --hash /folder and afterwards again.
Hope I didn't fully confuse you :D
Best regards
Ruben
Answer:ipfs pin ls --type recursive
It's simple. Just run that command.
Some further notes: the type can be "direct", "recursive", "indirect", and "all". I ran these commands with these results ("Error: context canceled" means that I canceled the command with ctrl+c):
ipfs pin ls --type all - took too long, "Error: context canceled"
ipfs pin ls --type direct - took too long, "Error: context canceled"
ipfs pin ls --type indirect - took too long, "Error: context canceled"
ipfs pin ls --type recursive - worked, showed multiple, probably all, pins of mine
I don't really know what types other than recursive mean. You can read about them from the output of this command: ipfs pin ls --help.
How can I uninstall IPFS completely and restart everything from scratch and get a new peer id? I tried to delete the go-ipfs folder but I can still get Error: ipfs configuration file already exists! when I do ipfs init.
The data store as well as the config will be stored in a subdirectory .ipfs of your home directory. So if you are on a UNIX based system $HOME/.ipfs. You would have to delete this directory and then run ipfs init to get an empty store and a new peer id.
Note that you can also configure the location of the store directory using the IPFS_PATH environment variable, which can be useful to get the IPFS store on a different mount point.
I already install ipfs via go ipfs. and I don't know why I encounter an error when I want to run "ipfs init " in my Terminal.
could anyone help me to figure out where is the problem?
here is an image of my Terminal:
The default IPFS repository path is ~/.ipfs.
Perhaps you can try to change your default path using:
export IPFS_PATH=/path/to/ipfsrepo
And then run:
ipfs init
Also, have you by any chance installed go-ipfs from snap?
In my case, I have faced this same problem. Because I also installed IPFS Desktop, it initialized some configurations. If you want a fresh start, you delete the initialized IPFS node:
> /Users/scbas/.ipfs
If you have not made any directory for IPFS, then using
ipfs init
will automatically generate a node in your base Users/Username directory.
If you want to change the directory, you can create one and then change the path using this command in the terminal according to the directory you created:
export IPFS_PATH=/Users/<your system username>/<the name of directory you created>
For example, I created an ipfs-repo folder for my IPFS node, so I have to run:
export IPFS_PATH=/Users/jonah/ipfs-repo
I already know that there is a simple command to serve static folder by PM2:
pm2 serve <path> <port>
But how can I add 404 redirection to it? e.g. redirect to 404.html while that happends. Can't find that on Google & PM2 doc.
By default, pm2 displays 404.html from the serving directory if it can't map a request to any static file of that directory. For example, assume that your static file directory name is foo, which contains three images - 1.jpg, 2.jpg, 3.jpg. You are serving those files using:
pm2 serve <path_of_foo> 8080
Now, if the server receives any request like http://localhost:8080/bar.jpg, pm2 will look for a file named 404.html on foo directory and display its contents, as bar.jpg does not exist. If it doesn't find any, it show a simple text message 404 Not Found. There is no options to make it configurable (CLI or JSON), as per my knowledge.
However, if you make this file path configurable, you can take a look at here and customize the source according to your need.