How to locate data in distributed network which is based on IPFS protocol if I don't know the hash value of that data.
I am new to IPFS. I know a little bit about IPFS and how it works. I came to know that IPFS is protocol which is content-addressed and user can retrieve the data by specifying the hash value of that data.
In a distributed network how will the 2nd user will be knowing the hash value of some data that 1st user added to the IPFS.
I went through some resources and IPFS site and came to know that it works on Distributed Hash Tables. But I am still not clear.
Please help me with the topic.
Related
In my project, I need to download data from ipfs by giving a CID.
What I do is:
ipfs pin add {CID}
ipfs get {CID}
But I found these two steps are quite time-consuming, it takes at least 1min above.
I tried localhost and infura.
What can I do to let it download faster?
When you just want to download files, there is no need to pin them first. This might save a (tiny) bit of overhead.
However, the bulk time is probably spent in
Looking up nodes in the distributed hash table that provide your data, and
Actually transferring the data from these nodes.
For small data sizes, the first item is probably the limiting factor. In general, the duration depends on the number of nodes that host your connects to and how fast these nodes (can) transfer the data to your client.
I'm looking to store data temporarily on IPFS, probably using a JS library, may or may not host an IPFS node. The technology is so new it isn't easy to locate answers. Please, I appreciate your help, especially in both cases of hosted and non-hosted solutions. Thank you.
Data is only stored persistently on nodes that have "pinned" the content hash. Nodes that request data will cache it for an indeterminate amount of time (as the data is immutable and referenced by hash, the cache would always be valid).
Once you are finished with the data you could remove it from your node, and as long as no one else requested it it would gradually disappear from the network. You could not rely on that happening (for instance if you have a requirement that the data be inaccessible after a certain amount of time).
You would need to be running your own node to initially host the file
This is more of a security question regarding using AES_ENCRYPT to generate encryption on data being inserted into a MySQL database.
What is the best location in which to store the key that is used to encrypt the data ? Obviously not in the database ! :)
Well, you don't have many options. Wherever you put that key (database, code, file), it is easily found as long as other people has access to the machine.
What you could do is that you encrypt that key with another key based on some password (which is not stored anywhere, at least not locally) and ask for that password on the startup of the application. This way, you can store encrypted AES_ENCRYPT key into your database, decrypt it after logging in with your password and start using it.
Security by obscurity!
If your webserver is compromised, then the attacker can access the key, no matter where it's stored - since the code must be able to find the key to do the encryption/decryption - and the code explains where it finds the key. The only scenario where this adds real value is in protecting the data outside of the application (e.g. on a backup tape). However since you're compromising the DBMs's ability to optimize queries and creating a much bigger data footprint, for such a purpose as a backup, it makes a lot more sense to encrypt the backup or the filesystem - not individual data items.
Even if you use keys which are not permanently stored within your application (e.g. a HTTP basic authentication password supplied over SSL) there are still a lot of risks that the data will be compromised - and you've got problems with sharing data between different users.
In order to provide a sensible answer we need to know what the threat model is and whether you have external constraints such as PCI-DSS
The issues of securely storing keys and passwords used in your PHP / Python / other application on a server is not only related to hiding the keys from an attacker who has gained root on your sever, although you can make it more difficult for an attacker who has gained root to access them, it can eventually be done.
However, keys / passwords can be lost in many other ways and so must be protected. For example, if your software is being updated from a development environment, i.e. being pushed and pulled through a git server, you do not want the keys to be included in plain-text in the source code. That would give anyone on your development team access to them.
One option to store keys "more securely" is to have them configured as environment variables and then include them in your application by accessing that environment variable instead of having the key in "plain-text" within your application.
However, this requires that you set the environment variable to be persistent so that if you reboot the sever it will automatically be set again, or else you must set it each time you reboot.
If you are using Apache web-server, you may also set Apache environment variables for sensitive keys / passwords in the httpd.conf file, and then access them from your PHP script. You can also restrict the permissions on the httpd.conf file for only root to have read/write.
// Example use of getenv()
$sensitive_key = getenv("VERY_SENSITIVE_KEY");
// Example use of apache_getenv()
$sensitive_key = apache_getenv("VERY_SENSITIVE_KEY");
This means that the key / password is not included in the application source code itself, and will be less likely to escape the server.
I'm about to add a feature on my website that will post stuff to the users' twitter accounts. I read that storing an "OAuth" token is better than storing their usernames and passwords (which makes sense).
What should I store in my MySQL database? The token, secret and username? Or just the token?
What data type(s) would you use? How big are they?
Thanks!
They're not very big (maybe 15 or 20 chars, can't remember exactly), and are strings. Note: this could change. Nothing in the OAuth spec describes how to generate tokens and secrets, nor what they look like. I expect they'll always be strings, but I might want to make the column a little bigger than currently necessary for contingency.
If you store them in the database, you need some other way of authenticating the users so that you know whose tokens to pull out and use so they can't be used maliciously. If you already have that and it's working then that makes sense, otherwise you might want to rethink it.
Another way to do it is to store their tokens in a cookie, the drawbacks of that are that (if you don't encrypt them or use SSL) they are traveling over the net in the clear, and if they change computers, browsers, or delete their cookies you have to make them go through the whole OAuth process again.
For a website with no other auth, I would encrypt them and store them in a cookie. If you already make them log in, then associate both the token and secret with the login and store them in the database.
There are other ways, too. You could use a randomly generated ID like a GUID/UUID in a cookie to point to a record in a database with the tokens if you didn't want to use encryption. All in all, it depends on how you want your application to behave from the users' point of view.
Make sure you handle the case where the user de-authorizes your app too.
The application secret and token, is specific to your application and that can be stored in database or some config file.
User token you should store in to databse with varchar (100) (at max) along with his twitter ID. you can get the twitter ID while getting the other details from twitter OAuth.
For further guidance, following this link
http://www.9lessons.info/2010/02/connect-twitter-api-with-oauth-using.html
I have a web application that uses a symmetric encryption algorithm.
How would you store the secret key and initialization vector? Storing as a literal in the code seems like a bad idea. How about app settings? What is the best practice here?
One standard approach in the webapp world is to split the key and put it in different places. E.g., you might split the key and put part of it in the filesystem (outside of the 'webapps' directory), part of it in the JNDI configuration (or .net equivalent), and part of it in the database. Getting any single piece isn't particularly hard if you're compromised, e.g., examining backup media or SQL injection, but getting all of the pieces will require a lot more work.
You can split a key by XOR-ing it with random numbers of the same size. (Use a cryptographically strong random number generator!) You can repeat this process several times if you want to split the key into multiple pieces. At the end of the process you want, e.g., three partial keys such that p1 ^ p2 ^ p3 = key. You might need to base64-encode some of the partial keys so they can be stored properly, e.g., in a JNDI property.
(There are more sophisticated ways to split a key, e.g., an n-of-m algorithm where you don't require all of the pieces to recreate the key, but that's -far- beyond what you need here.)
If you can require the user to actively enter the password, there are PBE (password-based encryption) algorithms that convert a password to a good symmetric key. You want to find one that requires an external file as well. Again it's a case the tape backups or the password itself isn't enough, you need both. You could also use this to split the password into two pieces with JNDI - you can use a plaintext passphrase in JNDI and an initialization file somewhere in the filesystem.
Finally, whatever you do be sure you can 'rekey' your application fairly easily. One approach is to use the password obtained above to decrypt another file that contains the actual encryption key. This makes it easy to change the password if you think it's been compromised without requiring a massive reencryption of all of the data - just reencrypt your actual key.
Is it possible for you to enter a password interactively whenever the application starts up? That way you don't have to store the key, or at least any keys (whether they are symmetric or private keys) can be encrypted with this "bootstrap" password.
If not, store your secret key in a file by itself and modify its permissions to make it accessible only to the user running the web application.
These approaches are platform-agnostic. For more concrete suggestions, information about your platform would be helpful.
By the way, an initialization vector should be used for only one message. And IVs do not have be kept secret, so you could store it anywhere, but storing it with the one message that uses it is customary.
I have used an approach where my application requires a symmetric key when it starts and looks for it in a certain file. Once the application has started up I remove the file. A copy of the file is kept remotely for any required restarts. Obviously this approach is not viable if your applciation has frequent restarts.
Another alternative would be a certificate manager such as the Windows Certificate Store. It can store certificates and their keys securely and it's also possible to mark private keys as non-exportable so it would require some serious hacking to get the key out. Your application could load its certificate from the Certificate Store and be able to call operations to sign requests or generate new symmetric keys. In addition you can assign permissions to different certifcate stores so that only certain privileged accounts would be able to access the certificate.
stick it in the web.config and encrypt that section
This SO question talks more about web.config encryption
This should help ...
http://msdn.microsoft.com/en-us/library/ms998280.aspx
But, you really should consider going to PKI if you are serious about protecting your data.
We have a slightly different, but related issue. We have keys generated every few days, and when decrypting, we have to try all our keys because we do not know which day the encryption took place. What we did was to encrypt the keys one more time and store them as secretes. This way, we only have one set of keys to manage.
For secure storing of encryption key you can use KMS service of AWS. Please use this service for storing such confidential keys. PFB url for kms service.
documentation : https://aws.amazon.com/kms/