The main purpose is to store data locally so it can be accessed without internet connection.
In my React application I will need to fetch JSON data (such as images, text and videos) from the internet and display it for a certain amount of time.
To add flexibility, this should work offline as well.
I've read about options such as localStorage and Firebase but all of them so far require either access to the Internet, or are limited to 10Mb which is too low for what I'll need.
What would be my best option to persist data in some sort of offline
database or file trough react?
I'd also be thankful if you could point me to a good tutorial about
any provided solution.
To store large amounts of data on client side you can use indexedDB.
IndexedDB is a low-level API for client-side storage of significant amounts of structured data, including files/blobs.
You can read more about indexedDB api here
Related
Question: What is the best approach for implementing client caching of huge data. I am using Angular4 with Asp.net Web API2.
Problem: I am developing an web analytical tool (support mobile browsers as well) which generates echart metrics based on JSON data returned from Asp.net web api2. Web page have filters and chart events which recalculates charts measures based on same JSON data on client side. To optimize the speed, I have stored the JSON data(minified) in browser localstorage. With this I have avoiding frequent calls to api which are made on filter change and chart events. This JSON data is refreshed with server data at every 20 mins as I have set expiry for each JSON data saved on localstorage.
Problem is localstorage has a size constraint of 10mb and above solution does not work when JSON data (multiple localstorage keys) exceeds 10mb.
Since my data size can vary and can go more than 10mb. What is the best approach to cache such data, as same data can be used for recalculating measures for metrics without making web api server calls.
I though about (below but not implemented yet),
a) client- memory caching (may cause performance issues for users having less memory).
b) Storing json data as a javascript variable and using it.
Please let me know better solution for large client cache.
I am trying to store an array in localStorage, It's working for 3000 records, but when records increases to 5-10 thousand code breaks.
Is there is anything so I can store huge data in localStorage.
LocalStorage has size limits that vary depending on the browser. This is to prevent malicious scripts from filling a user's hard drive.
You can test your browser's localStorage limits here: https://arty.name/localstorage.html
The simple answer is, you shouldn't try to store more than 5MB-10MB of data on the client, depending on the browser. Needing to store that much local data is a sign that you probably need to come up with a better solution.
One other possibility for storing data locally is IndexedDB, which has reasonable compatibility across modern browsers. It's a object store which acts a lot like document databases such as MongoDB. You can store objects without converting them to strings and you can query those objects the way you would a database.
Most browsers seem to have a "soft" limit of around 5MB on IndexedDB storage. It's a soft limit because it's not necessarily enforced so you can go store much more if the browser allows it. Your mileage may vary.
Max size for localstorage is 5MB for preventing malicious scripts from filling a user's hard drive. You can go for IndexedDB which is compatible with all modern browsers. The minimum or soft limit is 5MB- the browser will ask for permission to store the data. Maximum storage is the limit of your hardrive disk, as all of the data is stored locally on your machine disk. Basically if you have 20GB free storage than you can use all of the storage for IndexedDB.
I have gone through many resources online but could not get the memory architecture used by HTML5 local storage. Is the data from local storage brought in memory while working over it (something like caching)?
Also in case I want to implement my app working in offline mode (basic purpose of storing into local storage), is it fine to store data as global JSON objects rather than going for local storage.
In short , I am getting a lot of JSON data while I login to my app(cross platform HTML5 app). Shall i store this data as global object or rather store it in memory.
Well, it depends on how sensitive is your information and the approach you want to follow.
Local storage
You can use local storage for "temporal" data, passing parameters and some config. values. AFAIK local storage should be used with care in the sense that the stored information is not ensure to be there always, as it could be deleted to reclaim some device memory or cleaning process. But you can use it without much fear.
To store JSON in local storage you will have to stringify your object to store it in a local storage key. JSON.stringify() function will do the trick for you.
So far I haven´t found official information, but I think there is a limit of MB that you can store in local storage, however I Think that is not controlled directly via cordova. Again, is not official data, just take that in mind if your data in JSON notation is extremely big.
Store data as global objects
Storing data as global objects could be useful if you have some variables or data that is shared across functions inside the app, to ease access. However, bear in mind that data stored in global variables could be lost if the app is re-started, stopped, crashed or quit.
If it is not sensitive information or you can recover it later, go ahead and use local storage or global variables.
Permanent storage
For sensitive data or more permanent information I will suggest to store your JSON data in the app file system. That is write your JSON data in a file and when required recover the information from the file and store it in a variable to access it, that way if your app is offline, or the app is re-started or quit, you can always recover the information from the file system. The only way to loose that data is if the app is deleted from the device.
In my case I am using the three methods in the app I am developing, so just decide which approach will work the best for you and your needs.
I am currently looking into making a basic video game.
I have programming knowledge in Web technology, but not much in Android so I am most likely going to start by making a Web app on Android.
I haven't used Web storage much in the past, but I do know about local storage and indexed db, which could be considered as a way to store data, but this kind of data seems too easy to delete by mistake because people just generally wipe out Web data once in a while without filtering out important data.
What is the most appropriate way to store Web app game data?
The storage type relies on the characteristics of the data.
If you want to store small amount of data in key-value standards, you can use Key-Value Sets.
If you want to store structured data, you should consider to use a SQLite Database.
You can also use the Internal Storage to keep files.
Since you want an approach more user-proof, I prefer the SQLite approach. Be aware that no method is 100% guaranteed, since rooted phones can grant access to physical data. At least, consider to encrypt sensitive data before sending to the database.
You also have the option of hosting a web service that is consumed by the app to gather information. It can be a lot safer, it is easier to be accessed by all instances of the application, but introduces internet dependecy.
Hope it helps.
I am working on a project in which I need sync RSS feed from the url given by user. Then show the feed content in our css on the user page. There is mobile site involve, so i dont want to sync again and again when user open up the mobile site for the performance reason. I need to keep the content in some storage. What can be the best storage for keeping such large data (Data can be large based on the RSS feed content). I am using the MySQL DBMS. Can I store this data in database or should I adopt file system or some other storage media is available for these types of data. What should be the best practice as the user database can be too large.
If using the database to store files is more convenient than the file system for your needs, you can definitely do it. Neither solution is absolutely better than the other, but MySQL offers some benefits over the file system:
If you have a large amount of files, you'll need to split them into separate directories is you use the file system.
MySQL allows replication to multiple servers if you need load balancing.
You don't need separate file retrieval code when you can get the files directly from the database with the same query you get the other results.
For very large amounts of files, you may want to look into distributed filesystems. I have used and liked MogileFS, but there are many others available as well. These allow you to distribute your files over as many servers as you like and are much more fault tolerant than the other solutions.