html5 to store greater than 50 mb worth of data? - html

I am building a mobile app that must work in Blackberry, iPhone and Android phones. I am building this mobile app in HTML5, CSS and JQuery Mobile. Then I will use PhoneGap to allow the code to run as a native mobile app for the phone. The mobile app must be able to work without connection to the internet. The only time the app must require an internet connection is when it loads for the first time, because that's when the app will download over 50 MB worth of data. Next time the app has internet connectivity, it will poll the server for more recent data to consume.
My problem lies in where do I store 50 mb worth of data which can be updated when internet connectivity exists?
Additional Notes
I've been discouraged by others to use Web SQL Database because the specification is no longer in active maintenance. I've read that localStorage and cache manifest has a storage limit of about 3-5 mb depending on the browser. From experience, I've encountered a lot of challenges working with the PhoneGap API to create one set of code that will reliably write to all of the following file systems: ios, bb, android.

Based on your needs: (store 10,000 images, 10,000 pdfs, +videos etc.) I don't think a "lightweight" HTML5 application is going to suit your needs.
I think that a native application is likely going to work much better where you have proper filesystem access to store the files on the device as needed (and even structure them better).
The HTML5 localStorage (5mb) and the WebSQL (SQLLite) (5mb-~50mb) database options really won't give you what you need as far as I can tell.
WebApp "wrappers" like PhoneGap and WebWorks (for BlackBerry) will give you the filesystem access you need, but I'm not sure if either of them have limitations that you'll hit size/quantity wise... hopefully someone with experience in these wrapper frameworks can indicate any size limits they've encountered.

Related

PWA vs html5 webapp?

I know that the most Iconic feature of PWA are
Service Worker: which make user can use the app offline from cached resources
Add To Home Screen: With this feature, user can add a shortcut of the app on their mobile home screen, to get a experience like interacting with a Native App(But there still a huge different, in my opinion)
etc.
However, I can do like all of them on about 6/7 years ago by using the HTML5 technology at that time, I know that service worker comes recently but there also was "HTML5 App Cache, as well as the Local Storage, Indexed DB, and the File API specifications." can do similar things.
Is there anyone can explain what's the difference between PWA and HTML5 webapps? or they are just the in term of same? or a similar concept but different implementation? or PWA is the (next gen/extension) of HTML5 webapp?
I might have some misunderstanding on PWAs, since I am new to this term, Thanks.
If keep it simple PWA is ordinary site with 3 additional features.
responsive design - site should look well on all devices
manifest.json - site must have general description about itself stored in manifest.json
caching - site must work offline
I believe 1 and 2 is easey. And 3 is realy what PWA is all about. So the question is: how we can provide offline support?
First problem is how we can get our initial .html .css and .js files without internet connection? Answer is: we should use service worker or App Cache. But App Cache has a lot of problems and probably will be deprecated, in other hand service worker under developement and become better each month. You can read more about their difference here.
Second problem is how we can get server data without internet connection? We should store most vital data somwhere. But after we get our .js file from cache we have access to Local Storage, Indexed DB etc. So we can store vital data in any of this storage while we online and get it back from them when we offline. It is totally up to you how you will handle that.
I believe there no cleare and strict defentiton for PWA and HTML5 webapp (at least for now). So you can asume it is the same but today PWA is more common word.

Should I use PhoneGap even if I do not need to use any of its features such as the app stores or native device APIs

I am building an application with the following requirements:
Must run on both mobile phones and tablets
Must support online/offline scenarios
Requires geolocation
Requires local storage
Platform is unknown at the moment (due to the nature of this project, I cannot yet rely on a particular platform version, e.g. Android 4+ or Windows 8.1)
I am leaning towards building this in HTML5/CSS/jQuery/Bootstrap. Since HTML5 supports both geolocation and local storage, and since I do not need to use any device feature such as a camera, I don't think something like PhoneGap is necessary.
However, as I am new to mobile development and PhoneGap, I am not sure if I am missing something. Are there any reasons - apart from future-proofing against future requirements - why I should consider using PhoneGap even though I don't think I need it for any particular reason? Is there value in having its API sitting in front of any interaction with the devices upon which my application will be running?
Factors to consider with respect to Local Storage.
Local Storage size limit:
Web storage provides far greater storage capacity (5 MB per origin in Google Chrome,Mozilla Firefox,and Opera; 10 MB per storage area in Internet Explorer;25MB per origin on BlackBerry 10 devices) compared to 4 kB (around 1000 times less space) available to cookies. ( source Wiki)
Local Storage Method.
Web Storage:
Web SQL Database:
Indexed Database:
File Access:
source: html5rocks, sitepoint
All MOBILE broswers have varying degree of support for above local storage method.
The most widely supported method is Web Storage.
If you would like to store more than the limits mentioned above than you need to use the native (OS) features for storing data. Then PhoneGap is the obvious choice for accessing native features.
How are you planning to distribute your app!! If you want to distribute using a website, then may be you don't need phonegap. BUT..... if you want it as an mobile app, then you'll need phonegap to easily wrap your code. Phonegap will help you to build installation package for many platforms (e.g. APK for android)

How to store 40.000 images using HTML5 or Phonegap?

I have searched a long time for a way to store a large number of images in HTML5, or access them stored locally and display them.
The application is a product ordering and catalog, like an e-commerce web site but for bulk orders, not final clients, each image is a product, that is why there is a need to have 40k images. The need comes from the requirement that the salesperson can use the tablet in offline mode for days, weeks, and only sync with the ERP once he has connection or wants to sync.
Each salesperson has a dedicated Android tablet Tegra3 Quad-Core, 32Gb for the application. But the application also supports iPad and Chrome on the desktop.
However I have faced the following limitations:
Use html5 offline (app cache) - This is limited in iOS devices to
50Mb, and hard to manage the manifest file.
Use webSQL database with base64 encoded images. Same size limitations (50Mb).
Use localCache with base64, again size limitations (5 to 10Mb).
Convert my HTML5 app to use PhoneGap File API, but then I lose desktop Support for Windows PCs, and have some issues with web services.
So perhaps I am missing an alternative way to do this?
Is there an HTML5 way to store a large number of images for offline viewing? FileSystem API is not supported in mobile devices and being able to run on mobile devices is a strong requirement.
Shouldn't Phonegap's own Implementation of the Filesystem API work on mobile? I mean that's why Phonegap was created in the first place.
Straight from the Cordova API Docs for FileReader:
Supported Platforms
Android
BlackBerry WebWorks (OS 5.0 and higher)
iOS
Windows Phone 7 and 8
Windows 8
The solution is going to pass to a port of the HTML5 application to a phoneGap application using the File API.
HTML5 different implementations is the reason for this decision. Since AppCache is very limited in most mobile browsers, as well as the lack of support of the FileSystem API. The fact that you can only store from 16-52Mb on mobile browsers is a limiting factor to a HTML5 web app that requires large amounts of locally stored data (available offline).
Random though, sprite sheet? would make just one file and would minimize requests
How about running a local webserver that can only accept local connections? In Apache, somthing like "Allow from 127.0.0.1" in the configuration file should do the trick. Using a local webserver wold let you sidestep any HTML limitations.
You can simply store it in your sqlite database. You can add one more field (column) to your product detail table.
You can store image either in BLOB format OR in base64 format. (in your database).
using phoneGap you can easily access your database and different tables from your DB.
For more performance and to save bandwidth :
What you can do is you can apply lazy loading of images. When ever you download new image just store it into database.
This way you can avoid the I/O Operations and also if you are using MVC then in your product detail object (Modal class) you will have image object also.
hope this suggestion works for you :)

How to get AppEngine data to GWT client, only once

I have a 6MB binary file in my AppEngine backed app. I need to get this file to my GWT client to do many computations. I'm trying to avoid doing the computations on the server to save my instance hours quota. But this means I have to get the file to my client somehow.
I also don't want to download it to the app each time the app is opened, that would hurt my appengine outgoing bandwidth quota.
It would be nice to download it once, then have the client somehow store it for use the next time the app is opened.
How to best do this?
A ClientBundle? Use HTML5 Storage? Is this sort of thing just not done, and should I just do the computations on the server and send the result to the client?
The most easiest way to permanently cache the file is using application cache.
Another ways is HTML5 Storage (localStorage), but there is 2.5 MB limit. To use more than that amount, you have use IndexedDB (Chrome, Firefox, IE10) and WebSQL (Safari, Opera). A wrapper library like YDN-DB help in this case.
You can use DataResource:
https://developers.google.com/web-toolkit/doc/latest/DevGuideClientBundle#DataResource
The only problem you face is the file size. Different browsers have different limits - I've seen 5MB and 8MB numbers. If you can compress your file to be under 5MB, you should be fine with most browsers. (By the way, it's an enormous amount of data if it can't be compressed any further.)
Also, you can detect a mobile browser (or offer a different URL for mobile version), and do computations on the server for mobile users.

Best way to implement offline storage for web application (work on all browser + tablets )

I wonder about the best way to implement offline possiblities for a web application. I have searched the web but couldn't figured out what currently the best way is to create a web application offline (and will work in the future)
The lay-out of the web application i can save with Offline Application Caching...
But the problem is that i have a lot of key-value data that i need to store offline. With localStorage, sessionStorage the limit to save data offline is 5MB...And that's not enough...
I seach for the following things:
Gears from Google: not supported anymore
Web SQl database: not supported anymore
IndexedDb HTML5: not supported by all browsers (only chrome and firefox). I think this is currently the best way...
Is it possible to increase the Offline Application Caching and save the key-value data in a text format witch offline application caching??
https://github.com/marcuswestin/store.js is mature (~2 years), still updated and works in an amazing array of browsers.