I'm struggling to find a solution to what I thought would be a common requirement so I'm hoping someone can help me with some pointers on what to search for/areas to explore.
Background
I'm building an iOS mobile app. I'm storing data locally using realm.io. The app is preinstalled with a snapshot of the content of a Wordpress mySQL database (it uses custom types). The content of the WP database is only written via the Wordpress install, the mobile app cannot write data.
Objective
So, I want to be able to check for changes since a given date (whenever the local database was last updated) and send the changed records to the mobile app (via the wp JSON api?).
I think I can fetch "posts since a date" but I need a full list of all create, update and delete operations since a given date.
Since the app is read-only I thought this type of one-way sync would be pretty straight forward but I can't find a common solution.
Any ideas to point me in the right direction would be great. Obviously, if anyone has any experience of doing this sort if thing with realm.io then that would be amazing :-)
Realm doesn't support yet any sort of synchronization mechanism across different files. We have an issue about that though, but you're likely searching rather for a solution in the immediate future.
Update: Realm launched the Realm Mobile Platform. This offers synchronization functionalities and would greatly simplify the solution for this use case.
You could use e.g. the server-side Node.js binding to pull new data from your MySQL Wordpress installation and push them to a global Realm served by the Realm Object Server. This can be read-only synchronized from the mobile apps, which would automatically receive the deltas and provide updated data to your users.
Whatever mechanism you come up yourself though in the meantime, it would require that you have read-write access from your iOS application to the realm database, so that you can update it with new data.
Pushing changed records as you describe is likely not going to work.
Apple's Push Notification service (APNS), which is the only back communication channel that works when your app is in the background or suspended, allows you to send very small payloads. You would use that to signalize your iOS app, that something changed on the server-side and there is new data to load. You would then initiate a request to a JSON-based API, wait for the response, map the returned JSON to Realm objects and store them in your database.
You want probably read more in the "Downloading Content in the Background" section of background execution chapter in the official App Programming Guides for iOS.
While pre-seeding the database from the app bundle seems like a nice idea, because the user wouldn't need to wait initially after downloading the app, that will enlarge the app itself with data, which might become in the future completely irrelevant.
Related
Can someone explain to me why you can't connect to a MySQL DB directly through dart from a security point of view?
There is no hard guideline on whether to connect frontend directly to backend or not. It is just a design practice that has been widely accepted and evolved over many years.
Typical app structure consists of
FRONTEND -> SOME MIDDLE LAYER -> BACKEND
Where your middle layer handles all the interactions/processing with the database and the frontend uses this functionality with some sort of API structure. Having this layer is extremely helpful when the application goes to scale, it gives an added abstraction to the frontend.
It is not advisable to directly fuse your frontend(your flutter app), to the DB(MySQL) because any efficient hacker might use basic man-in-middle attack to know your DB structure/connections/queries(There are some pretty effective decompilers present out there), and alter your data and you might not even get to know what caused the data to update unless you've applied some checks on DB layer.
Also, your frontend logic needs to be more of end-user centric than to handle the data of the user. Any backend system(java, node, etc) gives you added functionality & freedom to parse and present the data from either side.
You can use the sqlite package available to store basic data, like your session tokens, your app configurations etc, but it is advisable to keep the main user data like the logins, etc in a separate place, or better yet, you can use the firebase plugin to store data in document structure in the cloud.
I'm working on a project that can take data from a Weintek HMI, put them on a webserver and then send them to an application that I created on android studio.
I've found firebase that can help me in this task.
In easybuilder that works with my hmi, I can create a mysql database that can store the data.
The problem is how can I update automatically firebase database with mysql database with an interval of time in order to access them on the android app.
If there is no solution with mysql, can someone suggest other method to extract the data and use some web server to sync it with the android app?
I don't know your specific need, in terms of data volume or application, but as a workaround, maybe this can help you:
I usually apply MQTT, which many Weintek HMIs have, to send telemetry data, and then use NodeRed to process and redirect the data to a database, email, SMS, Telegram, CSV, TXT... depending on the need , which in your case could be Firebase (I never used it).
It works great for me as I don't have to worry about HMI limitations.
The problem is the reliability of the data, in terms of confirming that when the HMI sends, the server listens and writes, but there are certainly ways to deal with this, and the fact that you need to have a server with NodeRed running.
If you have never done so, in Weintek HMIs you can send the MQTT payload cyclically using macros easily.
At the moment I am working on a CRUD app that I am going to deploy (someday) and use for my own startup company. However I am nowhere near finishing this product and I stumbled upon a question that I can't seem to figure out.
I am using Express to serve angular the data out of my MySQL database. To do this I had to create '/api/' routes. However if I go (for example) to '/api/clients' I will be able to see the entire list of clients in an ugly array. In this case that does not really matter since it's just the data they were able to see anyways.
However my question is, is it important to block these kind of routes from users? Will problems arise when a user goes to 'api/createClient'? Could this result in a DB injection that could ruin my db?
My project can be found here: https://github.com/mickvanhulst/BeheerdersOmgevingSA
The server-side routing code can be found: server > Dao > clientDao.js
Controllers, HTML & client-side routing can be found in the 'public' folder.
I hope my question is clear enough and someone will be able to answer my question. If not, please state why the question is not clear and I will try to clarify.
Thanks!
Looking at the code, it looks like your URLs can directly be accessed using browser and if yes, then this does pose a security concern.
Doing DB transaction with the user provided fields or values is major security concern, if these data are not validated and sanitised before making a database call.
I would recommend following minimum steps to follow before crafting APIs which is internal but can be accessed using browser -
If this is internal, then do not provide HEADER ACCESS CONTROL from the server or keep it confined only to your domain name. This prevents any ajax call to be made to your APIs from another domains.
Do sanitise and validate all the data thoroughly before doing any kind of database transactions. There are lots of material on this everywhere on how to do it.
If these APIs are meant to be used for internal purpose, then kindly provide some kind of authentication to your APIs before doing the logical work in your routes with the help of middle-wares. You can leverage cookie authentication for very simple API authentication management. You can also use JSON Web Tokens, if you want a more levels of security.
If you are manipulating your databases then I would highly recommend to use some kind of authentication in your APIs. Ofcourse, point number 2 is must.
We plan to use a MySQL database as the backend for our (Java or Ruby on Rails) based web application. After completing the web application, we want to port the application to iOS and Android.
We want to be able to run the application in "native" mode - that is, if a network connection is not available to the smart-phone, the system should be able to store the data locally, and sync with the backend when the network connection becomes available.
The best kind of framework for this kind of syncing is a library/framework such as Parse.com.
The question to which we want an answer is: is it possible to exchange data between the web application data stored in MySQL, and the Parse.com data which is stored in a proprietary format on the Parse servers?
Answer to your question: It is indeed possible, but syncing data is an advanced topic.
However, you also state that you want to access the parse data when offline. As I understand it, you want Parse to handle the offline state, and then sync to MySQL when connection is back up. Parse does not offer functionality to store data offline, other than caching requests. You probably need another service for your specific needs.
I might have misunderstood the use case. If so, my alternate understanding would be that ALL data for the smart phones will be handled by parse: both offline and online, with syncing. The answer is still: parse does not offer this kind of functionality.
I'm building a Phonegap application and i'm planning to set it to work in both Offline and Online mode, the idea is to get JSON Data from a PHP server side script and show it in the application, these data contains text, images so my question is is there anyway to set a dynamic cache manifest or any other way in a way that the user can see the already loaded data when he's offline, like caching the json result itself or anything else you can help me with
thanks
What do you mean by "work in both Offline and Online mode"? Are the users able to only see the user created data or can they modify it too? Is it correct to use a Manifest to control caching of user created data? I would have thought you should only really use it for caching the application templates and code. In any case I think you'll probably need something that you can exert more control over, something based perhaps using WebSQL, IndexedDb or LocalStorage.
I have been working on the modify-it-too question for many months and have a solution in the form of javascript(phonegap) <-> server synchronization that works somewhat like a version control system, with version numbers and conflict resolution. There's some pretty great docs and demo on the GitHub page and you can even see a presentation courtesy of SkillsMatter / LondonAJAX.. Currently there is only a JS/Node based server but I will probably do PHP based server because that's my day job If you look at the server code it's pretty easy to implement.
BTW I would use a different method to store the images, why not just download them and store them in a File, I don't know the storage limits, but I bet that is the way that Phonegap will let you get away with most.