I'm building a crossplatform HTML/Javascript app for iOS and Android using PhoneGap and jQueryMobile, and I am upgrading my app with (among others) a few new fields in one table of the local database (localdatabase/websql).
The challenge
I want to make sure that when the database is expanded with the new table fields, the existing user data, the user data will not be removed or become locked in an inaccesible older version of the database.
The background:
My app has a local database of the user's data (incomes and expenses, plus a few settings). These data need to be persitent, and the way to go, back when I started, was using the HTML5 localDatabase functionality, since that is both persistent, and available for the iOS and Android browsers as well as for most desktop browsers.
I am using a Javascript plugin/library/thingy called persistenceJS to make dealing with the localdb a little easier. But my question is not really specific to persistenceJS.
I am working on a new version of the app, which makes uses of a few new fields in the Settings table. So when these users download the new app and run it, it must test if their Settings table contains this field or not, and if not it must create the field.
How do I do this testing? I see two lines of thought:
Use the database label... that's used in the openDatabase function. This seems to be used by some developers to store a version number.
My trouble with this option is I only know how to use openDatabase to, well, open a database (and create a new one if none exists), and run a callback specifically if the database did not yet exist.
So if I open the table while specifying something like "v2" in the label, will it create a new table? If so, will it copy the old table's values into the new one?
Check for the existence of the table fields...
I could use openDatabase and then test for the existence of the table fields. If they don't, I could add them. The test would be run every time a user opens their app, which seems a little primitive.
By the way:
I know webSQL/localDb has been deprecated by the overlords, but it's still my tool and I want to stick to it for now.
I've found the answer here: http://blog.maxaller.name/2010/03/html5-web-sql-database-intro-to-versioning-and-migrations/.
Basically, you just apply the changeVersion method with the old and the new version label. If you didn't have a label, then the old label is "". While relabeling, webSQL quietly applies the new schema to the old database. Which in my case means adding the new fields.
The tutorial I linked to is really awesome (and so is the functionality).
I'm adding another answer because I've learned more about localDb opendatabase and migrating it.
As a reminder, openDatabase takes these parameters:
name - (string) name of the database
version label - (string) the version you want to open
display label - (string) a pretty useless display name that seems to be used nowhere
max size - (int) largest safe size is 5 * 1024 * 1024
newly created -= (function) to be fired if the db did not previously exist
It's wisest to assign the output of openDatabase to a variable. I.e.
myapp.db = openDatabase('mydb','','My database',5*1024*1024,newlyCreatedCallback);
First off, it seems wise to make use of the 'newly created' callback that's available as the fifth argument of openDatabase. It will fire only if there was no database with the parameters you specified. To prevent this callback from firing when your database did already exist, make sure you have the name, display label and maximum size set to exactly the values that were used to first create the database.
The reason to do this is that if the database was first created, you know for sure that you will not need to do any migrations. You can go straight to a function that adds tables and fields. I recommend using persistenceJS, a tool that helps you read and manipulate the local database.
Before calling openDatabase, it's wise to use jQuery to create a custom event 'dbopen' whose handler will execute migrations. This handler can be triggered by two events. The first is the 'newly created' callback we just discussed. The second is a setInterval that you define after call openDatabase. The interval must check for the existence of the myapp.db variable that you assigned the openDatabase output to.
The reason to create the dbopen custom event is that if you added a 'newly created' callback which triggers a whole bunch of events and continues the flow of your code afterwards, you will want a similar process for the 'not newly created' scenario. There is no callback for openDatabase that does this, so you will have to manually detect the creation of the local database and trigger 'dbopen' as soon as it has come into existence.
I use a window.setInterval for this. Make sure that you create the custom 'dbopen' event using jquery's .one() function, which will fire at most once. Otherwise if the database was newly created, you will fire the open event once when the 'newly created' callback fires, and once when the myapp.db variable comes into existence.
Related
I'm looking to add a new property to a higher level (main level, I forget the terminology) of the json tree in my firebase real time database. I usually make my edits through the console. I've been able to add new properties at deeper levels but going up the json tree the console doesn't let me by saying: Read-only & non-realtime mode activated in the data viewer to improve browser performance
Select a key with fewer records to edit or view in realtime
I thought about exporting, adding, then importing again (I've done this before at lower levels) but this seems a little scary having to essentially reimport the database just to add a new property. I've read the docs and they suggest using set method. How is this normally done?
I was able to figure this out.
You can use the browser URL to create a new node in the json tree if you cannot do it in the console because of it's offline mode.
Just enter the node name in to URL as if you were going to an already existing url and it'll get added
example: http://www..com/newnode
Good evening,
I am currently developing a way to import machine created data from a csv sheet into a database.
The question I have is, is there a way to react to a change in a csv file with Lua.
The file gets a line in this format:
17162H,"801234500001",9/23/2016 12:33:30 PM,"INV"
Every time a scanner is finishing a scan process, added under the old lines, but there is no direct connection to the database, to trigger the script.
It doesn't matter if the change is detected via different file size, foldersize (of the folder that contains the file) or a change within the file information (like date of last opening), but I can't open and read in it permanently due performance reasons.
Also this is the first time I ask here, so sorry for my clunky way, I'll try to improve myself with that over time.
Take a look at linotify, it has lua bindings for inotify and looks like it should do the trick, using the "modify" event to trigger your script.
I use LibUV based variant in my spylog apllication
Usage:
file_monitor(path_to_file, {eol = '\r?\n'}, function(line)
...
end)
If you need to run this on Windows, you can use winapi library, which supports file watchers. Here is an example of how it's used in one of my projects; you'll need to call winapi.sleep() to allow time for the check to trigger.
Let's say I have a form for editing a furniture.
In my business logic, a furniture can have finishings.
In my ORM logic, "finishing" is a separate entity from "furniture", with a many-to-many relationship.
So in my "edit/furniture" form I present the user a "finishings" subsection,
with 2 UI (form) elements:
an autosuggest field, feeded by finishings.json (a file),
allows the user to attach already existing finishings to the furniture being edited,
a "+" button, that make fields appear, to enter one or more finishings.
allows the user to add finishings on the fly.
I want the user to be able to add finishings without leaving the "furniture/add[edit/$id]" REST url, so he doesn't experience a page reload.
To this end, I use the "form within form" trick, which means I'm not submitting the entire furniture form : I don't want the user to leave the page, nor do I want it to reload it. So the "submit new finishing" is a fake submit button that triggers the "finishing/add/" REST url. The REST part of the app is responsible for rewriting an updated finishings.json file that reflects the modified DB. Pretty standard stuff.
The real trick is : I would like the autosuggest field to reflect changes in realtime, doing this by "re-reading" the finishings.json file.
Here are 2 options:
Instead of the "form within form", I could just update the UI and push new finishings entries into a JSON object built upon the initial finishings.json file reading. I could push the new entries into this $scope object which would give the UI what it needs. Upon submitting the "edit furniture" form, I would prep data and sort stuff out: go through every finishing attached bu the user to the furniture, and separate the existing ones from the "just added" ones.
I keep my "form within form", because I want my finishings.json file, not a json object, to be the "source of truth".
I think I can manage option 1, but I'd really prefer to go with 2.
How would you do that?
What is the proper way in AngularJS to reload the finishings.json file on demand, and having it refeed the autosuggest with a fresh new batch of options, in real time?
Is the purpose of having this fresh data from the finishings.json file to enable every application user to be aware of new data in that file?
I mean... if me as user A am adding a new finishing, you want my finishing to be accounted by every other application user? Is that part of the suggestion algorithm?
If this is the case, and in fact what you're looking for is some sort of changes listener, you would have two options to solve this:
a) Using $timeout to pool the file every x seconds (I personally don't like this kind of approach).
b) Create a service that uses some form of WebSockets implementation (Socket.io, SignalR, etc). At an high level it would work this way: Your clients connect to the socket server/hub and subscribe to a data change event. Then, everytime a finishing is added to the file, you would then emit a data change event to the socket server which in turn would broadcast that event to every connected client.
I have the following problem:
I Am using a Django framework.
One of the parts in a system (non-django) writes to the database, in the same database that django is using.
I want to have a signal when an object is being saved. It's a django model object but not saved via django, but directly in the mysql database.
Is there a way django can watch save-actions in his database when it's not being saved by django?
The neatest way would be: create an Api, and let the save action run through this api. The save signal can than be django default. (but this depends on some work of externals... so not the prefered route... for future development it sure is).
Another option is to implement celery and create a task that frequently looks whether one of the saved objects has had no follow up..... (also quit some puzzling I guess to get this up and running)
But there might be an easier... for me unknown?
I saw django watchdog solutions for file systems... not for databases (probably because django has this build in... when properly done through django)
to complex it: I test and develop locally with sqlite .... but the save signal I can put in my tests without needing to get this locally working.... as long as it works in mysql, I Am happy.
You can try this solution:
Create a new table 'django_watch' with one column 'object_id' (add other columns like 'created_datetime' etc according to your standards);
Lets say your main table is 'object'. Add a mysql trigger for the INSERT event on this table.
You should add an extra insert query inside the trigger to insert the object_id into 'django_watch' table.
Now you can have a cronjob that will be inpecting the new table 'django_watch' (for updations in Django objects) and perform necessary actions. You can run this cronjob continuously with some 1 minute delay (upto you).
In the end, I wrote an api that can be called by the thirdparty module. I delivered the code to logon on django using c-code to this api and call the GET of this api. (using django rest framework). This api just saves the object (the id given in the url), and from there on it's default django. The only thing the third party had to do is build in my code to call the api as well....
Maybe not the best solution, but the best to implement for my problem....
There are many times that I've needed to execute some code after a number of events have fired, and I've come up with counters and such but I feel there must be a better way.
For example, say five files need to be loaded, after which a UI component will become active.
If I set up a counter that increments each time a file is requested, then decrements each time one has loaded, I run the risk that the first two or three files may somehow get completely loaded before my code gets around to requesting the fourth and fifth, which would mean that my counter would be at zero when I still have two files to load, thus allowing the UI component to be prematurely activated.
There are some cases where you could know the number that need to be loaded before the requests go out, but it's possible that the first file contains the paths (and therefore the number of) files. (And this file-loading scenario is only an example of the pattern I'm trying to explain.)
Does anyone have an elegant solution for this? (Does my description make sense?) Thanks!
You could do something with a task framework like spicelib
Using that as an example
Create a FileRecursionLoadTask which grabs a file and completes when that file and any references it makes are loaded.
Add each FileRecursionLoadTask to a SequentialTaskGroup.
When the TaskGroup is completed, then you know all of the file loads have completed.
There are also plenty of other task frameworks which you might like better. For example, Spring ActionScript also has one.
Before executing a request, store a reference (a unique request uri, the loader object or a special command object) in a list. When a loader has finished, remove that object and call a function that checks if there are remaining active tasks in the list.
This isn't specific to file requests nor request in general, it can be used for anything that needs to wait for multiple actions to finish. Multiple list can be used to process multiple types of action at the same time. The object stored in the list could be implemented as a command object, which could provide more information about the task. This is called command pattern.
If you're doing just loading, like Jacob, I would also suggest a library that handles loading
If the case of a more complicated situation like mixing loaders and other event listeners, I would suggest using an event that fires whenever there is any change to any of the dependencies. In addition all the objects/classes would have a state.
Then I would create a listener adding function for the class that would need to do the function or initiate it, that would have 3 parameters
object with event dispatcher (assuming they all use the same update event) ie. assetLoader
name of object state ie. headerLoaded
state value's desired ie. true
the function would add the listener to a chain of listeners, and any time any of the listeners fires, all objects would check if the state value.
This would allow for regression as well (like when a user presses a button, the content starts loading, but then the user presses cancel, even if all the assets load, the state of one object would be false, thus not allowing the item to complete) If you were using counters, it would be the equivalent to adding instead of subtracting, but much more reliable.
Looking for a design pattern? Try the command pattern (http://johnlindquist.com/2010/09/09/patterncraft-command-pattern/)
(The video is a great example of what command pattern is and how it works - using Starcraft as an example.
The implementation is that you queue your load commands so that they do not execute out of order, and you can add the enable or disable commands to your command que. So the command pattern will play back your commands something like: load, load, load, enable ui item, load, load, enable another item
Good luck