In this thread someone pointed me to use sqlalchemy-migrate to help with a fast-changing web application using sqlalchemy.
However a Do It Yourself method was also recommended consisting in manually writing CSV columns for the new database schema, and finally import them.
The problem is that I can't find real-world examples of sqlalchemy-migrate. Ressources that I have found at best decribe adding a single column or a column rename. The official documentation essentially describes the API and it's hard to see how to use migrate effectively. From the doc I cannot even know if migrate could help changing the database engine, from sqlite to mysql for example, while the DIY solution would to the job.
I really want to see code that would make some non-trivial transformations of a database schema and proving that migrate is really a useful tool.
Where can I find good examples/tutorials for sqlalchemy-migrate ?
Thanks !
Don't forget about google code search when looking for real work examples of code. For instance the follow search:
http://www.google.com/codesearch?hl=en&lr=&q=%22from+migrate+import%22+lang:python&sbtn=Search
Will pull up a number of real migration scripts. It basically looks for Python files with "from migrate import" in the file.
Work through some of these and see if you can follow what they're doing.
Related
I would like to get some feedback from anyone who's had experience indexing MySQL data with ElasticSearch for full-text searching. How did you accomplish this? I've been researching this a bit and unfortunately I've noticed that ElasticSearch has no official plugin to accomplish this although I've come across three different 3rd party tools:
elasticsearch-river-jdbc
go-mysql-elasticsearch
elasticsearch-river-mysql
I'm unsure which one would be best in terms of performance although I suspect the Go tool might have an advantage due to it's compiled nature and the fact that it uses the mysql binary logs. I would appreciate any advice or examples anyone could provide me with.
Thanks!
UPDATE:
you can now use Logstash to do it..here is an useful blog about it click ere
well you can use mysql UDF functions in mysql triggers to execute external scripts, which will index the newly updated or inserted data in elasticsearch. thats one way. check this to see how to's on mysql UDF
I hope this is not an obvious question. However, I cant find any resources online, since everytime i google wordpress with pyqt, i get websites made with wordpress talking about pyqt.
Are there are resources or tutorials on integrating pyqt with wordpress. Or is this too vague, and it should be pyqt with mysql? Or is this just a bad way to write a program in python that gets data from a mysql database?
My understanding what you're looking for here is not to integrate PyQt with WordPress, but rather Python itself. PyQt it simply a toolkit that offers the GUI (and many more) module that you're likely using, but Python is still the "backbone" of your application.
In order to manipulate the database that WordPress uses, you'd have to manipulate that very database. For that, I recommend MySQLdb Python module. Alternatively, you could use the QtSql that PyQt comes with, but I honestly don't see the need for that.
However, I have to advise you that altering the database directly could have unwanted consequences and it could potentially break your WordPress installation - so make sure to backup the database often. Also, this poses a certain security risk. If I were you, I'd create a WordPress API (there may already be one, I'm not that familiar with WordPress) that will take HTTP requests and send responses as well, which your Python program would then use.
I was given a task to have a better understanding of several ETL packages that were created in a Database project using Business Intelligence Development Studio(SQL 2005).
Currently I have to open each master package, package and then data flow and so on to discover the relationships that exists with either the source tables and the destination tables.
I realized that probably a good way to more easily get that information would be having a tool similar to what SchemaSpy does with a normal Database. That would provide my a high level detail of the relationships that exist.
Anyone knows an application/script that could help me achieving this result?
I tried to search, but I must admit that I was getting the feeling that I wasn't really searching in the right direction as most of my searches ended up pointing for database comparisons.
Turned out, the only way I found to do this was to parse the xml inside the packages and extract the relationships. And then using Graphviz (the same visual component used by schema spy) create the diagrams.
Unfortunately this was an expensive thing to do and I never finished the project. Mainly due to lack of knowledge around the xml structure but it is definitely possible to be achieved
In our rails app we sometimes have db entries created by users that we'd like to make part of our dev environment, without exporting the whole table. So, we'd like to be able to have a special 'dev and testing' dump.
Any recommended best practices? mysqldump seems pretty cumbersome, and we'd like to pull in rails associations as well, so maybe a rake task would make more sense.
Ideas?
You could use an ETL tool like Pentaho Kettle. Once you have initial transformation setup that you want you could easily run it with different parameters in the future. This way you could also keep all your associations. I wrote a little blurb about Pentaho for another question here.
If you provide a rough schema I could probably help you get started on what your transformation would look like.
I had a similar need and I ended up creating a plugin for that. It was developed for Rails 2.x and worked fine for me, but I didn't have much use for it lately.
The documentation is lacking, but it's pretty simple. You basically install the plugin and then have a method to_sql available on all your models. Options are explained in README.
You can try it out and let me know if you have any issues, I'll try to help.
I'd go after it using a Rails runner script. That will allow your code to access the same things your Rails app would, including the database initializations. ActiveRecord will be able to take advantage of the model relationships you've defined.
Create some "transfer" tables in your production database and copy the desired data into those using the "runner" script. From there you could serialize the data, or use a dump tool, since you'll be dealing with a reduced amount of records. Reverse the process in the development environment to move the data into the database.
I had a need to populate the database in one of my apps from remote web logs and wrote a runner script that fired off periodically via cron, ftps the data from my site and inserts the data.
could anyone advise me direct me to a site that explains the best way to go about this I'm sure I could figure it out with allot of time invested but just looking for a jump start. I don't want to use the migration tool either as I just want to put fmp xml files on the server and it create new MySql databases based on the fmpxml results provided
thanks
Technically you can write a XSLT to transform the XML files into SQL. It's pretty much straightforward for data (except data in container fields), but with some effort you can even transfer the scheme from DDR reports (but I doubt it worth it for a single project).
Which version of MySQL? v6 has LOAD XML which will make things easy for you.
If not v6, then you are dealing with stored procedures, which can be a pain. If you need v5, it might make sense to install MySQL6, get the data in there using LOAD XML, and then do a mysqldump, which you can import into v5.
Here is a good link:
http://dev.mysql.com/tech-resources/articles/xml-in-mysql5.1-6.0.html