Find out the data update and data ingestion frequency of CiviCRM, phpMyAdmin - mysql

Hi I'm using CiviCRM for membership management. The database is not set up by me and it's old. I wonder where can I find out the database update frequency. I access the database via phpMyAdmin.
I had a look on the information schema but didn't see anything useful.
thanks!

if you mean code updates, which may impact db, civicrm.org/blog/tags/release
if you mean other minimum requirements (php and mysql) try here docs.civicrm.org/installation/en/latest/general/requirements

If you want to see the version of CiviCRM installed, you can see it in the column version of the table civicrm_domain :
SELECT version FROM civicrm_domain;
Since CiviCRM 5+, there is exactly one release every month. As we are at 5.50, it's quite easy to know how old is your CiviCRM. Also see here for the list of releases : https://civicrm.org/blog/tags/release
CiviCRM is not a standard module / plugins, it's more of it's own software that can be integrated with the CMS (Drupal / WordPress). There is no auto update so you need to do it manually using this procedure depending on the CMS :
https://docs.civicrm.org/sysadmin/en/latest/upgrade/drupal7/
https://docs.civicrm.org/sysadmin/en/latest/upgrade/drupal8/
https://docs.civicrm.org/sysadmin/en/latest/upgrade/wordpress/
https://docs.civicrm.org/sysadmin/en/latest/upgrade/joomla/
Lastly, if you have more questions on your CiviCRM, there is a specific StackExchange here : https://civicrm.stackexchange.com/

Related

Multiple insert / update in session table of Joomla

I have a Joomla 1.5 based intranet system. User load on this system vary between 100-6k. We have customized this system a lot and using Joomla core functionalities only. I know that Joomla 1.5 is an outdated system and need to upgrade to the newer version or at least 1.5.26. But, we have upgraded PHP and apache and managed the security vulnerabilites. So, upgrading Joomla is not an issue in this question.
We have seen that Joomla is using jos_session table to manage session in the application. Update query is getting executed in jos_session table on each click of every user. Due to this, I can see that multiple insert (for new session) update query (update the timestamp) on jos_session is getting executed.
Is there any specific reason about why Joomla doing this?
Is there any better alternative solution to get rid of multiple insert / update on a single table?
Joomla implements its own custom session management (instead of using the one provided by PHP). And it's session management is implemented via a database table.
Each time a user loads your page a session is created or updated (if already exists). This is why you see all the activity on that table.
By default Joomla would use an MyISAM engine for job_session table. This would cause many table crashes for me so I had to change it to InnoDB and that fixed the crashes issue. It may also improve performance, but cannot comment on that.
In the Joomla back end: Site -> Global Configuration -> System you can set the Session Handler to 'None' and this will stop using the database and it will default to using PHP sessions. However not everybody recommends you do this. You could do your own tests and see what works better for you.
Of course, make sure you have a backup for your website before starting to make tests with the settings.

How to log mysql database structural changes

I'm working with a project which is using mysql as the database. The application is hosted with many clients and we are doing upgrades for the current live systems often.
There are some instances where the client has change the database structure(adding new tables) and causes some unexpected db crashes.
I need to log all the structural changes which were done at that database, so we can find the correct root cause for that. We can't do it 100% correct with diff tool because it will not show the intermediate changes.
I found http://www.liquibase.org/ tool but seems little bit complex.
Is there any well known technique or a tool to track database structural changes only.
well from mysql studio you can generate all object's schema definition and compare them with your standard schema definition and this way you can compare two database schema...
generate scrips of both database (One is client's Database and One is master copy database) and then compare it using file compare tool would be the best practice according to me because this way you can track which collumn was added, which column was deleted, which index was added like wise without any tool download.
Possiable duplication of Compare two MySQL databases ?
Hope this helps.
If you have an application for your clients to manage these schema changes, you can use a mechanism at application level. If you have a Python and Django-based solution, you could probably use South which provides schema change tracking and rollbacks.

How can I explore mysql database files (*.frm, *.myd and *.myi) without MySQL?

The old versions of our product allowed to capture the current state of the system in a single archive file, which also contains the MySQL database files - lots of <XXX.frm, XXX.myd, XXX.myi> triples .
Now we have the next generation of the product, which does not do anything stupid like capturing the database files, but it must know to read the archives produced by the old versions.
Our product is a commercial closed source product, but it is not very expensive. We had to stop using MySQL, because of the second reason (Oracle has changed the MySQL licensing) and we cannot use MariaDB, because of the first one (their licensing freaked the s*t out of the company lawer).
So, my question is there another way to read these MySQL database files? A commercial light weight solution is fine - after all, we are talking about read-only exploration of the database files. Free/Open Source alternatives are welcome too, as long as they do not mean that the code using them must be Open Source too.
Thanks.
EDIT
Besides the issue whether I can or cannot continue using the old version of MySql to read the old MySql database files, the question remains how can I read them? I mean, MySql is no longer our database, so even if I can bundle with the old MySql implementation, do I have to install the full blown database engine to just read the files? I'd rather avoid that.
If you want to go thru tables structure it would be enough to read the following links.
MySQL internals (all), File Format, MyISAM
If it is not enough and you database size less then 10G you can use Ms SQL Server Express (which is free with DB less than 10G. Page to compare different versions of Ms SQL Server is here). Search for the way to convert MySQL files to Ms SQL Server. Here is the first link a got from Bing: link1 (I suppose not all of them need MySQL server)
If it is not suitable. You can try another MySQL forks like: XtraDB, OurDelta, Drizzle, PBX and so on.
Hope you will find something useful.
We have found a solution. Unfortunately, it involves MySQL, so there are potential licensing issues. Here it is - http://dev.mysql.com/doc/refman/5.0/en/libmysqld.html
All it takes is download the MySql source code and help yourself with:
libmysqld.dll
libmysqld.lib
header files from the include folder
Then it is possible to read the files using the embedded MySQL database engine inside libmysqld.dll.

How do I populate a rational multi-table MySQL database from an existing one table database?

Basically have many huge delimited files that I know I can import as a table, but I need to map that data to an existing rational multi-table MySQL database. There should not be any conflict with datatypes, but I'm super new to this, so please point out anything I should be watching for. Clearly I'm not going to run this in production either until I know it works.
Not 100% sure stackoverflow is the right place to ask a database question, but I couldn't find any other Stack Exchange that was a better fit.
Posted this question on SuperUser looking for a GUI to do this, but I up for coding this is it gets the job done. As such there is no target language, just the requirement that the database be MySQL.
Also, found this stackoverflow Q/A that deals with MS-SQL's SSIS (which I'm not planning on using due to cost, but the content and issues faced are of the same nature it appears.) --
Loading Multiple Tables using SSIS keeping foreign key relationships
I'd suggest using the ETL(extract translate load) tool from the Pentaho Business Intelligence package. It's got a bit of a learning curve but it'll do exactly what you're looking for. Their ETL tool is called Kettle and it's extremely powerful once you get the hang of it.
There are two versions of Pentaho, an enterprise version that has a free trial, and a free community version. The community version is more than capable but you might give the enterprise version a test ride too.
Here's some links
Pentaho Community Edition Site
Kettle Site
Pentaho Enterprise Site
Update: Multiple table outputs
One of the key steps in your transformation is going to be a combination lookup-update. This step checks a given table to see if a record from your data-stream exists and inserts a new record if it does not. Regardless of whether it's a new or old record it's going to append the key field from that record into your data-stream. As you keep going you'll use these keys as foreign keys as you import data into related tables.

How to synchronize development and production database

Do you know any applications to synchronize two databases - during development sometimes it's required to add one or two table rows or new table or column.
Usually I write every sql statement in some file and during uploading path I evecute those lines on my production database (earlier backing it up).
I work with mySQL and postreSQL databases.
What is your practise and what applications helps you in that.
You asked for a tool or application answer, but what you really need is a a process answer. The underlying theme here is that you should be versioning your database DDL (and DML, when needed) and providing change scripts to be able to update any version of your database to a higher version.
This set of links provided by Jeff Atwood and written by K. Scott Allen explain in detail what this ought to look like - and they do it better than I can possibly write up here: http://www.codinghorror.com/blog/2008/02/get-your-database-under-version-control.html
For PostgreSQL you could use Another PostgreSQL Diff Tool . It can diff two SQL Dumps very fast (a few seconds on a db with about 300 tables, 50 views and 500 stored procedures). So you can find your changes easily and get a sql diff which you can execute.
From the APGDiff Page:
Another PostgreSQL Diff Tool is simple PostgreSQL diff tool that is useful for schema upgrades. The tool compares two schema dump files and creates output file that is (after some hand-made modifications) suitable for upgrade of old schema.
Have scripts (under source control of course) that you only ever add to the bottom off. That combined with regular restores from your production database to dev you should be golden. If you are strict about it, this works very well.
Otherwise I know lots of people use redgate stuff for SQLServer.
Another vote for RedGate SQL Compare
http://www.red-gate.com/products/SQL_Compare/index.htm
Wouldn't want to live without it!
Edit: Sorry, it seems this is only for SQL Server. Still - if any SQL Server users have the same question I'd definitely recommend this tool.
If you write your SQL statements for your development database (which are, I imagine, series of DDL instructions such as CREATE, ALTER and DROP), why don't you keep track of them by recording them in a table, with a "version" index? You will then be able to:
track your version changes
make a small routine allowing the "automatic" update of your production database by sending the recorded instructions to the database.
I really like the EMS tools.
There tools are available for all popular DB's and you have the same user experience for every type of DB.
One of the tools is the DB Comparer.
TOAD
saved many an ass several times in the past. Why do people run sql with no exit strategy?
the redgate one is good also.
Siebel (CRM, Sales, etc. management product) has a built-in tool to align the production database with the development one (dev2prod).
Otherwise, you've got to stick with manually executed scripts.
Navicat has a structure synchronisation wizard that handles this.
I solve this by using Hibernate. It can detect and autocreate missing tables, columns, etc.
You could add some automation to your current way of doing things by using dbDeploy or a similar script. This will allow you to keep track of your schema changes and to upgrade/rollback your schema as you see fit.
Here's a straight linux bash script I wrote for syncing Magento databases... but you can easily modify it for other uses :)
http://markshust.com/2011/09/08/syncing-magento-instance-production-development
DBV - "Database version control, made easy!" (PHP)