Back in september '17 I installed an ejabberd-instance on a (debian 9-)server my dad still pays for, but is no longer using. The small circle of friends that initially used it, quickly grew and now there's a few hundred of us chatting away.
Time has come to update the ejabberd (I installed 16.09-4) because we want to stay secure and be able to enjoy all the latest whistles and bells. I thought some simple 'apt-get upgrade'-magic would do the trick, but boy, was I wrong.
I looked at the docs (Specific version upgrade notes) and tried to update to 17.03, by following the steps described there.
These updates fail miserably (missing links to /usr/sbin/ causing failure to to restart server after update. Adding symlinks does not help. I believe I chose to install to /opt/ last year, which would explain why).
Since I have some experience with MySQL and none whatsoever with Mnesia, I also decided in the beginning to add "default_db : SQL" in my ejabberd.yml file. I (wrongly) assumed all data would be stored in a MySQL-DB, and no longer in the Mnesia. This does not seem to be the case: I see all of the registered users in Mnesia (using ejabberdctl) but only part of the complete list in MySQL.
I guess my question is twofold:
What would be the most efficient way to update from 16.09-4 to the latest version ? (I am aiming for a solution without having to install a new ejabberd-instance on a different server and simply backing up and migrating the MySQL)
How can I avoid using Mnesia completely and only use MySQL ? And, related: How can I push all existing Mnesia-data into MySQL ?
What would be the most efficient way to update from 16.09-4 to the latest version ?
As you have some kind of confusion about what and where is installed, problems in links, paths, etc, I propose as "the most efficient" a solution that has many many steps, but at least will work whatever you have:
First handle the old one:
Find your config dir and mnesia spool (try 'ejabberdctl help')
Stop ejabberd
Move to a safe location your config dir and mnesia spool
Uninstall ejabberd and all ejabberd related files and dirs
Now comes the new one:
Install ejabberd in the way you prefer: Debian package, source, whatever
Let it start, so it creates default config files and minimal mnesia spool
You can even register an account, login, check it runs correctly
Find the new config dir and mnesia spool (maybe they are different paths due to different install process)
Stop ejabberd
And now recover your old information
Replace the new config dir with old one, and the new mnesia spool with your old one
Start ejabberd again, with old config and old mnesia. It may complain about useless or unknown options, it may also report it's updating the database schema.
If it at least starts, check you have all the users, and also update the config file in case there were renamed options...
Of course, there are easier and faster ways to update, but your special case requires special steps.
How can I avoid using Mnesia completely and only use MySQL ?
Why exactly do you want to avoid it at all costs? It's required for some stuff in ejabberd, like client sessions, internal routes, commands... Other stuff can be moved to MySQL, like roster, passwd, MUC, vcards...
And, related: How can I push all existing Mnesia-data into MySQL ?
You can configure many modules to use sql instead of mnesia, and then run the export2sql command that produces a *.sql file that you will import into your new MySQL database. After that, you can delete content of mnesia tables using the WebAdmin.
Related
tried to serach in documentation but I have a lot of doubts...
I'm running ejabberd version 2.1.9 on an old debian server (5.0.8) with more than 500 users...
Now I was asked to update to the latest version, but I need some advice;
in the documentation I saw there are specific upgrade instructions between versions also from 2.1.x to 16.02
if I upgrade to 16 version can I jump directly to 22.10 or I have to upgrade to every intermediate releases?
or as an alternative, is possible to export users, passowrds, shared rooster ecc..and restore them on a fresh installed 22.10 ?
#badlop
Thank you very much for detailed info :-)
but I'm stuck with backup database... If I use the plain text backup "ejabberdctl dump" give me errors importing table "pubsub_node" on the new server
** Table pubsub_node already exists on ejabberd#localhost, just entering data
Problem 'error {case_clause,
{aborted,
{bad_type,
{pubsub_item,
{"751ca223b3f58d185f3afef05d0e3d4eb236c376",218},
{{1317,45407,740776},{"donkeykong","acme.com",[]}},
{{1317,133197,815914},
{"donkeykong","acme.com","stc"}},
[{xmlelement,"metadata",
[{"xmlns","urn:xmpp:avatar:metadata"}],
[{xmlelement,"info",
[{"id","751ca223b3f58d185f3afef05d0e3d4eb236c376"},
{"type","image/png"},
{"bytes","16541"},
{"width","96"},
{"height","94"}],
[]}]}]}}}}' occurred executing the command.
Stacktrace: [{ejabberd_admin,load_mnesia,1},
{ejabberd_ctl,call_command,3},
{ejabberd_ctl,try_call_command,3},
{ejabberd_ctl,process2,3},
{ejabberd_ctl,process,1},
{rpc,'-handle_call_call/6-fun-0-',5}]
tried editing and removing the pubsub_item, but the problem will show on the next pubsub_item.
If I try to use "ejabberdctl backup" the server reply with this error
Can't store backup in "/tmp/jabba.backup" at node ejabberd#jabba: {"Cannot prepare checkpoint (replica not available)",
[temporarily_blocked,
{{1670,
255465,
408029},
ejabberd#jabba}]}
even if I have only one node
The import error of dump came up even if I try to make a little jump from 2.1.9 to 2.1.13
any Suggestion?
with more than 500 users
An ejabberd server with around 500 online users? That's a small server, I guess you don't even use SQL database, so that's one less thing to worry about.
An ejabberd deployment is composed of:
the code (source or binary)
configuration file (or files)
mnesia internal database, that is stored in the mnesia spool dir (see system install)
SQL database (if you configure it)
log files (only useful for your interest and consult)
When upgrading ejabberd code, it is usual that the release notes mention some changes in the configuration, and some changes in SQL schemas. The changes in the mnesia internal database are implemented inside ejabberd and applied automatically when needed.
I have to upgrade to every intermediate releases?
In general no need. In general when jumping a few releases, you just read the release notes and apply the configuration and SQL schema changes of all the intermediate releases.
But in your case... as it's a big jump, I recommend you to not touch the production server yet. First test the upgrade in another machine (your personal machine, or other unused server, or a laptop, or a docker container...), so you learn how to do it perfectly without annoying the users.
There are many ways; but I were you, or if I were sitting in a chair next to you, this is how I would do it. I would be optimistic and try to jump from from 2.1.9 to 22.10, but slowly, and using a temporary server for testing the process and learning:
In a testing machine install the desired ejabberd version (22.10 or whatever). It should work perfectly, as it's empty and has default configuration. Notice most XMPP clients allow you to login to an account whatever#example.com specifying the IP address and port of the server: in your case you will have to specify the IP address of the new ejabberd server, as the XMPP domain it doesn't match the DNS name.
Obviously, the configuration in the new server will lack customizations that are essential for you (the served domain, what accounts are admins, certificate files, port numbers, custom modules...)
Copy the old configuration file to the new server. Notice they use different formats (ejabberd.cfg in erlang format... and ejabberd.yml in YAML format).
Manually and slowly apply one of your customizations in the new configuration file. Restart ejabberd, check it starts and works correctly, and then repeat with another customization.
Now you have a new ejabberd server running, with all (or most) of your desired configuration. Two things are lacking: the users data, and of course replacing the old ejabberd with the new ejabberd.
In the production ejabberd server, export the mnesia database using "ejabberdctl backup" and "ejabberdctl dump" (better to have two alternatives, in case one doesn't work). Copy those files to the machine that has the new ejabberd server.
In the new ejabberd, run restore specifying the binary backup
(or load specifying the text dump). With some luck, this will end correctly.
Restart ejabberd. It will notice that the mnesia tables use very old schemas, and will automatically update them. This may take a while. With some luck it will end correctly.
If steps 7 or 8 fail, and you are lucky to know what username or what data is problematic, you can try to delete or "fix" that in the text dump file before loading it).
Once you are happy with the new ejabberd server (the new configuration looks great, has all your customizations, and all the users are correctly imported), then it's time to replace the old ejabberd with the new one.
Alternatively, you can try to upgrade from 2.1.9 to 16.02, and later from 16.02 to 22.10
In any case, general recommendations:
take it slowly because there are many steps
be patient because you may face small problems that require fixing before continuing to the next step
be kind to yourself because you never did this
annotate every major problem you face, and how you fixed it (in case you find this problem again this day or in the next days, or eventually somebody else in your organization)
be sure this can be done (in the worst case, with a progressive upgrade from one version to the next :)
Godaddy is forcing old servers offline and I have no choice but to migrate my old mediawiki service to a new server.
I had to maintain the old versions due to use of extensions which I fear are either gone or different now.
Here are the relevant versions:
Source: mediawiki 1.14.0, PHP 5.1.6 (apache2handler), MySQL 5.0.77
Target: mediawiki 1.32.0, PHP 7.2.15, MySQL 5.7.25, ICU 60.2
I have copies of all contents of /var/lib/mysql/ from the old server copied to the new one in a backup directory, and I have mysql dumps copied over too.
Is it possible to simply put the database tree from Source into Target under a different name and then modify LocalSettings.php?
If not, can the mysql dump be restored to do the job?
Extensions have changed and I will have to review hundreds of pages to check for problems.
Any advice on how to do this with the least problems is welcome.
Also, I might consider hiring a contractor to review the task and supervise my work. It would have to be someone who has done migrations of this sort before, or someone with other clear sufficient experience.
The short answer to your question is, painfully. That's about a decade's worth of changes. There isn't any magic spell that will make applying them less hard than it sounds; you'll have to import the old database, run update.php, and prepare for debugging any errors that might occur as you will be using an upgrade path that few if any people have taken before you.
Alternatively, you might export the content (maintenance/dumpBackup.php existed in 1.14) and reimport into the new wiki, and accept losing and having to recreate everything that's not content or content history (like user records).
Is it possible to simply put the database tree from Source into Target under a different name and then modify LocalSettings.php?
Yes, although I'm not sure what you win by using a different name.
I was facing a similar issue. I can only strongly recommend to read the release notes for any version carefully, especially for database changes and changes on how skins and extensions are loaded in the LocalSettings.php. For example the way the skins work change around version 1.24 IIRIC.
Also, be sure to create a renamed copy of your 1.14 LocalSettings.php and create a fresh one during the upgrade via the installer, the old one will cause a multitude of exceptions. You may also need to run the database upgrade several times.
Also make sure you move your old 1.14 wiki installation to an extra folder and upload a fresh mediawiki installation, do not just overwrite your existing files in the 1.14 one with the new mediawiki version.
In the end, I went the long route and upgraded version by version first on a local XAMPP dump, then on the server after I identified all the issues for each version upgrade, checked the release notes for each version and made the required changes to my custom skins, replaced deprecated extensions etc.
Good luck.
So once a week or a month we do an update to many server machines.
Sometimes a Git pull is enough, some times an SVN update, sometimes there are changes to the database. Or a combination of those. Also there is this project that has many little servers that have a very simplified version of our system and very unreliable internet. Sometimes it might be done from one of the servers, sometimes from the local working computers.
I would like to do our work a bit easier by going through all our servers and doing the appropriate actions. I have found a couple of useful Perl packages: Net::SSH::Perl and Net::SSH::Expect;
The Net::SSH::Perl fails me.
Also I have not found out how to use its cmd command in succession. For example:
my $ssh = Net::SSH::Perl->new($host);
$ssh->login($user, $pass);
my($stdout, $stderr, $exit) = $ssh->cmd('cd web/scripts && ls -la ');
warn Dumper $stdout;
my($stdout, $stderr, $exit) = $ssh->cmd('ls -la');
warn Dumper $stdout;
The 2 ls -la commands return different results. As far as I can understand (and is explained in the documentation) the ssh executes the command and then exits. It is said that I could use SSH 2 version of the SSH protocol (or something like it) and should not have this problem, but it persists (or I don't understand how to use it).
Also if a password is asked of me (for example if I run mysql -u user -p) I am unable to provide it. I've tried it with the $ssh->cmd($cmd, [ $stdin ]) , second option but to no result. The mysql is just an example. I might wish to add an IP to /etc/hosts and be prompted for su password or svn update a file and be asked for my SVN password. I know that most of those processes can be configured to not ask for passwords but we want them to ask for password.
The $ssh->shell option seems like would do the trick but when I do something like this:
$ssh->shell();`ls -la`;
the backslashed command doesn't go to the ssh-shell. Actually I have no idea where it goes or if does anything.
The Net::SSH::Expect fails me when there is a bad internet connection.
For MySQL purposes I have created a Perl script that makes connections with each different host and does the changes I want. But it would be great if I could make it all in one script.
I would be very grateful to gain some more understanding on this topic.
This is what Ansible is made for. It uses SSH to communicate to multiple hosts, and provides a decent variable scoping system and flow control for applying various tasks to various hosts.
You can build your own configuration management with perl, but ansible with raw commands (which don't require python on the remote system) or more full fledged modules (which do) is already implemented and takes the same approach. Do yourself a favor- don't reinvent this wheel.
Ansible is far from perfect, but it covers your use case very well.
I personally run it from a docker container because python's package installation story is almost as bad as Perl's :P
On a pure perl basis you also have Rex, see rexify Web site
Which is a kind of Ansible, it got ssh, parallel job and plenty of features but more perly.
OK, it's simpler than Ansible, but it worth a try.
Topic: MySQL Connections
Specific Question: In a 2-3 week process of learning MySQL and attempting to get a Python stock info scrapping program to feed data into a MySQL schema (database, table) I’ve ended up creating a number of connections. (MySQL57, MySQL57_1,MySQL57Nov13) Even though I believe I’ve deleted the connections through Workbench, they still show up as Window services. How do I completely get rid of these beasts, both in MySQL and as Windows services?
System: Windows 8 64bit on a Xeon Ivy Bridge processor, large SSD’s, large HD’s, mucho RAM, dual X-fired video cards, ASUS Sabertooth MB (self-built a yr ago)
Software: MySQL 5.7, complete installation except for module for Visual Basic and Python3 (running 2.7). Attempting to use the above hardware as both a server & client.
Even though I did my first Fortran programming in 1964, and spent my career on Unix machines, please regard me as a newbie when it come to the jargon associated with databases and client/server communications.
Why am I asking this question? My “down the road question” will be about getting data from Python into the MySQL table, but “first things first” --- I would like to get a nice clean installation & am concerned with these connections showing up as windows services.
By the way, I’ve thoroughly scoured this site and many others but have not found any answers that fit my requirements; they’re either for a non-Windows environment, too full of jargon for me to understand, or too simplistic to be useful. Hope that didn't come across wrong :-)
TIA, Doug
You found the uninstallation command already, so let me show you how to make sure not to remove the wrong service.
Open the services management (e.g. via Start menu -> Computer -> right click -> Manage -> Services and Applications -> Services). Scroll to the MySQL server services and double click one that you want to uninstall. A dialog opens with various information about the service including a line "Path to executable:". The followoing line contains the full path to the server this service is for. You should only have one service for each of the servers. If that is not the case then you probably have to go through the registry and search for the service name and remove that key manually, until only one is left. Otherwise just use the path to run your uninstallation if that is for a server/service you don't want anymore.
My problem (as described in my original question) has been solved thanks to the comments that Michael made.
In a command prompt with admin privileges
net stop MySQL57Nov13 (service name)
to stop the service (all 3 of my services)
Note: I had cd'd to D:\Program Files\MySQL\MySQL Server 5.7\bin to run the net stop
commands. I have a hunch that the commands would have worked independent of the directory chosen
to run them from....didn't experiment further though. Likely depends on the PATH env var.
Then go to C:>
sc delete MySQL57Nov13
for all 3 services I wanted to delete
As a reference I used http://dev.mysql.com/doc/refman/5.7/en/windows-troubleshooting.html
The above was written prior to Mike's answer.
Later update: This is a *%#&!! nightmare! Obviously I deleted the key windows service (was one of MySQL57, MySQL57_1, or MySQL57Nov13). This effectively cratered the entire project ... couldn't create a new, fresh connection or anything else. So I uninstalled everything (to the best of my knowledge) using the internal MySQL Uninstall routine as opposed to Windows Program Management uninstall. Fortunately I've got no data which I have to save.
I'll start a new question later today.... I'm going to uninstall the mess again, and start from scratch (the new question will be concerning the MySQL import hanging when I tried to import a .sql file tied to a tutorial I've been watching)
Let's consider my Question solved with many thanks to Michael and Mike
I have written a module that is refusing point blank to create the tables within my mysql4-install-1.0.0.php file....but only on the live server.
The funny thing is that on my local machine (which is a mirror of the live server (i.e. identical file structure etc)) the install runs correctly and the table is created.
So based on the fact that the files are the same can I assume that it is a server configuration and or permissions problem? I have looked everywhere and I can find no problems in any of the log files (PHP, MySQL, Apache, Magento).
I can create tables ok in test scripts (using core_read/write).
Anyone see this before?
Thanks
** EDIT ** One main difference between the 2 environments is that on the live server the MySQL is remote (not localhost). The dev server is localhost. Could that cause issues?
Is the module which your install script is a part of installed on the live server? (XML file in app/etc/modules/, Module List Module for debugging.)
Is there already a record in the core_resource table for your module? If so, remove it to set your script to re-run.
If you file named correctly? The _modifyResourceDb method in app/code/core/Mage/Core/Model/Resource/Setup.php is where this file is include/run from. Read more here
Probably a permissions issue - a MySQL account used by public-facing code should have as few permissions as possible that still let it get the job done, which generally does NOT allow for creating/altering/dropping tables.
Take whatever username you're connecting to mysql with, and do:
SELECT User, Host
FROM mysql.user
WHERE User='your username here';
This will show you the user#host combos available for that particular username, then you can get actual permissions with
show grants for username#host;
Do this for the two accounts on the live and devlopment server, which will show you what permissions are missing from the live system.
In the Admin->System->Advanced section is your module present and enabled?
Did you actually unpack your module to the right space, e.g. app/code/local/yourcompany/yourmodule ?
Do you have app/etc/modules/yourmodule.xml - I believe that this could be the overlooked file giving rise to your problem.
the cache could be the culprit, if you manually deleted the core_resource row for your module in order to make the setup sql run again, you have to also flush the cache
probably a difference between dev and production servers is cache settings, that would explain why you only see this in production
For me, the issue appeared using Windows for development. Linux system is case sensitive. In my config.xml the setup section was named camelCase while the folder was named all-lowercase. Making them the same made the script run.