I am trying to migrate samba domain controller (version 3.6, passdb backend = tdbsam) to a new clustered environment with CTDB (same samba version).
I have a functional ceph storage and the ctdbd configured according to the documentation.
However, all available guides are for the fresh installations and I can't migrate current data into the CTDB.
Simply restoring tdb database into the CTDB doesn't work and it ends with the following error:
2015/05/25 22:25:04.849302 [ 5636]: Invalid version of database dump.
File is version 7308332182664266836 but expected version was 1
Converting tdb database with ltdbtool doesn't bring any joy too.
How to import tdb samba databases into the ctdb?
Thanks,
Marko
Firstly a few words of warning: It is not common to cluster a domain controller with CTDB. Clustering/CTDB is made for file servers which are usually member servers in a domain. The domain controller's primary purpose is to offer logon services for the clients and usually different replication services are used (e.g. storing the DB in ldap instead of TDB). Clustering a domain controller can be problematic since the whole cluster needs to present itself consistently as a single server to the clients. This may pose problems with nmbd / NetBIOS name resolution. Furthermore, one problem is that you need to make sure in the cluster that all underying user accounts are created on all ctdb cluster nodes with the same user IDs.
Generally, it is best practice to separate file servers roles from domain controller roles as much as possible. Also keep in mind that starting with Samba 4.0 there is now the functionality of a Active Directory domain controller which comes with multi-master replication of the user database at the protocol level.
All that being said, the important bit is that you can not freely replace samba and ctdb versions. Samba 3.6 should work with ctdb 2.5, but Samba needs to be recompiled against the ctdb version to be used. I.e. if Samba has been compiled against an older version of CTDB (like 1.0.114.X), then this binary package won't work with CTDB 2.5.
If you have a working set of samba and ctdb packages, then the process of migrating a previous install into this new install depends on whether the previous install was also a CTDB-cluster. If so, you can do
ctdb getdbmap
on the old install and then run
ctdb backupdb
on all databases marked PERSISTENT. On the new install you use
ctdb restoredb
to restore all these DBs.
If the previous install is a non-clustered Samba, then your hint to use ltdbtool is correct. You have to
copy the important databases off the old install, at least:
secrets.tdb
passdb.tdb
winbindd_idmap.tdb
group_mapping.tdb
account_policy.tdb
share_info.tdb
convert them to ctdb format with ltdbtool
make sure ctdb is OFF on all nodes
copy the converted dbs to /var/lib/ctdb/persistent/ on node number 0 suffixing the file name with .0 e.g. `/var/lib/ctdb/persistent/secrets.tdb.0'
start ctdb on node 0
start ctdb on all other nodes
after that, the these DBs should be in sync on the cluster.
Related
tried to serach in documentation but I have a lot of doubts...
I'm running ejabberd version 2.1.9 on an old debian server (5.0.8) with more than 500 users...
Now I was asked to update to the latest version, but I need some advice;
in the documentation I saw there are specific upgrade instructions between versions also from 2.1.x to 16.02
if I upgrade to 16 version can I jump directly to 22.10 or I have to upgrade to every intermediate releases?
or as an alternative, is possible to export users, passowrds, shared rooster ecc..and restore them on a fresh installed 22.10 ?
#badlop
Thank you very much for detailed info :-)
but I'm stuck with backup database... If I use the plain text backup "ejabberdctl dump" give me errors importing table "pubsub_node" on the new server
** Table pubsub_node already exists on ejabberd#localhost, just entering data
Problem 'error {case_clause,
{aborted,
{bad_type,
{pubsub_item,
{"751ca223b3f58d185f3afef05d0e3d4eb236c376",218},
{{1317,45407,740776},{"donkeykong","acme.com",[]}},
{{1317,133197,815914},
{"donkeykong","acme.com","stc"}},
[{xmlelement,"metadata",
[{"xmlns","urn:xmpp:avatar:metadata"}],
[{xmlelement,"info",
[{"id","751ca223b3f58d185f3afef05d0e3d4eb236c376"},
{"type","image/png"},
{"bytes","16541"},
{"width","96"},
{"height","94"}],
[]}]}]}}}}' occurred executing the command.
Stacktrace: [{ejabberd_admin,load_mnesia,1},
{ejabberd_ctl,call_command,3},
{ejabberd_ctl,try_call_command,3},
{ejabberd_ctl,process2,3},
{ejabberd_ctl,process,1},
{rpc,'-handle_call_call/6-fun-0-',5}]
tried editing and removing the pubsub_item, but the problem will show on the next pubsub_item.
If I try to use "ejabberdctl backup" the server reply with this error
Can't store backup in "/tmp/jabba.backup" at node ejabberd#jabba: {"Cannot prepare checkpoint (replica not available)",
[temporarily_blocked,
{{1670,
255465,
408029},
ejabberd#jabba}]}
even if I have only one node
The import error of dump came up even if I try to make a little jump from 2.1.9 to 2.1.13
any Suggestion?
with more than 500 users
An ejabberd server with around 500 online users? That's a small server, I guess you don't even use SQL database, so that's one less thing to worry about.
An ejabberd deployment is composed of:
the code (source or binary)
configuration file (or files)
mnesia internal database, that is stored in the mnesia spool dir (see system install)
SQL database (if you configure it)
log files (only useful for your interest and consult)
When upgrading ejabberd code, it is usual that the release notes mention some changes in the configuration, and some changes in SQL schemas. The changes in the mnesia internal database are implemented inside ejabberd and applied automatically when needed.
I have to upgrade to every intermediate releases?
In general no need. In general when jumping a few releases, you just read the release notes and apply the configuration and SQL schema changes of all the intermediate releases.
But in your case... as it's a big jump, I recommend you to not touch the production server yet. First test the upgrade in another machine (your personal machine, or other unused server, or a laptop, or a docker container...), so you learn how to do it perfectly without annoying the users.
There are many ways; but I were you, or if I were sitting in a chair next to you, this is how I would do it. I would be optimistic and try to jump from from 2.1.9 to 22.10, but slowly, and using a temporary server for testing the process and learning:
In a testing machine install the desired ejabberd version (22.10 or whatever). It should work perfectly, as it's empty and has default configuration. Notice most XMPP clients allow you to login to an account whatever#example.com specifying the IP address and port of the server: in your case you will have to specify the IP address of the new ejabberd server, as the XMPP domain it doesn't match the DNS name.
Obviously, the configuration in the new server will lack customizations that are essential for you (the served domain, what accounts are admins, certificate files, port numbers, custom modules...)
Copy the old configuration file to the new server. Notice they use different formats (ejabberd.cfg in erlang format... and ejabberd.yml in YAML format).
Manually and slowly apply one of your customizations in the new configuration file. Restart ejabberd, check it starts and works correctly, and then repeat with another customization.
Now you have a new ejabberd server running, with all (or most) of your desired configuration. Two things are lacking: the users data, and of course replacing the old ejabberd with the new ejabberd.
In the production ejabberd server, export the mnesia database using "ejabberdctl backup" and "ejabberdctl dump" (better to have two alternatives, in case one doesn't work). Copy those files to the machine that has the new ejabberd server.
In the new ejabberd, run restore specifying the binary backup
(or load specifying the text dump). With some luck, this will end correctly.
Restart ejabberd. It will notice that the mnesia tables use very old schemas, and will automatically update them. This may take a while. With some luck it will end correctly.
If steps 7 or 8 fail, and you are lucky to know what username or what data is problematic, you can try to delete or "fix" that in the text dump file before loading it).
Once you are happy with the new ejabberd server (the new configuration looks great, has all your customizations, and all the users are correctly imported), then it's time to replace the old ejabberd with the new one.
Alternatively, you can try to upgrade from 2.1.9 to 16.02, and later from 16.02 to 22.10
In any case, general recommendations:
take it slowly because there are many steps
be patient because you may face small problems that require fixing before continuing to the next step
be kind to yourself because you never did this
annotate every major problem you face, and how you fixed it (in case you find this problem again this day or in the next days, or eventually somebody else in your organization)
be sure this can be done (in the worst case, with a progressive upgrade from one version to the next :)
I have a task to Migrate MySQL DB to Oracle (its my requirement) i tried to Migrate using SQL developer as defined in below link.
https://www.packtpub.com/books/content/migrating-mysql-table-using-oracle-sql-developer-15
As the DB is huge, the constraints are not copied properly from MySQL to Oracle, i need to define/alter/add constraints explicitly, which is time consuming (SQL developer migrates data 300rec/min from mysql to Oracle) & the entire procedure, views, functions is need to re write.
How can i ensure that data has migrated properly or not.?
Is this is a right approach to migrate?
Should i move to any tool which helps to Migrate? If yes please suggest the tool..!!
Or it is the right thing to Move from MySQL to Oracle.
Thanks in Advance.
No specific answer, but some genaral thoughts based on my experiences with migration.
I've found that there normally isn't one tool that does the whole migration job well, and by whole job I mean:
Fast
Handles all data types, scenarios
And that is from Oracle to Oracle!!
Last project we tried Oracle Golden Gate, and found there were issues with that.
We always end up with a hybrid approach, somethings like:
Extract all DDL manually and pre-create objects - there are weaknesses in the stagndard tools that confound them when extracting DDL, e.g. we found 10g expdp did not handle some quirky PLSQL well, so we resorted to extracting this ourselves.
Some tables work well with SQL Loader, others with GG, others (rare) with a custom extract and load process. We had over 3,500 tables and identified about 100 that worked better done as SQLLoader rather than GG. When I say better I mean with data handling and speed of migration. We created different groups of processing each group having a different method.
Once we have an overall hybrid scheme that works, we tune, mainly by splitting that task into parallel processes, both the export and import side.
All my migrations have been big projects where we have shifted from one Oracle system/server to another, always with the target being a newer version of OS and Oracle.
So, I would imagine that migration between non-Oracle and Oracle will through up even more challenges, and probably not as trivial as imply clicking a few buttons in SQL*Developer.
You may find the expected content from the SQL developer documentation at the Oracle website.
There are migration information available for all Microsoft Access users, MySQL users, Microsoft SQL Server and Sybase Adaptive Server users.
You can also download the tutorial in forms of PDF (best for offline viewing and printing), ePub (best for most mobile devices) and Mobi (best for Amazon Kindle devices).
Recently, I have successfully migrated the MySQL database to Oracle database. Below are detail steps:
Operating System: Desktop Ubuntu local and Desktop Ubuntu on amazon aws
Please Note: Here I am using aws desktop ubuntu server because my mysql
database was pretty big. In my case there were 800 tables, 200 views,
procedures, triggers, and functions. The total size of the database was almost
20GB. In case of small database I would recommend to use local ubuntu server.
Tools Used: SQL Developer, VNCServer, Remote Desktop Client, JAVA 8, Third Party MySql JDBC Driver
1. Setup ec2 ubuntu desktop server : https://www.youtube.com/watch?v=ljvgwmJCUjw
2. Install SQL Developer on #1
Download the SQL Developer package from this link :
http://www.oracle.com/technetwork/developer-tools/sql-developer/downloads/index.html
Accept the license agreement and download "Other Platforms" for ubuntu.
Install the SQL developer package as the following.
sudo apt-get install sqldeveloper-package debhelper openjdk-7-jdk
openjdk-7-jre icedtea-7-plugin
Now all that you need to do is to run the command (you might have a
different version)
make-sqldeveloper-package sqldeveloper-4.1.3.20.78-no-jre.zip
This will generate a debian package that you can use to install SQL developer.
Now install the resulting .deb package using the command (Your deb
might have a different version too)
sudo dpkg -i sqldeveloper_4.1.3.20.78+0.2.4-1_all.deb
In my case, I have used java 8.
3. Once you have done with your SQL developer installation on your newly created ec2 instance with VNCServer then all you need to do is to connect to that ec2 instance with the Remote Desktop Client by default available in your ubuntu local machine.
Use IP:1 with user/pass setup for VNCServer in #1
You can see the remote ec2 ubuntu desktop server. You have to grab the keyboard inputs from the Remote Desktop tool if you want to tab inside the remote server.
Once you get connected with the remote client, open SQL Developer from the terminal or from the explorer.
sqldeveloper
Follow the migration steps provided by Oracle corporation:
http://www.oracle.com/technetwork/database/migration/mysql-093223.html
Please Note: While following the migration steps provided by the
oracle they will ask for the destination database connection i.e. oracle
database connection. This is not the database where your MySQL
database will be migrated. Instead, this database connection will
be used for the migration process. Your database connection user
must have user and database create privileges. Once your connection
have user create privilege, then migration process automatically create
the corresponding database user in Oracle database[if you have mysql_test_db in MySQL
database, same mysql_test_db will be created in Oracle db too].
I recently used sqline's tool http://www.sqlines.com/cmd to convert a dump from mysql in the form of an .sql script to an (almost) Oracle-compatible sql script.
sqlines31113\sqlines.exe -s=mysql -t=oracle "-in=$infile"
I just had to (semi-manually) fix some things in the output and then I could run it on my oracle database.
Specific Question: In a 2-3 week process of learning MySQL and attempting to get a Python stock info scrapping program to feed data into a MySQL schema (database, table) I’m having serious difficulties with connections. Having been bitten by doing something wrong & having to re-install MySQL about 8 times now, I’m getting rightfully leery of just about everything in the installation process. Thus this question:
Is “Local instance MySQL” which shows up in the upper left corner of the first (black) MySQL Workbench screen the proper one to use to connect to the database?
As a follow-up,
What is the difference between the windows service MySQL57, the connection MySQL57, and Local instance MySQL57?
During the very painful process of trying to get MySQL up and running, I think I've gotten further when with a fresh installation the first screen reads “MySQL57” as opposed to “Local instance …”
My setup parameters are pretty vanilla with the exception that I’m putting the MySQL program on my D: drive as opposed to the C: drive (an SSD reserved for the operating sys).
System: Windows 8 64bit on a Xeon Ivy Bridge processor, large SSD’s, large HD’s, mucho RAM, dual X-fired video cards, ASUS Sabertooth MB (self-built a yr ago)
Software: MySQL 5.7, complete installation except for module for Visual Basic and Python3 (running 2.7). Attempting to use the above hardware as both a server & client.
Even though I did my first Fortran programming in 1964, and spent my career on Unix machines, please regard me as a newbie when it come to the jargon associated with databases and client/server communications.
Why am I asking this question? My “down the road question” will be about getting data from Python into the MySQL table, but “first things first” --- I would like to get a nice clean installation that I can trust.
By the way, I’ve thoroughly scoured this site and many others but have not found any answers that fit my requirements; they’re either for a non-Windows environment, too full of jargon for me to understand, or too simplistic to be useful.
TIA, Doug
Seems there's some confusion about certain terms. So let's have a look at each:
MySQL is a server application that you can install and connect to via
TCP/IP (on any platform), named pipes (on Windows) or sockets (on
*nix like platforms). This server is totally independent and has no name or such for identification.
A Windows service to run the MySQL server: this is just a helper to allow running the server when your machine starts up. A service is not strictly necessary to run a MySQL server but without it you would have to manually start/stop the server. You can create as many service entries for a server as you want, but usually it makes no sense to create more than one. Since there can be more than one server (each with an own service) on the box it is highly recommended to give the services speaking names (like MySQL57 for a 5.7 MySQL server, MySQL56 or a 5.6 server etc.).
MySQL Workbench connections are entries that store parameters to connect to a MySQL server. They are not tied to a specific service in general (you can connect to a non-Windows server too), but might be associated with such a service if they are configured for a local server (on a Windows machine) and you want to be able to start/stop this server from within MySQL Workbench (see Administrator section). They use the service name to act on a particular service (you configure that in the Connection manager, System profile). On Windows MySQL Workbench attempts to find installed service (to help new users to get up and running quicker) when it is started and no connection is currently defined. It then automatically creates a connection for each service it found (it uses "mysql*" as pattern to search for services). You can retrigger this process by removing all connections and restart MySQL Workbench.
The created connnections (like the “Local instance MySQL”) appear as tile you can simply click to open that connection. Be aware that clicking on the rightmost 1/4 of the tile will open an information popup only, not open the connection as such.
Now to your question:
What is the difference between the windows service MySQL57, the
connection MySQL57, and Local instance MySQL57?
I don't understand where you see a connection MySQL57. There's usually only a service MySQL57 and connection in MySQL Workbench to the server this service is for involved.
I'm also not sure why this is such a dreadful process for you. Especially for Windows a lot of effort has been put into an easy installation experience. The normal process should be like:
Download the MySQL Installer
Run the installer. Select a typical scenario from the presets or select the individual products you want to have installed, say, a MySQL server + MySQL Workbench.
The installer will download and install the selected components.
Configure your server (like giving it a proper service name, TCP/IP port etc.).
After finishing the installer you start MySQL Workbench and can immediately work with your new server (as described above MySQL Workbench picks up the service automatically).
Depending on your download speed this should be something like a 5 - 10 mins. process.
HTH
with due respect sir , windows service mysql is a service created by windows upon successful to help you start / stop mysql. You can use Local Instance Mysql57 without a glitch, as that is just a way of MySQL representing your local mysql instance in the workbench.
I would like to replace MySQL 5.1 on my Debian Lenny 32-bit server to Percona Server with XtraDB. The main reason is better performance of Percona.
It's production server with many services running. Many other software may depend on mysql-client and other mysql shared libraries.
Is it safe to replace MySQL?
By "safe" I mean: 1. remove mysql, 2. install percona 3. everything works as before
Will it break dependencies in third party software?
Will it require to change configuration of third party software (ie. socket path, server port, shared libraries path)?
Will it require to install trillion of additional packages?
I really don't want situation with broken libraries, missing or incompatible header files and so on
We are currently in the process of upgrading from MySQL 4.1 to Percona Server 5.5 at work and Percona is as they say on their site a complete drop-in replacement for MySQL, the binaries use the same names, it uses the same libraries, same config file placement, takes the same parameters and it has exactly the same SQL syntax. They should also be data file compatible on the same version (MySQL 5.1 to Percona 5.1, etc) but it's nothing I've personally verified.
You are able to do an apt-get install percona-server-server-5.1 after adding their repositiories and it will automatically replace MySQL because it marks it as a conflicting package. But you must take an SQL dump of your database first, of course.
We currently have replication set up from a MySQL 4.1 master to a couple of Percona 5.5 slaves and have had no problems inserting SQL dumps either.
... don't know.
In my experience, the only way is to make up a copy of your existing setup on an old machine and run some tests. Then swap over to the new DB and run same tests again.
I just swapped a set of applications from Tomcat 5 to Tomcat 6 and in theory, with one or two tweaks, all should have worked fine. First time I tried it OpenJava was installed and the garbage collection fouled things up. Second time around with Sun Java, some dodgy date handling fouled things up and had to be corrected, seems to run fine now.
I've been trying to setup a MySQL Cluster for a few days using the MySQL Cluster Manager on 3 Ubuntu nodes (3 identical VM instances with 1GB RAM each).
I've followed the video on MySQL Cluster Manager on the MySQL site. There's not much other documentation/tutorials on it (probably because it's a commercial product).
I start the cluster and show the status, but the mysqld nodes never start, they just remain as "added". If I install mysql-server using "sudo apt-get install mysql-server" then I get the normal local server running and the nodes register as "started", but I can't see how to connect to the cluster rather than the individual MySQL servers running on the mysqld nodes.
I'm also at a loss as to how the Java connector for MySQL Cluster is organised, it appears that there are multiple libraries so I don't even know which library I need or how to get them (some are created when compiling MySQL Cluster???). Could someone please explain how the connectors work to interact with NDB from Java and how to get them?
Thanks for any answers.
First of all, the official documentation for MySQL Cluster Manager can be found by navigating to the Cluster documentation on dev.mysql.com (called "MySQL Cluster Manager"). You are correct that MySQL Cluster Manager is commercial software although MySQL Cluster itself is available under a commercial or GPL license.
It sounds as though you've already configured the agents and have them running and so if you want to get a Cluster up and running quickly then refer to this simple worked example of using MySQL Cluster Manager
In terms of understanding why the MySQL Servers (mysqlds) are not starting up, there aren't many clues in your question and so we need to narrow it down (one reason could be if you had multiple mysqlds defined on the same host that are trying to use the default port (3306)).
To check what the manager has been doing, take a look in the file called mysql-cluster-manager.log. You can adjust the level of logging using the cluster manager configuration file.
To see what MySQL Cluster itself thinks has happened, check the directories storing the cluster data files (if you haven't over-written the defaults then this would be under /clusters/ and then you'll see a directory for each node in the cluster). The first one to check is ndb__cluster.log and other logs that you'll find in the "data" sub-directory of the id associated with the ndb_mgmd node. There will also be per-node log files so also check the mysqld_out.err and mysqld_out.log files stored in the data directory associated with mysqld node-ids.
Most important point is do not use the mysqld that gets installed with "sudo apt-get install mysql-server" as this version will not be compatible with MySQL Cluster - always use the binaries that come with the MySQL Cluster tar ball (or if using Cluster Manager that should be transparent to you anyway.
Note that if you want to get MySQL Cluster up and running on a single host without MySQL Cluster Manager then refer to the quick-start guide located on the MySQL Cluster download site (on mysql.com rather than e-Delivery).
For the java access, try out this MySQL Cluster ClusterJ tutorial.