I'm new to Couchbase. Does anyone know how to connect between Couchbase (CB) servers in order to migrate data? I want to migrate data from a production CB server to my local server.
I found here how to migrate between buckets, but within the same server.
Also, I could do the migration between different servers using a backend application (I'm using C#.Net) using N1QL, but I want to learn how to do the server to server migration since it is a pretty standard and common operation .
Many thanks in advance.
Do you need continuous migration? Or backup/restore would be enough?
If latter, you can use cbbackup/cbrestore tools http://developer.couchbase.com/documentation/server/current/backup-restore/backup-restore.html
Either cbbackup and/or cbrestore or just turning on XDCR to get the changes from production to your local server would work too.
Definitely some security implications on doing this, but that's up to you to figure out! :)
Thanks to #Ade Stringer who finally gave me the best solution, which is to use the cbtransfer tool. This tool simply needs the source and target urls of the servers (and the names of the buckets), which is ideal since -- in general-- one doesn't have access to the file system of the servers.
Here is a sample call:
cbtransfer http://10.10.10.68:8091 http://localhost:8091 -b SourceBucketName --bucket-destination TargetBucketName
Note that the first parameter is the source CB server (http://10.10.10.68:8091) and the second one is the target CB server (http://localhost:8091). The value of the -b parameter is the name of the source bucket and the value of the --bucket-destination parameter is the name of the target bucket.
In order to run this command in Windows, you must first go to the following folder:
C:\Program Files\Couchbase\Server\bin\
If one prefers to use the cbbackupmgr and cbbackup tools which were mentioned in the other answers, one needs to have access to the file system, which in my case was not possible. But still both tools are also useful nevertheless and I appreciate their answers.
Related
So I have an SQLite3 database with Sequelize on a VPS and a Website I hosted on Hostinger.
Is it possible to access and edit the data in the database from the website?
Any help is appreciated.
[my code is in nodejs]
Yes and No.
Easy and simple answer - now;
Longer - yes, but you would need to map the directories, so server running your sequelize will think it accesses normal directory.
You can download the database, edit it and then send it back. sqlite3 is file database, you dont really connect to it (usually)
SQLite is a database format based on a single file. It's simple but not massively performant. It has no networking features.
The SQLite client needs to have access to the filesystem that the database lives on.
If they are on different computers, then you could do it by using some kind of network filesystem (e.g. via fuse) but this would be really slow.
A better approach would be to run a webservice on the computer the file is stored on, and access the database through there.
I am working at a company that has some CRM software running in a remote Windows XP server that uses a SQLAnywhere 9 db to store its data; I have access to this remote server with an administrator account.
I would like to extract the db into a .sql file so that I can run the db locally on my machine without affecting the running db in the server (since it is key for the company's day to day operation).
The reason I need this is that we are going to test some BI Software and we need data from this database to test it, but we don't know the structure of the database since the developers of the CRM software didn't give us any documentation on it. So we need to have the database locally so that, without affecting the running CRM, we can:
understand the structure by looking at the DDL
make queries to it to get sample data
I researched a bit, and the most common solution to my problem was to use dbunload on the remote server to unload the db into a reload.sql file that contained what I needed. But most tutorials on the subject mention that I have to stop the db first (which would be catastrophic). If this is the only option, then I guess I am willing to do it on the weekend when the CRM is not used, but I wanted to know if there was another solution first.
If there is no other solution, can you point me to where I can find the proper and safer way to do this?
I have researched a lot, but prior to this day I have never even heard of SQLAnywhere, so I really need all the help I can get. My main concern is doing something that impacts negatively the CRM software.
Thank you.
You can run dbunload across the network, you just have to tell it to do an "external" unload. The default is to do an internal unload which would only work from the machine where the database server is running.
I don't have SQL Anywhere 9 documentation right now to look up the exact switch, but dbunload -? should show you all the possible switches.
Edit:
-an will create a new database and load the data and schema from another data
-xi switch will do external unload and internal reload.
-c parameters to connect to your remote database
Firstly, my aim is to create a database to store statistical information about my wiki articles, that is derived using extensions that I have written for my MediaWiki server. Then via another extension on my server, I wish to access this secondary database and use the information to edit the output of the final article page.
I wanted to try and make a separate database to avoid introducing errors to the existing MediaWiki schema. Is this possible? I can't find any information about running an additional mySQL database alongside a MediaWiki installation.
I feel as though the better solution may be to add a table to the existing schema, but I'm not sure how to deal with the existing infrastructure without breaking it. I believe I can just use standard CREATE commands but mySQL is unfortunately not my strong point.
Thanks
Hope i understood your problem. say you have media wiki server or a mysql server running on port 3308 and you want to create another mysql instance.
Follow this step:
bkp my.cnf file
start first instance on port 3308
chnage my.cnf settings & make it run on 3307
start second instance of mysql
If you have good h/w,swap/RAM, you can easily create such 4-5 instances & all will be running good.
You can run a separate database along with the primary MediaWiki database. You just have to open a connection to it specifically using the PHP MySQL API.
Adding a new table to the existing database though is really easy and probably your best bet. You will be able to take advantage of the built-in database load balancing if your site ever gets big. To learn how to create a new table from an extension, check out this page of documentation.
I'm working with MySQL databases.
To simplify the problem, let's say I have two environments : the local one (development) and the remote one (production mode).
In the database, I have some tables that contain configuration data.
How can I automate cleanly the delivery from the development mode to the production mode when I modify the database schema and the configuration tables content ?
For instance, I dot it manually by doing a diff between the local and remote databases. But, I find that method not so clean and I believe there is good practice allowing that.
This might be helpful in cases where you have multiple environments and multiple developers making schema changes very often and using php.. https://github.com/davejkiger/mysql-php-migrations
Introduce parameter "version" for your database. This version should be written somewhere in your code and somewhere in your database. Your code will work with database only if they have equal versions
Create a wrapper around your MySQL connection. This wrapper should check versions and if versions are not compatible, it should start upgrade.
"Upgrade" is a process of sequential applying the list of *.sql files with SQL commands, which will move your database from one state to another. It can be schema changes or data manipulation commands.
When you do something with database, do it only through adding new *.sql file and incrementing version.
As a result, when you deploy your database from development enviroment to production, your database will be upgraded automatically in the same way as it was upgraded during development.
I've seen LiquiBase http://www.liquibase.org/ a lot in Java environments.
In most of my projects I use sqlalchemy(a Python tool to manage db plus an ORM). If you have some experience(little more than beginner) with Python I highly recommend using it.
You can check this tool with a little help of that. This is also very useful for migrating your db to other rdbms(for example mysql to postgres or oracle).
The title is self explanatory. Is there a way of directly doing such kind of importing?
The .BAK files from SQL server are in Microsoft Tape Format (MTF) ref: http://www.fpns.net/willy/msbackup.htm
The bak file will probably contain the LDF and MDF files that SQL server uses to store the database.
You will need to use SQL server to extract these. SQL Server Express is free and will do the job.
So, install SQL Server Express edition, and open the SQL Server Powershell. There execute sqlcmd -S <COMPUTERNAME>\SQLExpress (whilst logged in as administrator)
then issue the following command.
restore filelistonly from disk='c:\temp\mydbName-2009-09-29-v10.bak';
GO
This will list the contents of the backup - what you need is the first fields that tell you the logical names - one will be the actual database and the other the log file.
RESTORE DATABASE mydbName FROM disk='c:\temp\mydbName-2009-09-29-v10.bak'
WITH
MOVE 'mydbName' TO 'c:\temp\mydbName_data.mdf',
MOVE 'mydbName_log' TO 'c:\temp\mydbName_data.ldf';
GO
At this point you have extracted the database - then install Microsoft's "Sql Web Data Administrator". together with this export tool and you will have an SQL script that contains the database.
MySql have an application to import db from microsoft sql.
Steps:
Open MySql Workbench
Click on "Database Migration" (if it do not appear you have to install it from MySql update)
Follow the Migration Task List using the simple Wizard.
I did not manage to find a way to do it directly.
Instead I imported the bak file into SQL Server 2008 Express, and then used MySQL Migration Toolkit.
Worked like a charm!
In this problem, the answer is not updated in a timely. So it's happy to say that in 2020 Migrating to MsSQL into MySQL is that much easy. An online converter like RebaseData will do your job with one click. You can just upload your .bak file which is from MsSQL and convert it into .sql format which is readable to MySQL.
Additional note: This can not only convert your .bak files but also this site is for all types of Database migrations that you want.
Although my MySQL background is limited, I don't think you have much luck doing that. However, you should be able to migrate over all of your data by restoring the db to a MSSQL server, then creating a SSIS or DTS package to send your tables and data to the MySQL server.
hope this helps
I highly doubt it. You might want to use DTS/SSIS to do this as Levi says. One think that you might want to do is start the process without actually importing the data. Just do enough to get the basic table structures together. Then you are going to want to change around the resulting table structure, because whatever structure tat will likely be created will be shaky at best.
You might also have to take this a step further and create a staging area that takes in all the data first n a string (varchar) form. Then you can create a script that does validation and conversion to get it into the "real" database, because the two databases don't always work well together, especially when dealing with dates.
The method I used included part of Richard Harrison's method:
So, install SQL Server 2008 Express
edition,
This requires the download of the Web Platform Installer "wpilauncher_n.exe"
Once you have this installed click on the database selection ( you are also required to download Frameworks and Runtimes)
After instalation go to the windows command prompt and:
use sqlcmd -S \SQLExpress (whilst
logged in as administrator)
then issue the following command.
restore filelistonly from
disk='c:\temp\mydbName-2009-09-29-v10.bak';
GO This will list the contents of the
backup - what you need is the first
fields that tell you the logical names
- one will be the actual database and the other the log file.
RESTORE DATABASE mydbName FROM
disk='c:\temp\mydbName-2009-09-29-v10.bak' WITH MOVE 'mydbName' TO
'c:\temp\mydbName_data.mdf', MOVE
'mydbName_log' TO
'c:\temp\mydbName_data.ldf'; GO
I fired up Web Platform Installer and from the what's new tab I installed SQL Server Management Studio and browsed the db to make sure the data was there...
At that point i tried the tool included with MSSQL "SQL Import and Export Wizard" but the result of the csv dump only included the column names...
So instead I just exported results of queries like "select * from users" from the SQL Server Management Studio
SQL Server databases are very Microsoft proprietary. Two options I can think of are:
Dump the database in CSV, XML or similar format that you'd then load into MySQL.
Setup ODBC connection to MySQL and then using DTS transport the data. As Charles Graham has suggested, you may need to build the tables before doing this. But that's as easy as a cut and paste from SQL Enterprise Manager windows to the corresponding MySQL window.
For those attempting Richard's solution above, here are some additional information that might help navigate common errors:
1) When running restore filelistonly you may get Operating system error 5(Access is denied). If that's the case, open SQL Server Configuration Manager and change the login for SQLEXPRESS to a user that has local write privileges.
2) #"This will list the contents of the backup - what you need is the first fields that tell you the logical names" - if your file lists more than two headers you will need to also account for what to do with those files in the RESTORE DATABASE command. If you don't indicate what to do with files beyond the database and the log, the system will apparently try to use the attributes listed in the .bak file. Restoring a file from someone else's environment will produce a 'The path has invalid attributes. It needs to be a directory' (as the path in question doesn't exist on your machine).
Simply providing a MOVE statement resolves this problem.
In my case there was a third FTData type file. The MOVE command I added:
MOVE 'mydbName_log' TO 'c:\temp\mydbName_data.ldf',
MOVE 'sysft_...' TO 'c:\temp\other';
in my case I actually had to make a new directory for the third file. Initially I tried to send it to the same folder as the .mdf file but that produced a 'failed to initialize correctly' error on the third FTData file when I executed the restore.
The .bak file from SQL Server is specific to that database dialect, and not compatible with MySQL.
Try using etlalchemy to migrate your SQL Server database into MySQL. It is an open-sourced tool that I created to facilitate easy migrations between different RDBMS's.
Quick installation and examples are provided here on the github page, and a more detailed explanation of the project's origins can be found here.