As while using Laravel, we have an option to seed our database or create tables anytime, like
class UsersTableSeeder extends Seeder
{
public function run()
{
User::truncate();
User::create([
'username' => 'Junaid',
'email' => 'darulehsan03#gmail.com',
'password' => '1234'
]);
User::create([
'username' => 'Junaid Farooq',
'email' => 'aba#bcd.com',
'password' => '4321'
]);
}
}
we can seed our database anytime, but what if we have a large no of rows in our table, which are not being seeded but added by the users , then how can we put it that way, like a Seeder file so, anytime at anyplace , we can load all those rows through our Seeder File?
not asking about to save .SQL file and then import or Export it, But a way to backup them in a Seeder file
I don't know of any method of backing up the data to a seed file. You can, like you've already said, export and import your data.
There are also a couple of packages available to backup and restore a database.
laravel-backup, which seems to be a Laravel-specific package that allows you to backup your database and restore it.
database-backup, which is framework agnostic but does come with a Laravel service provider for easier integration with Laravel.
Both seem to allow you to backup and restore from Amazon S3. Having used neither I can't say which is better or why. You'll have to try both out and make that decision for yourself.
Related
All my previous projects used DatabaseCleaner, so I'm used to starting with an empty DB and creating test data within each test with FactoryGirl.
Currently, I'm working on a project that has a test database with many records. It is an sql file that all developers must import in their local test environments. The same DB is imported in the continuous integration server. I feel like having less control over the test data makes the testing process harder.
Some features allow their tests to focus on specific data, such as records that are associated to a certain user. In those cases, the preexisting data is irrelevant. Other features such as a report that displays projects of all clients do not allow me to "ignore" the preexisting data.
Is there any way to ignore the test DB contents in some tests (emulate an empty DB and create my own test data without actually deleting all rows in the test DB)? Maybe have two databases (both in the same MySQL server) and being able to switch between them (e.g., some tests use one DB, other tests use the other DB)?
Any other recommendations on how deal with to this scenario?
Thank you.
I would recommend preserving your test_database, and the 'test' environment as your 'clean' state. Then you could setup a separate database that you initially seed as your 'dirty' database. A before hook in your rails_helper file could also be setup with something like this:
RSpec.configure do |config|
config.before :each, type: :feature do |example|
if ENV['TEST_DIRTY'] || example.metadata[:test_dirty]
ActiveRecord::Base.establish_connection(
{
:adapter => 'mysql2',
:database => 'test_dirty',
:host => '127.0.0.1',
:username => 'root',
:password => 'password'
}
)
end
end
end
Your database.yml file will need configurations added for your 'dirty' database. But I think the key here is keeping your clean and dirty states separate. Cheers!
I have found that adding the following configuration to spec/rails_helper.rb will run all DB operations inside tests or before(:each) blocks as transactions, which are rolled back after each test is finished. That means we can do something like before(:each) { MyModel.delete_all }, create our own test data, run our assertions (which will only see the data we created) and after the end of the test, all preexisting data will still be in the DB because the deletion will be rolled back.
RSpec.configure do |config|
config.use_transactional_fixtures = true
end
Question
Are there any existing tools able to generate an implementation of Doctrine\DBAL\Migrations\Provider\SchemaProvider reflecting the state of an existing database?
I googled for a while and looked at similar questions in SO but i couldn't find an answer. They are all related to Symfony + Doctrine ORM.
Why do i need this?
I want to use doctrine migrations to manage/track changes on an existing database. I cannot jump into using doctrine ORM because I would need to refactor the database and it would break other ( non php ) applications that depend on it.
I know i can use migrations without ORM, but i need to provide a concrete implementation of Doctrine\DBAL\Migrations\Provider\SchemaProvider (documentation), in my case this would mean to rewrite the entire database.
If i had the SchemaProvider generated for the first time to reflect the state of the database. Eg
<?php
use Doctrine\DBAL\Schema\Schema;
use Doctrine\DBAL\Migrations\Provider\StubSchemaProvider;
final class MySchemaProvider implements SchemaProvider {
public function createSchema()
{
$schema = new Schema();
$table = $schema->createTable('foo');
$table->addColumn('id', 'integer', array(
'autoincrement' => true,
));
$table->setPrimaryKey(array('id'));
//and so on for the rest of the databse...
return $schema;
}
}
I would be able to edit the generated class and create migrations with:
$ ./doctrine migrations:diff
i have now new structure of my database, but i need to import the old data in the new format. For that reason i want to use the Laravel seeder, but i need somehow to connect to the old database and make select queries and to tell the seeder how to put the data in the new database.
Is that possible ?
Try:
Examples:
php artisan iseed my_table
php artisan iseed my_table,another_table
Visit: https://github.com/orangehill/iseed
Configure your laravel app to use two mysql connections (How to use multiple database in Laravel), one for the new database, the other for the old one.
I'll fake it like old and new.
In your seeds read from the old database and write into the new.
$old_user = DB::connection('old')->table('users')->get();
foreach ($old_users as $user) {
DB::connection('new')->table('users')->insert([
'name' => $user->name,
'email' => $user->email,
'password' => $user->password,
'old_id' -> $user->id
// ...
]);
}
Make sure to add messages while seeding like $this->command->info('Users table seeded'); or even a progress bar (you can access command line methods) to know at which point of the import you are.
Download package from
Git repo : https://github.com/orangehill/iseed
then update below file src/Orangehill/Iseed/IseedCommand.php
Add below code at line number 75
// update package script
if($this->argument('tables') === null){
$tables = Schema::getConnection()->getDoctrineSchemaManager()->listTableNames();
}
and update getArguments method in same file with below code
array('tables', InputArgument::OPTIONAL, 'comma separated string of table names'),
and then run php artisan iseed so it will get all the tables from your existing db and start creating seeders for all tables
Grant: this works
I have the following puppet code:
mysql_grant {'my-user-name#1.2.3.4/my-database-name.*':
ensure => 'present',
options => ['GRANT'],
privileges => ['SELECT', 'INSERT', 'DELETE', 'UPDATE'],
table => 'my-database-name.*',
user => 'my-user-name#1.2.3.4',
}
and that does grant the permissions I expect.
Revoke: this doesn't work
If I change my mind and say this:
mysql_grant {'my-user-name#1.2.3.4/my-database-name.*':
ensure => 'absent',
options => ['GRANT'],
privileges => ['SELECT', 'INSERT', 'DELETE', 'UPDATE'],
table => 'my-database-name.*',
user => 'my-user-name#1.2.3.4',
}
I note that it doesn't revoke permission (not even if I change s/GRANT/REVOKE/). Any pointers on how to automate revocation? I haven't been able to find it in the manual or by googling.
Repeat: I'm lost without copy and paste
Now suppose I want to permit access from several hosts. My puppet-fu fails me on how to not repeat the block (i.e., just copy-paste with different IP addresses). I'm sure puppet defines tools for this, but I've not figured out that part yet.
Thanks for any pointers!
For the repeat part I can think of two ways:
puppetDB
hiera
PuppetDB
Whenever you want the fact of a node to do something on a second node, use puppetDB. This is called exported resources. This is also explained in the puppet-mysql documentation.
Example1: Add the SSH Hostkeys of all machines to the known_keys of all other machines.
Example2: Add all machines to monitoring, creating their own host definition.
Example3: On a certain class of machine, allow them to connect to MySQL.
In each case, you first install puppetDB via the puppet-puppetdb module. You will need puppet4 for this. PuppetDB will only start if you have 8+ GB of memory.
You then have to write the resource export and the resource import. On all nodes that have a fact that you want (like ip / fqdn), you write the export:
##mysql_grant {"my-user-name#${::ipaddress}/**my-database-name.*":
ensure => 'absent',
options => ['GRANT'],
privileges => ['SELECT', 'INSERT', 'DELETE', 'UPDATE'],
table => 'my-database-name.*',
user => "my-user-name#${::ipaddress}",
}
The '##' creates the export. Note that the exported resource is lower case. Also note the double quote instead of single quote whenever a variable is used.
What will happen whenever a node sees this, is that it will fill out this exported resource with its fact (in this case ::ipaddress), and send it to puppetDB. You can either add this part to all nodes you want to grant access, partially defeating its purpose, or you can have a manifest that is applied to all nodes and do something along the lines of:
if $::fqdn include? 'app'{
##mysql_grant {"my-user-name#${::ipaddress}/**my-database-name.*":
ensure => 'absent',
options => ['GRANT'],
privileges => ['SELECT', 'INSERT', 'DELETE', 'UPDATE'],
table => 'my-database-name.*',
user => "my-user-name#${::ipaddress}",
}
}
Then you need to write an import statement on the node that should apply this.
Mysql_grant <<| |>>
Please note the upper case.
Another quick example, which we apply to all our linux nodes:
# collect all the public host RSA keys for known hosts
##sshkey { $hostname:
ensure => present,
type => 'rsa',
host_aliases => [$::ipaddress, $::fqdn],
key => $sshrsakey,
}
# and populate known_hosts
Sshkey <<| |>>
#https://projects.puppetlabs.com/issues/21811
file { '/etc/ssh/ssh_known_hosts':
ensure => present,
path => '/etc/ssh/ssh_known_hosts',
mode => '0644',
}
Hiera
Hiera is build for exactly this purpose, to seperate code from data. Please refer to the hiera documentation for how to set it up.
What you end up doing is that you will create a yaml file that has all your data in it:
mysql::grants:
db1:
username: my-user-name
database: my-database-name
ip: 1.2.3.4
ensure: present
options:
- GRANT
privileges:
- SELECT
- INSERT
- DELETE
- UPDATE
table: my-database-name.*
db2:
username: my-user-name
database: my-database-name
ip: 1.2.3.5
ensure: present
options:
- GRANT
privileges:
- SELECT
- INSERT
- DELETE
- UPDATE
table: my-database-name.*
Then you just go ahead and put this in your mysql node (although creating a small module would be cleaner):
$grants = hiera('mysql::grants', undef)
create_resources('mysql::grant', $grants)
Puppet will parse all of hiera, then creating a grant for every db found.
Try using mysql_grant on a new user, then using puppet apply with the -d (debug) and -v (verbose) options on your manifest.
This should give you a load of output that shows what it's doing. What it will be doing is running sql commands on your database such as
grant all on db.* to user
These will also be shown when you run
show grants for user
Then change to 'absent', and repeat.
Now you know exactly what SQL commands puppet is running on your DB.
Then you can try those commands directly in the DB to see if they do what you expect.
Note: using ensure => 'absent' is the correct thing to do to remove permissions, changing grant to revoke won't help.
In cakephp I want to be able connect to a different database from one action on the site. The action determines which database and host to connect to. Using cakephp 1.3.
Ive seen where you can change the db connection in beforeFilter for a controller, but I want to be able handle this from the action, because that is where I find the database and/or host, that I need to connect to.
I can write my own SQL inside there. I don't need to go through models. Just want to do a simple add/update SQL statement.
You can easily configure more than one database connection to use in your app.
In config/database.php, create another variable for your database configuration, in addition to the existing $default:
var $otherDatabase = array(
'driver' => 'mysql',
// more settings...
);
Then, in your model, set $this->useDbConfig = 'otherDatabase' or in your controller $this->MyModel->useDbConfig = 'otherDatabase'. Any subsequent find()s will use the configured database.