In my Yii2/PHP project I need to have both databases integrated:
MySQL for meta data, Web-UI, ...
InfluxDB for measurement data (heavy loads of timeserie data)
To reduce complexity I'd like to start with MySQL only and add InfluxDB later.
My idea is to create an abstraction/superclass for both databases (for measurement data only) which allow to do implementation and perform tests with MySQL and enable speedup with InfluxDB at a later stage in the project.
The abstraction should have methods for:
database connection management
writing data
reading data (raw data, aggregations)
Since I am no InfluxDB expert (yet): Does this architecture make sense or are both datamodels and schemes fundamentally different so an abstraction would be worthless? Are there projects out there to learn from?
First, you need to configure your databases like below this example take two mysql db:
return [
'components' => [
'db1' => [
'class' => 'yii\db\Connection',
'dsn' => 'mysql:host=localhost;dbname=db1name', //maybe other dbms such as psql,...
'username' => 'db1username',
'password' => 'db1password',
],
'db2' => [
'class' => 'yii\db\Connection',
'dsn' => 'mysql:host=localhost;dbname=db2name', // Maybe other DBMS such as psql (PostgreSQL),...
'username' => 'db2username',
'password' => 'db2password',
],
],
];
Then you can simply:
// To get from db1
Yii::$app->db1->createCommand((new \yii\db\Query)->select('*')->from('tbl_name'))->queryAll()
// To get from db2
Yii::$app->db2->createCommand((new \yii\db\Query)->select('*')->from('tbl_name'))->queryAll()
If you are using an active record model, in your model you can define:
public static function getDb() {
return Yii::$app->db1;
}
//Or db2
public static function getDb() {
return Yii::$app->db2;
}
Then:
If you have set db1 in the getDb() method, the result will be fetched from db1 and so on.
ModelName::find()->select('*')->all();
I'm not sure trying to fit MySQL and InfluxDB in the same mould would make a lot of sense.
A better approach IMHO, would be to have some sort of helper class for your computations (i.e.: Stats::getViews(), Stats::getVisitors(), ..) first using MySQL, and later rewrite it to use InfluxDB, keeping the same methods signatures and responses formats.
Related
I am a newbie in YII2 and started learning this on my own. Recently I was working on YII2 models. While learning some pre built model files I noticed that some model files are extends through yii\base\Model where as some from \yii\db\ActiveRecord.
So I want to know the reason when one should use active records and when model i.e. we can do the query in model too so why is there need of active records. Which are the tasks can be done by active records but not by models. In simple words I want to know specific work of both model and active records.
ActiveRecord is a Model that uses a database engine to store the model(s) data.
yii\base\Model is a Model that does not specify how the data is being stored.
Eg a Model could be without a data table and have the actual data stored inside the class code instead of a database. A good example is in the yii2-basic app - the User model here.
I has the data stored in the class code as:
private static $users = [
'100' => [
'id' => '100',
'username' => 'admin',
'password' => 'admin',
'authKey' => 'test100key',
'accessToken' => '100-token',
],
'101' => [
'id' => '101',
'username' => 'demo',
'password' => 'demo',
'authKey' => 'test101key',
'accessToken' => '101-token',
],
];
Active records use for working with data tables and forms, but yii\base\Models for only for forms.
I saw in my debug menu that I have some duplicate select queries. In particular, this is the one that I got 4 times SELECT * FROM page_adminlang WHERE (language=bg) AND (page_id=1) in 4 different files. Is there a right way to prevent such situations and is this big hit on the site speed performance? Shall I retrieve the row in some variable like $page = PageAdmin::findOne(1) and than call it where I need and is this the right way? I red other articles but they were mainly for duplicate rows on insertion. Not familiar with MySQL performance tuning but want to go deeper in this area.Thank you in advance!
Option 1 - Configure MemCache Component
An Easier method is to enable db caching, then it won't matter, put a 5second duration at first.
In your component db connection settings set the following properties:
'components' => [
'db' => [
'class' => 'yii\db\Connection',
'dsn' => 'mysql:host=localhost;dbname=dbname',
'username' => 'root',
'password' => 'password',
'charset' => 'utf8',
....
'enableQueryCache' => true,
'queryCacheDuration' => 5, // five seconds
....
]
]
See:
https://www.yiiframework.com/doc/guide/2.0/en/caching-data
https://www2.0/yii-db.yiiframework.com/doc/api/-connection
Option 2 - Custom query with cache
public function getResults()
{
// Cache expires every x seconds (60sec/min * 60min/hr * 6hrs)
$duration = 60*60*6;
$sql_query = "SELECT * FROM some_table";
return Yii::$app->db->createCommand($sql_query)->cache($duration)->queryAll();
}
I don't usually face that issue, but if I have to retrieve something from the database multiple times (and I know that won't change), I do something like that in my model
private $_myData = null;
public function getMyData() {
if ($this->_myData !== null) return $this->_myData;
$this->_myData = //query your data;
return $this->getMyData();
}
What I am trying to do is to cache all the results in a MySQL table that seldom changes, so as to minimize calls to database and increase query speed. There are about 100k records in there.
Is there a library that can sync changes made in this table, like say when a record is updated or inserted, the redis cache will also be invalidated and updated.
I have seen one for elasticsearch, but nothing for redis.
From this page:
Yii copying data from one model to another
There is this comment:
You can get all models attributes by:
$data = $model->attributes;
and assign them to another model
$anotherModel = new AnotherActiveRecord();
$anotherModel->setAttributes($data);
now another model will extract whatever it can from $data
I'm curious, can a Redis cache also "mirror" the data from a database table in a similar way?
Or is this just a bad idea overall, and its better off caching the query as it comes along, or is there a better way.
You can enable caching based on https://www.yiiframework.com/doc/guide/2.0/en/caching-data
[
'components' => [
'cache' => [
'class' => 'yii\redis\Cache',
'redis' => [
'hostname' => 'localhost',
'port' => 6379,
'database' => 0,
]
],
],
]
and then use Query Caching which natively defined on query builder level
$result = $db->cache(function ($db) {
// the result of the SQL query will be served from the cache
// if query caching is enabled and the query result is found in the cache
// ... perform SQL queries here ...
});
Also you can use Cache Dependencies based on your table (some criteria like if max(updated_at) is changed or not).
// Create a dependency on updated_at field
$dependency = new yii\caching\DbDependency(['sql' => 'select max(updated_at) from my_table']);
$duration = 60; // cache query results for 60 seconds.
$result = $db->cache(function ($db) {
// ... perform SQL queries here ...
return $result;
}, $duration, $dependency);
My application use many databases, they are in same structure, but data has no relation between different databases. I need to change database via request params.
In the config can only setup dsn, but I want to change database dynamically.
How can I do that.
I got myself:
$mongo = Yii::$app->get('mongodb');
$mongo->options['db'] = 'foo';
The simplest solution is to defined multiple connections in the configuration:
'components' =>
[
...
'mongodb' =>
[
'class' => '\yii\mongodb\Connection',
'dsn' => 'mongodb://localhost:27017/database1',
],
'othermongodb' =>
[
'class' => '\yii\mongodb\Connection',
'dsn' => 'mongodb://localhost:27017/database2',
],
...
]
You can then access your connections with Yii::$app->mongodb and Yii::$app->othermongodb (or using the get()-method if you prefer). This also allows you to specify the correct database for the ActiveRecord classes that come from a different database:
class MyOtherDBMongo extends \yii\mongodb\ActiveRecord
{
public static function getDb()
{
return \Yii::$app->get('othermongodb');
}
}
I have quite the complex problem to tell you about Stackoverflow, you see I have three tables that I need to define associations between for an application. These tables are: engineers, tickets, and testcases. I need to have a $hasMany relationship between engineers and tickets, and a hasAndBelongsToMany relationship between testcases and tickets. Here is the catch, engineers and testcases are both on a mysql database, while tickets is on a sqlite3 database(trac) on a separate server. The separate server part is not an issue, because we have the server mounted on the same machine that my application is on. I am basically wondering how you would setup these relationships, so that as each model is loaded its dependencies are loaded as well. I will literally use just about any solution that gets the job done. I am using CakePHP by the way.
HABTM associations are not supported across multiple databases in CakePHP. In order to make the associations you will need to change the core. At least one person have achieved that. Look at his method.
You could just not define the association and do the querying manually. I.e., you associate the Ticket model with its HABTM join table TestcaseTickets in a belongsTo relationship (assuming they're both in the SQLite database) and query it manually:
$testcases = $this->Testcase->find(…);
$tickets = $this->Ticket->TestcaseTickets->find('all', array(
'conditions' => array(
'TestcaseTickets.testcase_id' => Set::extract('/Testcase/id', $testcases)
)
));
It takes away a bit of convenience, but doesn't make a big difference in the end, especially if you do this automatically in the afterFind callback of the Testcase model.
I don't think it's a problem to work with 2 (or more) database connections in CakePHP.
Basically you need 2 connection strings in your /app/config/database.php i.e.:
class DATABASE_CONFIG {
var $mysql = array(
'driver' => 'mysql',
'persistent' => false,
'host' => 'localhost',
'login' => 'user',
'password' => 'password',
'database' => 'database',
'prefix' => '',
'encoding'=>'utf8'
);
var $sqlite = array(
'driver' => 'sqlite',
'persistent' => false,
'host' => 'localhost',
'login' => 'user',
'password' => 'password',
'database' => 'database',
'prefix' => '',
'encoding'=>'utf8'
);
}
btw. I am not quite sure about sqlite driver, but it should be this way.
And finally, you need to set to each model which connection to use. This could be done with:
class tickets extends AppModel {
...
var $useDbConfig = 'sqlite';
...
}
And especially if you don't use any special SQL "hacks" it should work.