We have created a T3 extension where each record can have a couple of related event dates. The event dates are declared as IRRE recordings.
Now as the event dates getting more it takes up to 50 secs. to open a basic record in the Backend, frontend is fast as usual.
Right now there are 600 base records and 17K IRRE records. Things started to slow down at about 8K event dates.
Anyone an idea how to speed things up?
thx for your help
for fronteand enable lazy load in domain model and disable collaps all in tca for backend:
'config' => array(
'type' => 'inline',
'foreign_table' => 'tx_xxx_domain_model_yyy',
'foreign_field' => 'rrrr',
'foreign_sortby' => 'sorting',
'maxitems' => 9999,
'appearance' => array(
'expandSingle' => 1,
'collapseAll' => 1,
'levelLinksPosition' => 'top',
'showSynchronizationLink' => 1,
'showPossibleLocalizationRecords' => 1,
'useSortable' => 1,
'showAllLocalizationLink' => 1
),
IRRE wasn't the bottleneck of that performance issue.
We added a label_userFunc to display a formatted date as label of the IRRE entries but called the userfunction in TCA with
label_userFunc (doesn't work with IRRE)
but it must be called with
formattedLabel_userFunc (works with IRRE)
as only that one works with IRRE.
For other performance problems consider matin his answer:
in TCA set collapseAll = 1 and expandSingle =1
Related
I saw in my debug menu that I have some duplicate select queries. In particular, this is the one that I got 4 times SELECT * FROM page_adminlang WHERE (language=bg) AND (page_id=1) in 4 different files. Is there a right way to prevent such situations and is this big hit on the site speed performance? Shall I retrieve the row in some variable like $page = PageAdmin::findOne(1) and than call it where I need and is this the right way? I red other articles but they were mainly for duplicate rows on insertion. Not familiar with MySQL performance tuning but want to go deeper in this area.Thank you in advance!
Option 1 - Configure MemCache Component
An Easier method is to enable db caching, then it won't matter, put a 5second duration at first.
In your component db connection settings set the following properties:
'components' => [
'db' => [
'class' => 'yii\db\Connection',
'dsn' => 'mysql:host=localhost;dbname=dbname',
'username' => 'root',
'password' => 'password',
'charset' => 'utf8',
....
'enableQueryCache' => true,
'queryCacheDuration' => 5, // five seconds
....
]
]
See:
https://www.yiiframework.com/doc/guide/2.0/en/caching-data
https://www2.0/yii-db.yiiframework.com/doc/api/-connection
Option 2 - Custom query with cache
public function getResults()
{
// Cache expires every x seconds (60sec/min * 60min/hr * 6hrs)
$duration = 60*60*6;
$sql_query = "SELECT * FROM some_table";
return Yii::$app->db->createCommand($sql_query)->cache($duration)->queryAll();
}
I don't usually face that issue, but if I have to retrieve something from the database multiple times (and I know that won't change), I do something like that in my model
private $_myData = null;
public function getMyData() {
if ($this->_myData !== null) return $this->_myData;
$this->_myData = //query your data;
return $this->getMyData();
}
What I am trying to do is to cache all the results in a MySQL table that seldom changes, so as to minimize calls to database and increase query speed. There are about 100k records in there.
Is there a library that can sync changes made in this table, like say when a record is updated or inserted, the redis cache will also be invalidated and updated.
I have seen one for elasticsearch, but nothing for redis.
From this page:
Yii copying data from one model to another
There is this comment:
You can get all models attributes by:
$data = $model->attributes;
and assign them to another model
$anotherModel = new AnotherActiveRecord();
$anotherModel->setAttributes($data);
now another model will extract whatever it can from $data
I'm curious, can a Redis cache also "mirror" the data from a database table in a similar way?
Or is this just a bad idea overall, and its better off caching the query as it comes along, or is there a better way.
You can enable caching based on https://www.yiiframework.com/doc/guide/2.0/en/caching-data
[
'components' => [
'cache' => [
'class' => 'yii\redis\Cache',
'redis' => [
'hostname' => 'localhost',
'port' => 6379,
'database' => 0,
]
],
],
]
and then use Query Caching which natively defined on query builder level
$result = $db->cache(function ($db) {
// the result of the SQL query will be served from the cache
// if query caching is enabled and the query result is found in the cache
// ... perform SQL queries here ...
});
Also you can use Cache Dependencies based on your table (some criteria like if max(updated_at) is changed or not).
// Create a dependency on updated_at field
$dependency = new yii\caching\DbDependency(['sql' => 'select max(updated_at) from my_table']);
$duration = 60; // cache query results for 60 seconds.
$result = $db->cache(function ($db) {
// ... perform SQL queries here ...
return $result;
}, $duration, $dependency);
I am completely new to Drupal. I inherited a very ugly and incorrect code, unfortunately. In fact I would like to implement a proper login-with-facebook feature, which was totally mis-implemented. It tried to identify users by their email address, however, for some reason, upon login with Facebook, users logged in with the wrong user. I would like to identify the user based on Facebook ID, however, there was no column for that purpose in the database.
As a result, I have implemented a small script, which added a facebook_id and a facebook_token to the table representing the users. However, these new columns are not seen by the drupal_get_schema function in bootstrap.
If I do this:
$schema = drupal_get_schema("users");
echo var_dump($schema["fields"]);
It shows the fields except the two newly created fields. This way a SchemaCache object is initialized. I assumed that the schema might be cached. So I tried something different:
$schema = drupal_get_schema("users", true);
echo var_dump($schema["fields"]);
to make sure that drupal_get_complete_schema(true) will be called. However, the fields are not seen this way either. Is there a way I can tell Drupal to acknowledge the existence of the two newly created columns? If not: what should I do? Should I remove the two columns from the database table and use db_add_field("users", "facebook_id") and db_add_field("users", "facebook_token") respectively? If so, where should I call these?
Sorry if the question is too simple or I am misunderstanding these technologies, but I have tried to solve this for hours and I am at a loss, because this is my first drupal/bootstrap project and the source-code using these does not help me at all.
EDIT:
Since, at the time of this writing I have not received any answers apart from a tool recommendation which did not address my question, I have continued my research in the area. I removed the columns from the database to create them in a Drupal way. I have implemented this function in user.module:
function user_schema_alter() {
db_add_field('users', 'facebook_id', array(
'type' => 'varchar', //was initially a bigint, but Drupal generated a query which always crashed
'length' => 20,
'not null' => TRUE,
'default' => ".", //was initially -1, but Drupal generated a query which always crashed
));
db_add_field('users', 'facebook_token', array(
'type' => 'varchar',
'length' => 300,
'not null' => TRUE,
'default' => 'unavailable',
));
}
and I invoke it from altconnect.module, like this:
$schema = drupal_get_schema("users");
if (!isset($schema["fields"]["facebook_id"])) {
user_schema_alter();
}
It creates the columns, but later the existence of those columns will not be known about and subsequently an error will be thrown as the code will try to re-create them. Besides the fact that I had lost a lot of time until I realized that Drupal is unable to support bigint fields having -1 as their default value I had to conclude that with this solution I am exactly in the same situation as I were initially, with the difference that with this Drupal solution I will always get an exception if the columns already exist, because the schema will not be aware of them and subsequently, the code will always enter that if.
I fail to understand why is this so difficult in Drupal and I totally fail to understand why trying
db_add_field('users', 'facebook_id', array(
'type' => 'bigint',
'length' => 20,
'not null' => TRUE,
'default' => -1,
));
throws an exception due to syntax error. Maybe I should just leave this project and tell anyone who considers using Drupal to reconsider :)
I was able to find out what the answer is, at least for Drupal 6.
In user.install we need to do the following:
//...
function user_schema() {
//...
$schema['users'] = array(
//...
'fields' => array(
//...
'facebook_id' => array(
'type' => 'varchar',
'length' => 20,
'not null' => TRUE,
'default' => ".",
),
'facebook_token' => array(
'type' => 'varchar',
'length' => 300,
'not null' => TRUE,
'default' => 'unavailable',
),
//...
),
//...
}
//...
/**
* Adds two fields (the number is some kind of version number, should be the biggest so far for the module)
*/
function user_update_7919() {
db_add_field('users', 'facebook_id', array(
'type' => 'varchar',
'length' => 20,
'not null' => TRUE,
'default' => ".",
));
db_add_field('users', 'facebook_token', array(
'type' => 'varchar',
'length' => 300,
'not null' => TRUE,
'default' => 'unavailable',
));
}
When this is done, log in with the admin user and go to http://example.com/update.php
There you will see the thing to be updated. Run it. If you wonder why do we have to do all this, why don't we run some scripts directly, then the answer is that this is how Drupal operates. It simplifies your life by making it complicated, but do not worry, while you wait for update.php to do the updates which would take less than a second if it was your script, you can ponder about the meaning of life, quantum-mechanics or you can try to find out the reason this is so over-complicated in Drupal and you can go out for a walk. When you focus again, if you are lucky, update.php has completed its job and the two columns should be in the database.
I'm new to Drupal.
So I was wondering if you can help me.
I saw a lot of documents regarding the Drupal API mysql thing-y and It's been bugging me that I have to study once more to finish my work done.
And here's the documentation that I'm applying to my problem
Regarding my problem about the INSERT function, I have this table entitled embed
and here is my data from the table embed.
Then on my basic page I'm trying to insert a query.
$id = db_insert("embed")
->fields(array(
'uid' => 1,
'fbp_id' => 22222,
'prom_stat' => 3333,
'status' => 1,
))
->execute();
Instead of inserting a data to the table, it outputs an error like this.
Anyone knows the solution for this stuff? I'm really confused about this right now.
As #steve has suggested in the comment, the issue is not on drupal side but on MySql side. You need to modify your insert code to
$id = db_insert("embed")
->fields(array(
'uid' => 1,
'fbp_id' => 22222,
'prom_stat' => 3333,
'status' => 1,
'prom_id' => 0,
'sweep_stat' => 0,
'sweep_id' => 0,
'comp_id' => 0,
'comp_stat' => 0,
'polls_stat' => 0,
'polls_id' => 0
))
->execute();
Since i can see that your MySql table already contains value, i assume the earlier inserts where done by explicitly providing all values, instead of relying on default values for the field in MySql configuration.
Whenever you have a PDOException you should read it carefully for clues. These kind of errors are really very verbal and gives lots of pointers to resolve the issues. For example in your case,
'prom_id' doesn't have a default value
explains a lot.
I was using the sfPropel15Plugin with Symfony 1.3.11, and everything was working great.
I decided to upgrade the plugin to the sfPropelORMPlugin, which uses Propel 1.6.
I did the normal steps to install the plugin. I was able to get the app back up and working
in most cases. However, I find that certain form filter criteria I was using before are now breaking and not generating valid sql.
Here's an example that works in 1.5, but not 1.6:
// select only users who are sales reps on an existing quote
$c = new Criteria();
$c->addJoin(sfGuardUserPeer::ID, sfGuardUserProfilePeer::USER_ID);
$c->addJoin(QuotePeer::SALESREP, sfGuardUserPeer::ID);
$c->addGroupByColumn(sfGuardUserPeer::ID);
$c->addAscendingOrderByColumn(sfGuardUserProfilePeer::FIRST_NAME);
$this->widgetSchema['salesrep'] = new sfWidgetFormPropelChoice(array(
'label' => 'Sales rep',
'add_empty' => true,
'order_by' => array('Username', 'asc'),
'model' => 'sfGuardUser',
'method' => 'getFullName',
'criteria' => $c,
'key_method'=> 'getId',
'multiple' => false
));
Which generates this invalid MySQL:
SELECT sf_guard_user.ID, sf_guard_user.USERNAME, sf_guard_user.ALGORITHM, sf_guard_user.SALT, sf_guard_user.PASSWORD, sf_guard_user.CREATED_AT, sf_guard_user.LAST_LOGIN, sf_guard_user.IS_ACTIVE, sf_guard_user.IS_SUPER_ADMIN, sf_guard_user.VANTIVE_ID FROM LEFT JOIN `sf_guard_user` ON (quote.SALES_ENGINEER=sf_guard_user.ID)
Mostly I'd like to determine if this is a bug, if I can fix it with a minor change, or if I should roll back to 1.5.