Moving Prestashop 1.5.6 website from one hosting account to another from the same company now consumes over 25% of the server resources? - mysql

I had a Prestashop 1.5.6 website working fine in one hosting account. After moving it to another of supposedly the same features, I started to receive warning it consumes over 25% of the server resources. I know the new account has a different mysql version, I dont know if the problem could be related to that.
The warning messages I got from the hosting provider are
CPU_TIME:606 table_rows_read:357770884 SELECTS:80 ROWS_UPDATED:0 ROWS_FETCHED:442135 BUSY_TIME:668 ONNECTED_TIME:673 BYTES_SENT:27099136 BYTES_RECEIVED:17676 WAIT_TIME:62
Top table row reads:
DB_USER: ********** -- TOTAL_CONNECTIONS: 4 -- CONNECTED_TIME: 673 -- CPU_TIME: 606 -- TABLE_ROW_READS: 357770884 -- SELECT_COMMANDS: 80 -- UPDATE_COMMANDS: -- BUSY_TIME: 668 -- BYTES_SENT: 27099136 -- BYTES_RECEIVED: 17676 -- WAIT_TIME (IO): 62
Top WAIT (IO) TIME:
DB_USER: ********** -- TOTAL_CONNECTIONS: 4 -- CONNECTED_TIME: 673 -- CPU_TIME: 606 -- TABLE_ROW_READS: 357770884 -- SELECT_COMMANDS: 80 -- UPDATE_COMMANDS: -- BUSY_TIME: 668 -- BYTES_SENT: 27099136 -- BYTES_RECEIVED: 17676 -- WAIT_TIME (IO): 62
SN 03:31 0:01 /index.php
SN 03:32 0:02 /index.php
SN 03:34 0:01 /index.php
SN 03:35 0:00 /index.php
Fri Feb 23 03:35:58 CST 2018
Running Processes:
S 03:08 0:00 dovecot/imap
SN 03:31 0:01 /index.php
SN 03:32 0:02 /index.php
SN 03:34 0:01 /index.php
SN 03:35 0:00 /index.php
Running Queries:
*************************** 1. row ***************************
USER: *********
DB: *********
STATE: Sending data
TIME: 34
COMMAND: Query
INFO: SELECT c.`name`, cl.`id_lang`, IF(cl.`id_lang` IS NULL, c.`value`, cl.`value`) AS value, c.id_shop_group, c.id_shop
FROM `ps_configuration` c
LEFT JOIN `ps_configuration_lang` cl ON (c.id_configuration = cl.id_configuration)
*************************** 2. row ***************************
USER: **********
DB: ***********
STATE: Copying to tmp table
TIME: 10
COMMAND: Query
INFO: SELECT h.id_hook, h.name as h_name, title, description, h.position, live_edit, hm.position as hm_position, m.id_module, m.name, active
FROM `ps_hook` h
INNER JOIN `ps_hook_module` hm ON (h.id_hook = hm.id_hook AND hm.id_shop = 1)
INNER JOIN `ps_module` as m ON (m.id_module = hm.id_module)
ORDER BY hm.position
*************************** 3. row ***************************
USER: **********
DB: **********
STATE: Copying to tmp table
TIME: 102
COMMAND: Query
INFO: SELECT h.id_hook, h.name as h_name, title, description, h.position, live_edit, hm.position as hm_position, m.id_module, m.name, active
FROM `ps_hook` h
INNER JOIN `ps_hook_module` hm ON (h.id_hook = hm.id_hook AND hm.id_shop = 1)
INNER JOIN `ps_module` as m ON (m.id_module = hm.id_module)
ORDER BY hm.position
*************************** 4. row ***************************
USER: **********
DB: **********
STATE: Copying to tmp table
TIME: 178
COMMAND: Query
INFO: SELECT h.id_hook, h.name as h_name, title, description, h.position, live_edit, hm.position as hm_position, m.id_module, m.name, active
FROM `ps_hook` h
INNER JOIN `ps_hook_module` hm ON (h.id_hook = hm.id_hook AND hm.id_shop = 1)
INNER JOIN `ps_module` as m ON (m.id_module = hm.id_module)
ORDER BY hm.position
Open connections
Current Site Requests:
index.php?id_category=20&controller=category&id_la
index.php?id_product=212&controller=product&id_lan
index.php?id_product=212&controller=product&id_lan
index.php?id_product=247&controller=product&id_lan
index.php?id_product=88&controller=product&id_lang
The table structures are

Did you try profiling ? There is a great tool inside Prestashop that you can activate by modifying config/defines.inc.php :
define('_PS_DEBUG_PROFILING_', true);
I also would suggest that you take a look at PHP Versions, as it seems like their would be some major change in how scripts are handled for example between 5.2 and 7.1 versions.

Related

Delta index updates not automatic

Unfortunately, I think the error is not so that he has automatically updated the delta
I have this table addet in "database"
# in MySQL
CREATE TABLE sph_counter
(
counter_id INTEGER PRIMARY KEY NOT NULL,
max_doc_id INTEGER NOT NULL
);
source database
{
type = mysql
sql_host = localhost
sql_user = root
sql_pass = root
sql_db = database
sql_port = 3306 # optional, default is 3306
sql_query = \
SELECT ID, name, dir, UNIX_TIMESTAMP(ctime) AS ctime, \
FROM database
sql_field_string = dir
sql_field_string = name
}
source delta : database
{
sql_range_step = 2
sql_query_pre = SET NAMES utf8
sql_query = SELECT id, title, body FROM documents \
WHERE id>( SELECT max_doc_id FROM sph_counter WHERE counter_id=1 )
sql_query_pre = REPLACE INTO sph_counter_rls SELECT 1, MAX(ID) FROM `database`
}
index delta : database
{
source = database
path = /home/data/delta
}
index database
{
source = database
path = /home/data/database
docinfo = extern
#charset_type = sbcs
morphology = none
stopwords =
# minimum indexed word length
# default is 1 (index everything)
min_word_len = 1
charset_table = 0..9, A..Z->a..z, a..z, -, U+0028, U+0029
#enable_star = 1
min_prefix_len = 0
min_infix_len = 2
ngram_len = 0
}
edit: i have addet : index delta ...
indexer --all
Sphinx 2.2.11-id64-release (95ae9a6)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file '/etc/sphinxsearch/sphinx.conf'...
indexing index 'database'...
collected 100 docs, 0.0 MB
sorted 0.0 Mhits, 100.0% done
total 100 docs, 8982 bytes
total 0.078 sec, 114887 bytes/sec, 1279.08 docs/sec
indexing index 'delta'...
collected 100 docs, 0.0 MB
sorted 0.0 Mhits, 100.0% done
total 100 docs, 8982 bytes
total 0.063 sec, 140585 bytes/sec, 1565.19 docs/sec
total 212 reads, 0.000 sec, 0.3 kb/call avg, 0.0 msec/call avg
total 24 writes, 0.000 sec, 6.3 kb/call avg, 0.0 msec/call avg
You 'delta' index can't be built at all since it doesn't defined in the config. Only 'delta' SOURCE is defined. And there's only INDEX - database which uses source 'database', not 'delta'

Why do my values show as NULL when pivoting table in MySQL

I'm using MySQL - Rows to Columns and this tutorial http://stratosprovatopoulos.com/web-development/mysql/pivot-a-table-in-mysql/#comment-6128 to help me pivot a table and it's working pretty well. Starting with this:
mediaID q_short_name start_time stop_time audio_link
ee CVV Number 208 210 j.mp3
ee Expiration Date 308 310 j.mp3
ff CVV Number 124 127 k.mp3
ff Expiration Date 166 169 k.mp3
The goal is this:
mediaID CVVNumStartT CVVNumStopT ExpDateStart_time ExpDateStop_time Aud
ee 208 210 308 310 k.mp3
ff 124 127 166 169 j.mp3
I got part of the way there with this code:
CREATE VIEW test__extension AS (
SELECT amr_text.*,
CASE WHEN q_short_name = 'CVV Number' THEN amr_text.start_time END AS
CVV_Start_Time,
CASE WHEN q_short_name = 'CVV Number' THEN amr_text.stop_time END AS
CVV_Stop_Time,
CASE WHEN q_short_name = 'Expiration Date' THEN amr_text.start_time END
AS Expiration_Start_Time,
CASE WHEN q_short_name = 'Expiration Date' THEN amr_text.stop_time END
AS Expiration_Stop_Time, FROM amr_text);
CREATE VIEW test_extension_pivot AS (SELECT mediaID,
SUM(CVV_Start_Time) AS CVV_Start_Time,
SUM(CVV_Stop_Time) AS CVV_Stop_Time,
SUM(Expiration_Start_Time) AS Expiration_Start_Time,
SUM(Expiration_Stop_Time) AS Expiration_Stop_Time,
FROM test_extension GROUP BY mediaID);
This creates columns exactly like the goal table. But now the values for everything except the mediaIDs are rendered as NULL. My questions are, why did they get replaced by NULL, and what can I use instead of SUM to render the values of Expiration and CVV Start and Stop Time as they are in the original table?

I found my google compute instance in terminated state

I found this lines in /var/log/auth.log ( Ubuntu 15.04 )
Aug 6 20:46:52 <> systemd-logind[641]: Power key pressed.
Aug 6 20:46:52 <> systemd-logind[641]: Powering Off...
Aug 6 20:46:52 <> systemd-logind[641]: System is powering down.
Aug 6 20:46:53 <> sshguard[1440]: Got exit signal, flushing blocked addresses and exiting...
Aug 6 20:46:53 <> sshd[1419]: Received signal 15; terminating.
What does it mean? Who or what turn off my instance?

Sphinx Error: failed to merge index

While merging index with delta on Sphinx, I got this error:
~: /usr/local/bin/indexer --merge myindex myindexDelta --rotate;
Sphinx 2.0.6-release (r3473)
Copyright (c) 2001-2012, Andrew Aksyonoff
Copyright (c) 2008-2012, Sphinx Technologies Inc ( http://sphinxsearch.com )
using config file '/usr/local/etc/sphinx.conf'...
merging index 'myindexDelta' into index 'myindex'...
read 414.6 of 414.6 MB, 100.0% done
FATAL: failed to merge index 'myindexDelta' into index 'myindex': failed to open /server/sphinx/data/myindex.sps: No such file or directory
My configuration on sphinx.conf is as following
source myindex
{
type = mysql
sql_host = localhost
sql_user = db
sql_pass =
sql_db = db
sql_query_pre = SET SESSION query_cache_type=OFF
sql_query_pre = REPLACE INTO sph_counter SELECT 1, MAX(id) FROM mytable
sql_query_pre = SET NAMES utf8
sql_query = \
SELECT id,title FROM mytable \
WHERE id<=( SELECT max_doc_id FROM sph_counter WHERE counter_id=1 )
sql_ranged_throttle = 0
}
source myindexDelta : myindex
{
sql_query_pre = SET SESSION query_cache_type=OFF
sql_query_pre = SET NAMES utf8
sql_query = \
SELECT id,title FROM mytable \
WHERE id > ( SELECT max_doc_id FROM sph_counter WHERE counter_id=1 )
}
index myindex
{
source = myindex
path = /server/sphinx/data/myindex
min_word_len = 3
min_infix_len = 0
}
index myindexDelta : myindex
{
source = myindexDelta
path = /server/sphinx/data/myindexDelta
min_word_len = 3
min_infix_len = 0
}
indexes files listings with permissions:
~: ls -lh /server/sphinx/data/
-rw-r--r-- 1 root root 0 Nov 11 21:40 myindexDelta.spa
-rw-r--r-- 1 root root 290K Nov 11 21:40 myindexDelta.spd
-rw-r--r-- 1 root root 328 Nov 11 21:40 myindexDelta.sph
-rw-r--r-- 1 root root 106K Nov 11 21:40 myindexDelta.spi
-rw-r--r-- 1 root root 0 Nov 11 21:40 myindexDelta.spk
-rw------- 1 root root 0 Nov 11 21:40 myindexDelta.spl
-rw-r--r-- 1 root root 0 Nov 11 21:40 myindexDelta.spm
-rw-r--r-- 1 root root 223K Nov 11 21:40 myindexDelta.spp
-rw-r--r-- 1 root root 1 Nov 11 21:40 myindexDelta.sps
-rw-r--r-- 1 root root 0 Jul 3 21:17 myindex.spa
-rw-r--r-- 1 root root 7.0G Jul 3 23:54 myindex.spd
-rw-r--r-- 1 root root 290 Jul 3 23:54 myindex.sph
-rw-r--r-- 1 root root 397M Jul 3 23:54 myindex.spi
-rw-r--r-- 1 root root 0 Jul 3 23:54 myindex.spk
-rw------- 1 root root 0 Nov 11 21:08 myindex.spl
-rw-r--r-- 1 root root 0 Jul 3 21:17 myindex.spm
-rw-r--r-- 1 root root 9.2G Jul 3 23:54 myindex.spp
I am sure the code explains everything, adding description is not necessary.
I'm guessing that the original 'myindex' was made by a different version of sphinx. (ie dont think 2.0.6-release would of been available in July)
And somewhere in that version update, the requirement for a .sps file has changed - the new version requires it, whereas the old doesnt. You have no string attributes hence why the file contains no data in the delta.
I would suggest either rebuilding myindex with your current version of indexer - so they versions are identical.
Or maybe you could try copying myindexDelta.sps to myindex.sps. It contains no data (1 dummy byte!) so it shouldn't corrupt anything. Would only need to do this once.

A way to improve performance of the following query (update table)

This question is related to a question I published a while ago, and can be found here: Update values of database with values that are already in DB .
I've the following situation: A table that stores data from different sensors (I've a total of 8 sensors). Each row of the table has the following structure:
SensorID --- TimestampMS --- RawData --- Data
So, for example, for a temperature sensor called TEMPSensor1 I have the following:
TEMPSensor1 --- 1000 --- 200 --- 2 TEMPSensor1 --- 2000 --- 220 --- 2.2
And so on, for each sensor (in total I've 8). I've some problems reading the data, and there are rows which data "is not correct". Precisely when the rawdata field is 65535, I should update that particular row. And what I would like to do is put the next value (in time) to that "corrupted data". So, if we have this:
TEMPSensor1 --- 1000 --- 200 --- 2 TEMPSensor1 --- 2000 --- 220 --- 2.2
TEMPSensor1 --- 3000 --- 65535 --- 655.35 TEMPSensor1 --- 4000 --- 240 --- 2.4
After doing the Update, the content of the table should be changed to:
TEMPSensor1 --- 1000 --- 200 --- 2 TEMPSensor1 --- 2000 --- 220 --- 2.2
TEMPSensor1 --- 3000 --- 240 --- 2.4 TEMPSensor1 --- 4000 --- 240 --- 2.4
I've ended up doing the following:
UPDATE externalsensor es1
INNER JOIN externalsensor es2 ON es1.sensorid = es2.sensorid AND (es2.timestampms - es1.timestampms) > 60000 AND (es2.timestampms - es1.timestampms) < 120000 AND es1.rawdata <> 65535
SET es1.rawdata = es2.rawdata, es1.data = es2.data
WHERE es1.rawdata = 65535
Because I know that between two reads from a sensor there are between 60000 and 120000 ms. However, if I have two following "corrupted" readings, that won't work. Can anyone suggest a way to do this more efficiently, with no use of subquery selects, but JOINS? My idea would be to have a JOIN that gives you all the possible values for that sensor after its timestampms, and just get the first one, but I don't know how I can limit that JOIN result.
Appreciate.
Here's a solution without correlated subqueries, but with a triangular join (not sure which is worse):
UPDATE externalsensor bad
INNER JOIN (
SELECT
es1.SensorID,
es1.TimestampMS,
MIN(es2.TimestampMS) AS NextGoodTimestamp
FROM externalsensor es1
INNER JOIN externalsensor es2
ON es1.SensorID = es2.SensorID AND
es1.TimestampMS < es2.TimestampMS
WHERE es1.RawData = 65535
AND es2.RawData <> 65535
GROUP BY
es1.SensorID,
es1.TimestampMS
) link ON bad.SensorID = link.SensorID AND
bad.TimestampMS = link.TimestampMS
INNER JOIN externalsensor good
ON link.SensorID = good.SensorID AND
link.NextGoodTimestamp = good.TimestampMS
SET
bad.RawData = good.RawData,
bad.Data = good.Data
It is assumed that the timestamps are unique within a single sensor group.
A completely different approach, using a procedure that runs through the whole table (in descending time order for every sensor):
DELIMITER $$
CREATE PROCEDURE updateMyTable()
BEGIN
SET #dummy := -9999 ;
SET #dummy2 := -9999 ;
SET #sensor := -999 ;
UPDATE myTable m
JOIN
( SELECT n.SensorID
, n.TimestampMS
, #d := (n.RawData = 65535) AND (#sensor = n.SensorID) AS problem
, #dummy := IF(#d, #dummy, n.RawData) as goodRawData
, #dummy2 := IF(#d, #dummy2, n.Data) as goodData
, #sensor := n.SensorID AS previous
FROM myTable n
ORDER BY n.SensorID
, n.TimeStampMS DESC
) AS upd
ON m.SensorID = upd.SensorID
AND m.TimeStampMS = upd.TimeStampMS
SET m.RawData = upd.goodRawData
, m.Data = upd.goodData
WHERE upd.problem
;
END$$
DELIMITER ;
Since you don't want to use Dems solution from the previous question, here's a "solution" with JOIN's:
UPDATE myTable m
JOIN myTable n
ON m.SensorID = n.SensorID
AND n.RawData <> 65535
AND m.TimestampMS < n.TimestampMS
JOIN myTable q
ON n.SensorID = q.SensorID
AND q.RawData <> 65535
AND n.TimestampMS <= q.TimestampMS
SET
m.RawData = n.RawData,
m.Data = n.Data
WHERE
m.RawData = 65535
;
EDIT
My query above is wrong, dead wrong. It appears to be working in my test db but the logic is flawed. I'll explain below.
Why the above query works fine but is dead wrong:
First, why it's wrong.
Because it will not return one row for every (sensorID, bad timestamp) combination but many rows. If m (m.TimestampMS) is the bad timestamp we want to find, it will return all combinations of that bad timetsamp and later good timestamps n and q with n.TimestampMS <= q.TimestampMS. It would be a correct query if it found the MINIMUM of these n timestamps.
Now, how come it actually works all right in my test db?
I think it's because MySQL, when it comes to use the SET ... and has a lot of options (rows) it just uses first option. But lucky me, I added the test rows in increasing timestamp order so they were saved in that order in the db, and (again) lucky me, this is how the query plan is scheduled (I presume).
Even this query works in my test db:
UPDATE myTable m
JOIN myTable n
ON m.SensorID = n.SensorID
AND n.RawData <> 65535
AND m.TimestampMS < n.TimestampMS
SET
m.RawData = n.RawData,
m.Data = n.Data
WHERE
m.RawData = 65535
;
while being flawed for the same reasons.