How to debug a query in extbase? - mysql

Question

$query = $this->createQuery();
return $query->matching($query->like('linker', "$linkerKey=$linkerValue"))
->setOrderings(array('crdate' => $ordering))
->execute();
How can i debug such a generated query in extbase? When creating the same query again (but without the execute() ) and trying to display it with var_dump or the internal t3lib_div::debug i just receive a blank page.

Answer 1

In version 8.7 or newer another way needs to be taken:
use TYPO3\CMS\Extbase\Persistence\Generic\Storage\Typo3DbQueryParser;
use TYPO3\CMS\Extbase\Utility\DebuggerUtility;
$queryParser = $this->objectManager->get(Typo3DbQueryParser::class);
$queryBuilder = $queryParser->convertQueryToDoctrineQueryBuilder($query);
DebuggerUtility::var_dump($queryBuilder->getSQL());
DebuggerUtility::var_dump($queryBuilder->getParameters());

Answer 2

This information is outdated and deprecated in TYPO3 8.7 and I'm only leaving the answer up for reference. Refer to #pgampe 's answer on how debug extbase queries in more recent versions of TPYO3.
Extbase now has a QueryParser for that. In your repository method, right before returning the executed query, insert:
$parser = \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance('TYPO3\\CMS\\Extbase\\Persistence\\Generic\\Storage\\Typo3DbQueryParser');
$queryParts = $parser->parseQuery($query);
\TYPO3\CMS\Core\Utility\DebugUtility::debug($queryParts, 'query');
The result is a table view of the query parts, split by SQL keywords, e.g.:
Keep in mind that the QueryResult that your Repository returns may still be different from the SQL query result. Extbase uses the PropertyMapper to try to convert every result row into an ExtbaseObject. If the PropertyMapper is misconfigured or the row contains data that cannot be converted to the data types according to the configuration, these Objects will silently be skipped.

Answer 3

$query = $this->createQuery();
$result = $query->matching($query->like('linker', "$linkerKey=$linkerValue"))
->setOrderings(array('crdate' => $ordering))
->execute();
$GLOBALS['TYPO3_DB']->debugOutput = true;
return $result;

Answer 4

This hack to extbase is dirty, but useful:
In typo3/sysext/extbase/Classes/Persistence/Storage/Typo3DbBackend.php
edit the method buildQuery(array $sql)
before the return statement, add:
t3lib_div::debug($statement, 'SQL Query Extbase');
Remove after use, and don't forget this will affect everything that runs on extbase, so use in dev environment only
Source: http://sancer-media.net/2011/extbase-schneller-mysql-debug.html

Answer 5

An easy way without changing any Typo3 core code and not mentioned in any forum so far is using the php "serialize()" method:
$result = $query->execute();
echo (serialize($result));
In the result object you find the SQL query (look for "statement;" ...)

Answer 6

This works as long as $GLOBALS['TYPO3_DB'] is supported.
It will show you the complete build SQL query.
/**
* #param \TYPO3\CMS\Extbase\Persistence\QueryResultInterface $queryResult
* #param bool $explainOutput
* #return void
*/
public function debugQuery(
\TYPO3\CMS\Extbase\Persistence\QueryResultInterface $queryResult,
$explainOutput = false
) {
$GLOBALS['TYPO3_DB']->debugOuput = 2;
if ($explainOutput) {
$GLOBALS['TYPO3_DB']->explainOutput = true;
}
$GLOBALS['TYPO3_DB']->store_lastBuiltQuery = true;
$queryResult->toArray();
\TYPO3\CMS\Extbase\Utility\DebuggerUtility::var_dump(
$GLOBALS['TYPO3_DB']->debug_lastBuiltQuery
);
$GLOBALS['TYPO3_DB']->store_lastBuiltQuery = false;
$GLOBALS['TYPO3_DB']->explainOutput = false;
$GLOBALS['TYPO3_DB']->debugOuput = false;
}
So with that function you can do something like this in your controller:
$all = $this->repository->findAll();
$this->repository->debugQuery($all);

Answer 7

According to #pgampe's answer and
#FranzHolzinger's comment I created AbstractonRepository in my ext wit some extracted methods, that can be extended in your own repository to bring debug possibility little bit more comfortable.
NOTE That's tested and working for TYPO3 ver 10.x, most probably will work since 8.7+ as well, requires testing.
<?php
namespace VENDOR\Extkey\Domain\Repository;
use TYPO3\CMS\Extbase\Utility\DebuggerUtility;
use TYPO3\CMS\Core\Exception;
use TYPO3\CMS\Extbase\Persistence\Generic\Storage\Typo3DbQueryParser;
use TYPO3\CMS\Extbase\Persistence\QueryInterface;
/**
* Class AbstractRepository brings methods for query debugging in TYPO3 ver. 10.x
* Based on StackOverflow answer by `pgampe` answer and `FranzHolzinger` comment
* source https://stackoverflow.com/a/44286155/1066240
*
* All repositories in this extension should extend it.
*
* #author Marcus Biesioroff <biesior#gmail.com>
* #package VENDOR\Extkey\Domain\Repository
*/
abstract class AbstractRepository extends \TYPO3\CMS\Extbase\Persistence\Repository
{
/**
* #param mixed $query TYPO3\CMS\Core\Database\Query\QueryBuilder | TYPO3\CMS\Extbase\Persistence\Generic\Query
* #param string|null $title Optional title for var_dump()
* #param bool $replaceParams if true replaces the params in SQL statement with values, otherwise dumps the array of params. #see self::renderDebug()
*
* #throws Exception
*/
protected function debugQuery($query, string $title = null, bool $replaceParams = true): void
{
if ($query instanceof \TYPO3\CMS\Core\Database\Query\QueryBuilder) {
$sql = $query->getSQL();
$params = $query->getParameters();
$this->renderDebug($sql, $params, $title, $replaceParams);
} elseif ($query instanceof \TYPO3\CMS\Extbase\Persistence\Generic\Query) {
$this->parseTheQuery($query, $title, $replaceParams);
} else {
throw new Exception('Unhandled type for SQL query, curently only TYPO3\CMS\Core\Database\Query\QueryBuilder | TYPO3\CMS\Extbase\Persistence\Generic\Query can be debugged with ' . static::getRepositoryClassName() . '::debugQuery() method.', 1596458998);
}
}
/**
* Parses query and displays debug
*
* #param QueryInterface $query Query
* #param string|null $title Optional title
* #param bool $replaceParams if true replaces the params in SQL statement with values, otherwise dumps the array of params. #see self::renderDebug()
*/
private function parseTheQuery(QueryInterface $query, string $title = null, $replaceParams = true): void
{
/** #var Typo3DbQueryParser $queryParser */
$queryParser = $this->objectManager->get(\TYPO3\CMS\Extbase\Persistence\Generic\Storage\Typo3DbQueryParser::class);
$sql = $queryParser->convertQueryToDoctrineQueryBuilder($query)->getSQL();
$params = $queryParser->convertQueryToDoctrineQueryBuilder($query)->getParameters();
$this->renderDebug($sql, $params, $title, $replaceParams);
}
/**
* Renders the output with DebuggerUtility::var_dump()
*
* #param string $sql Generated SQL
* #param array $params Params' array
* #param string|null $title Optional title for var_dump()
* #param bool $replaceParams if true replaces the params in SQL statement with values, otherwise dumps the array of params.
*/
private function renderDebug(string $sql, array $params, string $title = null, bool $replaceParams = true): void
{
if ($replaceParams) {
$search = array();
$replace = array();
foreach ($params as $k => $v) {
$search[] = ':' . $k;
$type = gettype($v);
if (in_array($type, ['integer'])) {
$replace[] = $v;
} else {
$replace[] = '\'' . $v . '\'';
}
}
$sql = str_replace($search, $replace, $sql);
DebuggerUtility::var_dump($sql, $title);
} else {
DebuggerUtility::var_dump(
[
'SQL' => $sql,
'Parameters' => $params
],
$title);
}
}
}
It can be used in your repository like in to ways:
<?php
namespace VENDOR\Extkey\Domain\Repository;
use TYPO3\CMS\Core\Database\Connection;
use TYPO3\CMS\Core\Database\ConnectionPool;
class FooRepository extends \VENDOR\Extkey\Domain\Repository\AbstractRepository
{
public function findByName($name)
{
$query = $this->createQuery();
$query->matching(
$query->equals('name', $name)
);
$this->debugQuery($query, 'Debug SQL in repository with QueryInterface');
return $query->execute();
}
public function queryByName($name)
{
/** #var ConnectionPool $pool */
$pool = GeneralUtility::makeInstance(ConnectionPool::class);
$connection = $pool->getConnectionForTable('tx_extkey_domain_model_yourmodel');
$queryBuilder = $connection->createQueryBuilder();
$query = $queryBuilder
->select('*')
->from('tx_extkey_domain_model_yourmodel')
->where("name like :name")
->setParameter('name', "%{$name}%");
$this->debugQuery($query, 'Debug SQL in my repository with QueryBuilder');
return $query->execute()->fetchAll();
}

Answer 8

In v6.2x or later, you can debug result object in extBase like:
In repository:
return $query->execute(true); // "true" will return array result
Or also you can debug result object in Controller:
$resultObject = $this->yourRepository->findAll();
\TYPO3\CMS\Extbase\Utility\DebuggerUtility::var_dump($resultObject);

Answer 9

Here I post a method you can enter for debugging in any class, making a trait of it would be surely possible too. Authorship and source is mentioned in the comment, Usage too:
/**
* Render the generated SQL of a query in TYPO3 8
*
* #author wp_bube https://www.typo3.net/forum/user-profil/benutzer/zeige/benutzer/wp-bube/
* #src https://www.typo3.net/forum/thematik/zeige/thema/125747/
*
* Usage: $this->debugQuery($query);
*
* #param \TYPO3\CMS\Extbase\Persistence\QueryInterface $query
* #param bool $format
* #param bool $exit
*/
private function debugQuery($query, $format = true, $exit = true)
{
function getFormattedSQL($sql_raw)
{
if (empty($sql_raw) || !is_string($sql_raw)) {
return false;
}
$sql_reserved_all = array( 'ACCESSIBLE', 'ACTION', 'ADD', 'AFTER', 'AGAINST', 'AGGREGATE', 'ALGORITHM', 'ALL', 'ALTER', 'ANALYSE', 'ANALYZE', 'AND', 'AS', 'ASC', 'AUTOCOMMIT', 'AUTO_INCREMENT', 'AVG_ROW_LENGTH', 'BACKUP', 'BEGIN', 'BETWEEN', 'BINLOG', 'BOTH', 'BY', 'CASCADE', 'CASE', 'CHANGE', 'CHANGED', 'CHARSET', 'CHECK', 'CHECKSUM', 'COLLATE', 'COLLATION', 'COLUMN', 'COLUMNS', 'COMMENT', 'COMMIT', 'COMMITTED', 'COMPRESSED', 'CONCURRENT', 'CONSTRAINT', 'CONTAINS', 'CONVERT', 'CREATE', 'CROSS', 'CURRENT_TIMESTAMP', 'DATABASE', 'DATABASES', 'DAY', 'DAY_HOUR', 'DAY_MINUTE', 'DAY_SECOND', 'DEFINER', 'DELAYED', 'DELAY_KEY_WRITE', 'DELETE', 'DESC', 'DESCRIBE', 'DETERMINISTIC', 'DISTINCT', 'DISTINCTROW', 'DIV', 'DO', 'DROP', 'DUMPFILE', 'DUPLICATE', 'DYNAMIC', 'ELSE', 'ENCLOSED', 'END', 'ENGINE', 'ENGINES', 'ESCAPE', 'ESCAPED', 'EVENTS', 'EXECUTE', 'EXISTS', 'EXPLAIN', 'EXTENDED', 'FAST', 'FIELDS', 'FILE', 'FIRST', 'FIXED', 'FLUSH', 'FOR', 'FORCE', 'FOREIGN', 'FROM', 'FULL', 'FULLTEXT', 'FUNCTION', 'GEMINI', 'GEMINI_SPIN_RETRIES', 'GLOBAL', 'GRANT', 'GRANTS', 'GROUP', 'HAVING', 'HEAP', 'HIGH_PRIORITY', 'HOSTS', 'HOUR', 'HOUR_MINUTE', 'HOUR_SECOND', 'IDENTIFIED', 'IF', 'IGNORE', 'IN', 'INDEX', 'INDEXES', 'INFILE', 'INNER', 'INSERT', 'INSERT_ID', 'INSERT_METHOD', 'INTERVAL', 'INTO', 'INVOKER', 'IS', 'ISOLATION', 'JOIN', 'KEY', 'KEYS', 'KILL', 'LAST_INSERT_ID', 'LEADING', 'LEFT', 'LEVEL', 'LIKE', 'LIMIT', 'LINEAR', 'LINES', 'LOAD', 'LOCAL', 'LOCK', 'LOCKS', 'LOGS', 'LOW_PRIORITY', 'MARIA', 'MASTER', 'MASTER_CONNECT_RETRY', 'MASTER_HOST', 'MASTER_LOG_FILE', 'MASTER_LOG_POS', 'MASTER_PASSWORD', 'MASTER_PORT', 'MASTER_USER', 'MATCH', 'MAX_CONNECTIONS_PER_HOUR', 'MAX_QUERIES_PER_HOUR', 'MAX_ROWS', 'MAX_UPDATES_PER_HOUR', 'MAX_USER_CONNECTIONS', 'MEDIUM', 'MERGE', 'MINUTE', 'MINUTE_SECOND', 'MIN_ROWS', 'MODE', 'MODIFY', 'MONTH', 'MRG_MYISAM', 'MYISAM', 'NAMES', 'NATURAL', 'NOT', 'NULL', 'OFFSET', 'ON', 'OPEN', 'OPTIMIZE', 'OPTION', 'OPTIONALLY', 'OR', 'ORDER', 'OUTER', 'OUTFILE', 'PACK_KEYS', 'PAGE', 'PARTIAL', 'PARTITION', 'PARTITIONS', 'PASSWORD', 'PRIMARY', 'PRIVILEGES', 'PROCEDURE', 'PROCESS', 'PROCESSLIST', 'PURGE', 'QUICK', 'RAID0', 'RAID_CHUNKS', 'RAID_CHUNKSIZE', 'RAID_TYPE', 'RANGE', 'READ', 'READ_ONLY', 'READ_WRITE', 'REFERENCES', 'REGEXP', 'RELOAD', 'RENAME', 'REPAIR', 'REPEATABLE', 'REPLACE', 'REPLICATION', 'RESET', 'RESTORE', 'RESTRICT', 'RETURN', 'RETURNS', 'REVOKE', 'RIGHT', 'RLIKE', 'ROLLBACK', 'ROW', 'ROWS', 'ROW_FORMAT', 'SECOND', 'SECURITY', 'SELECT', 'SEPARATOR', 'SERIALIZABLE', 'SESSION', 'SET', 'SHARE', 'SHOW', 'SHUTDOWN', 'SLAVE', 'SONAME', 'SOUNDS', 'SQL', 'SQL_AUTO_IS_NULL', 'SQL_BIG_RESULT', 'SQL_BIG_SELECTS', 'SQL_BIG_TABLES', 'SQL_BUFFER_RESULT', 'SQL_CACHE', 'SQL_CALC_FOUND_ROWS', 'SQL_LOG_BIN', 'SQL_LOG_OFF', 'SQL_LOG_UPDATE', 'SQL_LOW_PRIORITY_UPDATES', 'SQL_MAX_JOIN_SIZE', 'SQL_NO_CACHE', 'SQL_QUOTE_SHOW_CREATE', 'SQL_SAFE_UPDATES', 'SQL_SELECT_LIMIT', 'SQL_SLAVE_SKIP_COUNTER', 'SQL_SMALL_RESULT', 'SQL_WARNINGS', 'START', 'STARTING', 'STATUS', 'STOP', 'STORAGE', 'STRAIGHT_JOIN', 'STRING', 'STRIPED', 'SUPER', 'TABLE', 'TABLES', 'TEMPORARY', 'TERMINATED', 'THEN', 'TO', 'TRAILING', 'TRANSACTIONAL', 'TRUNCATE', 'TYPE', 'TYPES', 'UNCOMMITTED', 'UNION', 'UNIQUE', 'UNLOCK', 'UPDATE', 'USAGE', 'USE', 'USING', 'VALUES', 'VARIABLES', 'VIEW', 'WHEN', 'WHERE', 'WITH', 'WORK', 'WRITE', 'XOR', 'YEAR_MONTH' );
$sql_skip_reserved_words = array('AS', 'ON', 'USING');
$sql_special_reserved_words = array('(', ')');
$sql_raw = str_replace("\n", " ", $sql_raw);
$sql_formatted = "";
$prev_word = "";
$word = "";
for ($i = 0, $j = strlen($sql_raw); $i < $j; $i++) {
$word .= $sql_raw[$i];
$word_trimmed = trim($word);
if ($sql_raw[$i] == " " || in_array($sql_raw[$i], $sql_special_reserved_words)) {
$word_trimmed = trim($word);
$trimmed_special = false;
if (in_array($sql_raw[$i], $sql_special_reserved_words)) {
$word_trimmed = substr($word_trimmed, 0, -1);
$trimmed_special = true;
}
$word_trimmed = strtoupper($word_trimmed);
if (in_array($word_trimmed, $sql_reserved_all) && !in_array($word_trimmed, $sql_skip_reserved_words)) {
if (in_array($prev_word, $sql_reserved_all)) {
$sql_formatted .= '<b>' . strtoupper(trim($word)) . '</b>' . ' ';
} else {
$sql_formatted .= '<br/> ';
$sql_formatted .= '<b>' . strtoupper(trim($word)) . '</b>' . ' ';
}
$prev_word = $word_trimmed;
$word = "";
} else {
$sql_formatted .= trim($word) . ' ';
$prev_word = $word_trimmed;
$word = "";
}
}
}
$sql_formatted .= trim($word);
return $sql_formatted;
}
$queryParser = $this->objectManager->get(\TYPO3\CMS\Extbase\Persistence\Generic\Storage\Typo3DbQueryParser::class);
$doctrineQueryBuilder = $queryParser->convertQueryToDoctrineQueryBuilder($query);
$preparedStatement = $doctrineQueryBuilder->getSQL();
$parameters = $doctrineQueryBuilder->getParameters();
$stringParams = [];
foreach ($parameters as $key => $parameter) {
$stringParams[':' . $key] = $parameter;
}
$statement = strtr($preparedStatement, $stringParams);
if ($format) {
echo '<code>' . getFormattedSQL($statement) . '</code>';
} else {
echo $statement;
}
if ($exit) {
exit;
}
}

Answer 10

I build a static function of a utilitiee class for the use in TYPO3 9.5.
it should work in TYPO3 10.4. and TYPO3 11
<?php
namespace MyExtension\MyVendor\Utilities;
/**
* used in TYPO3 9.5.x
*/
use TYPO3\CMS\Core\Database\Query\QueryBuilder;
use TYPO3\CMS\Core\Utility\GeneralUtility;
use TYPO3\CMS\Extbase\Object\ObjectManager;
use TYPO3\CMS\Extbase\Persistence\Generic\Storage\Typo3DbQueryParser;
use TYPO3\CMS\Extbase\Utility\DebuggerUtility;
class DebugQueryUtilities
{
public static function checkExtbaseQuery($query)
{
$objectManager = GeneralUtility::makeInstance(ObjectManager::class);
$queryParser = $objectManager->get(Typo3DbQueryParser::class);
/** #var QueryBuilder $doctrineQuery */
$doctrineQuery = $queryParser->convertQueryToDoctrineQueryBuilder($query);
$result = $doctrineQuery->getSQL();
$params = $doctrineQuery->getParameters();
DebuggerUtility::var_dump(
$result
); // generate an output in the frontpage
DebuggerUtility::var_dump(
$params
);
return [$result, $params]; // make result vieable with xDebug
}
}
Use in Repository
...
[$sql,$params] = DebugQueryUtilities::checkExtbaseQuery($query);
return $query->execute()->toArray();

Answer 11

fixed #biesior ´s answer:
<?php
declare(strict_types=1);
namespace YOUR\NAMESPACE\HERE;
use TYPO3\CMS\Core\Utility\GeneralUtility;
use TYPO3\CMS\Extbase\Persistence\Generic\Storage\Typo3DbQueryParser;
use TYPO3\CMS\Extbase\Persistence\QueryInterface;
class Debug
{
/**
* #param mixed $query TYPO3\CMS\Core\Database\Query\QueryBuilder | TYPO3\CMS\Extbase\Persistence\Generic\Query
* #throws \Exception
*/
public static function debugQuery(\TYPO3\CMS\Core\Database\Query\QueryBuilder|\TYPO3\CMS\Extbase\Persistence\Generic\Query $query): void
{
$sql = self::getQuerySql($query);
\TYPO3\CMS\Core\Utility\DebugUtility::debug(
$var = $sql,
$header = '',
$group = 'Debug'
);
}
/**
* #param QueryInterface $query Query
* #return string
* #throws \Exception
*/
public static function getQuerySql(QueryInterface $query): string
{
if ($query instanceof \TYPO3\CMS\Core\Database\Query\QueryBuilder) {
$sql = $query->getSQL();
$params = $query->getParameters();
$executeableSql = self::replaceQueryParams($sql, $params);
} elseif ($query instanceof \TYPO3\CMS\Extbase\Persistence\Generic\Query) {
/** #var Typo3DbQueryParser $queryParser */
//$queryParser = $this->objectManager->get(\TYPO3\CMS\Extbase\Persistence\Generic\Storage\Typo3DbQueryParser::class);
$queryParser = GeneralUtility::makeInstance(\TYPO3\CMS\Extbase\Persistence\Generic\Storage\Typo3DbQueryParser::class);
$sql = $queryParser->convertQueryToDoctrineQueryBuilder($query)->getSQL();
$params = $queryParser->convertQueryToDoctrineQueryBuilder($query)->getParameters();
$executeableSql = self::replaceQueryParams($sql, $params);
} else {
throw new \Exception('Unhandled type for SQL query, curently only TYPO3\CMS\Core\Database\Query\QueryBuilder | TYPO3\CMS\Extbase\Persistence\Generic\Query can be debugged with ' . static::getRepositoryClassName() . '::debugQuery() method.', 1596458998);
}
return $executeableSql;
}
/**
* #param string $sql Generated SQL
* #param array $params Params' array
* #return string
*/
private static function replaceQueryParams(string $sql, array $params): string
{
$search = array();
$replace = array();
foreach ($params as $k => $v) {
$search[] = ':' . $k;
$type = gettype($v);
if (in_array($type, ['integer'])) {
$replace[] = $v;
} else {
// escape \ (eg in TYPO3 STI)
$v = str_replace('\\', '\\\\', $v);
$replace[] = '\'' . $v . '\'';
}
}
$sql = str_replace($search, $replace, $sql);
return $sql;
}
}