I need help concerning the use of "Query Builder" in Symfony.
I would like to retrieve the values of an attribute (Note) from one of my tables (Avis) in my database. After that, I would like to average all of his scores for display on my site.
For now I have the SQL query which achieves what I want :
SELECT AVG(avis.note) AS notetotal FROM avis
But afterwards, I don't understand what to do, or at least how "Query Builder" works
With the QueryBuilder you can build your queries step by step in a programmatically way, but its actualy only build the DQL wich you can get with $queryBuilder->getDql(). There are some cases where its easier to use the QueryBuilder instead of concatinate a long DQL query string.
There are two possible ways to build the query with the QueryBuilder. But for most cases its recommended to use plain DQL where ever you can, as its more readable.
First with the Expr Class
$queryBuilder = $this->getDoctrine()->getManager()->createQueryBuilder('a');
$queryBuilder->select($queryBuilder->expr()->avg('a.note'))
->from(Avis::class, 'a');
$avg = $queryBuilder->getQuery()->getSingleResult();
// $avg = [1 => (int) AVG]
or without Expr Class
$queryBuilder = $this->getDoctrine()->getManager()->createQueryBuilder('a');
$queryBuilder->select('AVG(a.note) as notetotal')
->from(Avis::class, 'a');
$avg = $queryBuilder->getQuery()->getSingleResult();
// $avg = ['notetotal' => (int) AVG]
The same in DQL
$dql = 'SELECT AVG(a.note) AS notetotal FROM ' . Avis::class . ' a';
$avg = $this->getDoctrine()->getManager()->createQuery($dql)->execute();
// $avg = [0 => ['notetotal' => (int) AVG]]
Related
I have an array of integers that need to be inserted as a batch of rows. Each row needs some other data.
$ids = [1,2]
$thing = 1
$now = Carbon::now(); // This is just a timestamp.
$binding_values = trim(str_repeat("({$thing}, ?, '{$now}'),", count($ids)), ',');
The string $binding_values looks like this:
"(1, ?, '2019-01-01 00:00:00'), (1, ?, '2019-01-01 00:00:00')"
Then I prepare my query string and bind the parameters to it. The IGNORE is used because I have a composite unique index on the table. It doesn't seem relevant to the problem though so I've left the details out.
DB::insert("
INSERT IGNORE INTO table (thing, id, created_at)
VALUES {$binding_values}
", $ids);
This works almost all the time but every now and then I get an error SQLSTATE[HY000]: General error: 2031.
Is the way I'm doing this parameter binding some kind of anti-pattern with Laravel? What might the source of this error be?
Because there is no risk of injection in this method and there is no chance that this method would be extended to a use case with a risk of injection, I've modified it to bake in all the parameters and skip parameter binding. I haven't seen any errors so far.
I would still like to know what might cause this behaviour so I can manage it better in the future. I'd be grateful for any insight.
I don't see a big issue with your query other than baking parameters into the query, which is vulnerable to SQL injection.
From what I can see, your problem is that you need INSERT ÌGNORE INTO which is not supported out of the box by Laravel. Have you thought about using a third-party package like staudenmeir/laravel-upsert?
An alternative could be to wrap your query in a transaction and select existing entries first, giving you the chance to not insert them a second time:
$ids = [1, 2];
$thing = 1;
$time = now();
DB::transaction(function () use ($ids, $thing, $time) {
$existing = DB::table('some_table')->whereIn('id', $ids)->pluck('id')->toArray();
$toInsert = array_diff($ids, $existing);
$dataToInsert = array_map(function ($id) use ($thing, $time) {
return [
'id' => $id,
'thing' => $thing,
'created_at' => $time
];
}, $toInsert);
DB::table('some_table')->insert($dataToInsert);
});
This way you will only insert what is not present yet, but you will also stay within the framework capabilities of the query builder. Only downside is that it will be slightly slower due to a second roundtrip to the database.
According to Doctrine's documentation it is possible to bind parameters to a query. Example:
$qb->select('u')
->from('User u')
->where('u.id = ?1')
->orderBy('u.name', 'ASC')
->setParameter(1, 100);
Derived from this question I would like to know if it is possible to parametrize the select and the from statement as well? Like
$qb->select('?1')
->from('?2 u')
->where('u.id = 2')
->orderBy('u.name', 'ASC')
->setParameters(array(1 => 'mytable', 2 => 'mycolumn'));
I didn't manage to do so, but maybe I just did not know the proper way. Does anyone?
This is the way prepared statements work with PDO. The query and the statement are being send seperately. This allows your database to calculate the optimal query path for your query. The query path then makes use of the parameters to get the right results. The query path will try to optimize speed for the next time you make the same query. So for select and from you just have to do select($select). Note that the optimization will be lost of you put a new select each time.
Update:
This is a related answer: https://stackoverflow.com/a/182353/1833322
This is an example of how it should look in DQL:
$query = $em->createQuery('SELECT x FROM '.$mappingPlusEntity.' x WHERE x.id = ?1');
$query->setParameter(1, 321);
Has anyone ever come across this error: General error: 1390 Prepared statement contains too many placeholders
I just did an import via SequelPro of over 50,000 records and now when I go to view these records in my view (Laravel 4) I get General error: 1390 Prepared statement contains too many placeholders.
The below index() method in my AdminNotesController.php file is what is generating the query and rendering the view.
public function index()
{
$created_at_value = Input::get('created_at_value');
$note_types_value = Input::get('note_types_value');
$contact_names_value = Input::get('contact_names_value');
$user_names_value = Input::get('user_names_value');
$account_managers_value = Input::get('account_managers_value');
if (is_null($created_at_value)) $created_at_value = DB::table('notes')->lists('created_at');
if (is_null($note_types_value)) $note_types_value = DB::table('note_types')->lists('type');
if (is_null($contact_names_value)) $contact_names_value = DB::table('contacts')->select(DB::raw('CONCAT(first_name," ",last_name) as cname'))->lists('cname');
if (is_null($user_names_value)) $user_names_value = DB::table('users')->select(DB::raw('CONCAT(first_name," ",last_name) as uname'))->lists('uname');
// In the view, there is a dropdown box, that allows the user to select the amount of records to show per page. Retrieve that value or set a default.
$perPage = Input::get('perPage', 10);
// This code retrieves the order from the session that has been selected by the user by clicking on a table column title. The value is placed in the session via the getOrder() method and is used later in the Eloquent query and joins.
$order = Session::get('account.order', 'company_name.asc');
$order = explode('.', $order);
$notes_query = Note::leftJoin('note_types', 'note_types.id', '=', 'notes.note_type_id')
->leftJoin('users', 'users.id', '=', 'notes.user_id')
->leftJoin('contacts', 'contacts.id', '=', 'notes.contact_id')
->orderBy($order[0], $order[1])
->select(array('notes.*', DB::raw('notes.id as nid')));
if (!empty($created_at_value)) $notes_query = $notes_query->whereIn('notes.created_at', $created_at_value);
$notes = $notes_query->whereIn('note_types.type', $note_types_value)
->whereIn(DB::raw('CONCAT(contacts.first_name," ",contacts.last_name)'), $contact_names_value)
->whereIn(DB::raw('CONCAT(users.first_name," ",users.last_name)'), $user_names_value)
->paginate($perPage)->appends(array('created_at_value' => Input::get('created_at_value'), 'note_types_value' => Input::get('note_types_value'), 'contact_names_value' => Input::get('contact_names_value'), 'user_names_value' => Input::get('user_names_value')));
$notes_trash = Note::onlyTrashed()
->leftJoin('note_types', 'note_types.id', '=', 'notes.note_type_id')
->leftJoin('users', 'users.id', '=', 'notes.user_id')
->leftJoin('contacts', 'contacts.id', '=', 'notes.contact_id')
->orderBy($order[0], $order[1])
->select(array('notes.*', DB::raw('notes.id as nid')))
->get();
$this->layout->content = View::make('admin.notes.index', array(
'notes' => $notes,
'created_at' => DB::table('notes')->lists('created_at', 'created_at'),
'note_types' => DB::table('note_types')->lists('type', 'type'),
'contacts' => DB::table('contacts')->select(DB::raw('CONCAT(first_name," ",last_name) as cname'))->lists('cname', 'cname'),
'accounts' => Account::lists('company_name', 'company_name'),
'users' => DB::table('users')->select(DB::raw('CONCAT(first_name," ",last_name) as uname'))->lists('uname', 'uname'),
'notes_trash' => $notes_trash,
'perPage' => $perPage
));
}
Any advice would be appreciated. Thanks.
Solved this issue by using array_chunk function.
Here is the solution below:
foreach (array_chunk($data,1000) as $t)
{
DB::table('table_name')->insert($t);
}
There is limit 65,535 (2^16-1) place holders in MariaDB 5.5 which is supposed to have identical behaviour as MySQL 5.5.
Not sure if relevant, I tested it on PHP 5.5.12 using MySQLi / MySQLND.
This error only happens when both of the following conditions are met:
You are using the MySQL Native Driver (mysqlnd) and not the MySQL client library (libmysqlclient)
You are not emulating prepares.
If you change either one of these factors, this error will not occur. However keep in mind that doing both of these is recommended either for performance or security issues, so I would not recommend this solution for anything but more of a one-time or temporary problem you are having. To prevent this error from occurring, the fix is as simple as:
$dbh->setAttribute(PDO::ATTR_EMULATE_PREPARES, true);
While I think #The Disintegrator is correct about the placeholders being limited. I would not run 1 query per record.
I have a query that worked fine until I added one more column and now I have 72k placeholders and I get this error. However, that 72k is made up of 9000 rows with 8 columns. Running this query 1 record at a time would take days. (I'm trying to import AdWords data into a DB and it would literally take more than 24 hours to import a days worth of data if I did it 1 record at a time. I tried that first.)
What I would recommend is something of a hack. First either dynamically determine the max number of placeholders you want to allow - i.e. 60k to be safe. Use this number to determine, based on the number of columns, how many complete records you can import/return at once. Create the full array of data for you query. Use a array_chunk and a foreach loop to grab everything you want in the minimum number of queries. Like this:
$maxRecords = 1000;
$sql = 'SELECT * FROM ...';
$qMarks = array_fill(0, $maxInsert, '(?, ...)');
$tmp = $sql . $implode(', ', $qMarks);
foreach (array_chunk($data, $maxRecords) AS $junk=>$dataArray) {
if (count($dataArray) < $maxRecords)) { break; }
// Do your PDO stuff here using $tmp as you SQL statement with all those placeholders - the ?s
}
// Now insert all the leftovers with basically the same code as above except accounting for
// the fact that you have fewer than $maxRecords now.
Using Laravel model, copy all 11000 records from sqlite database to mysql database in few seconds. Chunk data array to 500 records:
public function handle(): void
{
$smodel = new Src_model();
$smodel->setTable($this->argument('fromtable'));
$smodel->setConnection('default'); // sqlite database
$src = $smodel::all()->toArray();
$dmodel = new Dst_model();
$dmodel->setTable($this->argument('totable'));
$dmodel->timestamps = false;
$stack = $dmodel->getFields();
$fields = array_shift($stack);
$condb = DB::connection('mysql');
$condb->beginTransaction();
$dmodel::query()->truncate();
$dmodel->fillable($stack);
$srcarr=array_chunk($src,500);
$isOK=true;
foreach($srcarr as $item) {
if (!$dmodel->query()->insert($item)) $isOK=false;
}
if ($isOK) {
$this->notify("Przenieśliśmy tabelę z tabeli : {$this->argument('fromtable')} do tabeli: {$this->argument('totable')}", 'Będzie świeża jak nigdy!');
$condb->commit();
}
else $condb->rollBack();
}
You can do it with array_chunk function, like this:
foreach(array_chunk($data, 1000) as $key => $smallerArray) {
foreach ($smallerArray as $index => $value) {
$temp[$index] = $value
}
DB::table('table_name')->insert(temp);
}
My Fix for above issue:
On my side when i got this error I fixed it by reducing the the bulk insertion chunk size from 1000 to 800 and it worked for me.
Actually there were too many fields in my table and most them contains the details descriptions of size like a complete page text. when i go for there bulk insertion the service caused crashed and through the above error.
I think the number of placeholders is limited to 65536 per query (at least in older mysql versions).
I really can't discern what this piece of code is generating. But if it's a gigantic query, There's your problem.
You should generate one query per record to import and put those into a transaction.
Mysql codeigniter query is not working properly.
Suppose if mysql table looks like this:
user_id|user_name
1|john
2|alex
3|sam
Here user_name is unique
The following query should return false if user_name=john and user_id=1 and true if say user_name=john and user_id=2.
$this->db->get_where('user', array('user_name' => $name,'user_id !=' => $userid));
But it returns true in the case user_name=john and user_id=1.
Can anyone suggest me an alternative way of querying not equal to.
print($this->db->last_query()) gives:
SELECT * FROM (user) WHERE user_name = 'john' AND user_id != '1'
Why dont you use simple $this->db->query('your query');
Simply try this, Add the desired condition in the where function.
$this -> db -> where('invitee_phone !=', $user_phone);
You can go follwoing way too. It work for me
$total = 5;
$CI = get_instance();
$CI->load->database();
$CI->db->order_by('id','asc');
$topusers = $CI->db->get_where('users',array('user_type != 1 && `status` =' => 1),$total,0);
echo $CI ->db ->last_query();
die;
and if still not work for you can go with #rohit suggest: $this->db->query('your query');
Type 1:
Using ->where("column_name !=",$columnname) is fine for one column.
But if you want to check multi columns, you have to form an array inside where clause.
Like this
$whereArray = array(
"employee_name" => $name,
"employee_id !=" => $id,
);
$this->db->select('*')->from('employee')->where($whereArray);
Type 2:
We can just write exactly what we want inside where.
Like
$thi->db->where(("employee_id =1 AND employee name != 'Gopi') OR designation_name='leader#gopis clan'");
Type 2 is good for working with combining queries, i mean paranthesis "()"
you can follow this code:
$query = $this->db->select('*')->from('employee')->where('user_name', $name)->where('user_id !=', $userid)->get();
$last_query = $this->db->last_query();
$result = $query->result_array();
if you pass $name = 'john' and $userid = '1' then it return empty array.
The problem with using $this->db->query('your query'); is that it is not portable. One of the most important reasons to embrace the query builder methods is so that no matter what database driver you use, CodeIgniter ensures that the syntax is appropriate.
If a bit of discussion was possible, I'd probably like to hear why you need composite primary identifiers in your table and I'd like to see what your table schema looks like. However, I think the time for discussion has long passed.
Effectively, you want to return a boolean result stating the availability of the combination of the username AND the id -- if one is matched, but not both, then true (available).
To achieve this, you will want to search the table for an exact matching row with both qualifying conditions, count the rows, convert that integer to a boolean, then return the opposite value (the syntax is simpler than the explanation).
Consider this clean, direct, and portable one-liner.
return !$this->db->where(['user_name' => $name,'user_id' => $userid])->count_all_results('user');
this will return false if the count is > 0 and true if the count is 0.
I have a query to select all the rows from the hire table and display them in a random order.
DB::table('hire_bikes')->order_by(\DB::raw('RAND()'))->get();
I now want to be able to put
concat(SUBSTRING_INDEX(description, " ",25), "...") AS description
into the SELECT part of the query, so that I can select * from the table and a shortened description.
I know this is possible by running a raw query, but I was hoping to be able to do this using Fluent or at least partial Fluent (like above).
How can I do it?
You can actually use select AS without using DB::raw(). Just pass in an array into the select() method like so:
$event = Events::select(['name AS title', 'description AS content'])->first();
// Or just pass multiple parameters
$event = Events::select('name AS title', 'description AS Content');
$event->title;
$event->content;
I tested it.
Also, I'd suggest against using a DB:raw() query to perform a concatenation of your description field. If you're using an eloquent model, you can use accessors and mutators to perform this for you so if you ever need a limited description, you can simply output it in your view and not have to use the same query every time to get a limited description. For example:
class Book extends Eloquent
{
public function getLimitedDescriptionAttribute()
{
return str_limit($this->attributes['description'], $limit = 100, $end = '...');
}
}
In your view:
#foreach($books as $book)
{{ $book->limited_description }}
#endforeach
Example Output (not accurate to limit):
The description of this book is...
I'd also advise against using the DB facade because it always utilizes your default connection. If you're querying a secondary connection, it won't take this into account unless you actively specify it using:
DB::connection('secondary')->table('hire_bikes')->select(['name as title'])->get();
Just to note, if you use a select AS (name AS title) and you wish to update your the model, you will still have to set the proper attribute name that coincides with your database column.
For example, this will cause an exception because the title column does not exist in your database table:
$event = Events::select('name AS title')->first();
$event->title = 'New name';
$event->save(); // Generates exception, 'title' column does not exist.
You can do this by adding a DB::raw() to a select an array in your fluent query. I tested this locally and it works fine.
DB::table('hire_bikes')
->select(
array(
'title',
'url',
'image',
DB::raw('concat(SUBSTRING_INDEX(description, " ",25),"...") AS description'),
'category'
)
)
->order_by(\DB::raw('RAND()'))
->get();
select(array(DB::raw('latitude as lat'), DB::raw('longitude as lon')))