laravel hydrateRaw/fromQuery and eager loading with pagination - laravel-5.4

I currently found out that you can hydrate an Raw sql query.
I have following query:
DB::table(DB::raw('(SELECT *, Y(location) AS longitude, X(location) AS latitude FROM meetings WHERE MBRCONTAINS(#quadrat, location)) AS sub'))
->select(DB::raw('(FLOOR(SQRT(POW((#ibk_breite - sub.latitude) * 111, 2) + POW((#ibk_laenge - sub.longitude) * 111 * ABS(COS(RADIANS(#ibk_breite))),2)))) AS distance, sub.*, latitude, longitude'));
which I hydrate as following
$meetings = Meeting::fromQuery($query->toSql());
In the blade view i need to get some additional data from different tables, for example:
$meeting->user
which references to the User Model. But if I'm not complety wrong that would result to a n+1 problem in a for each loop, because I'm not eager loading it?! So is it possible to eager load the required models as you would normally do with
->with('user', 'books', 'etc...')
??
Also is it possible to paginate it like $meetings = $query->paginate(5); and do $meetings->withPath('home');
EDIT:
Found a solution:
// Do your query stuff
// Get count before the query because it won't work with skip and take parameter
$count = $query->count();
$query->skip($skip);
$query->take($meetingsPerPage);
$meetings = Meeting::fromQuery($query->toSql());
$meetings->load('user', 'interest.image', 'img_thumbnail');
$meetings = new LengthAwarePaginator($meetings, $count, $meetingsPerPage);
$meetings->load acts as ->with().
As last step you need to create a paginator. IMPORTANT: Use query->count() before you set skip() and/or take() Otherwise it will not work.
Original answer from laracasts. Theres also another possibily stated that didn't work for me.

You can use setCollection() function with LengthAwarePaginator object (found in Illuminate/Pagination/AbstractPaginator.php). Use load() for eager loading.
$users = DB::table('users')->paginate();
$users->setCollection(User::hydrate($users->items())->load(['category']));

My solution:
// Do your query stuff
// Get count before the query because it won't work with skip and take parameter
$count = $query->count();
$query->skip($skip);
$query->take($meetingsPerPage);
$meetings = Meeting::fromQuery($query->toSql());
$meetings->load('user', 'interest.image', 'img_thumbnail');
$meetings = new LengthAwarePaginator($meetings, $count, $meetingsPerPage);
$meetings->load acts as ->with(). As last step you need to create a paginator. IMPORTANT: Use query->count() before you set skip() and/or take() Otherwise it will not work.
Original answer from laracasts. Theres also another possibily stated that didn't work for me.

Related

Laravel 5.4 formatting result set

Can someone help me convert this query so that my result set is in different format?
$sessions = new Session();
$results = $sessions->where('session_status', $status)->where('application_period_id', (int) ApplicationPeriod::all()->last()->id)->get()->pluck('speaker_id');
$speakers = Speaker::whereIn('id', $results)
->with('session.audiancesession.audiances')
->with('session.subjectsession.subjects')
->with(['session' =>
function ($query) use($status) {
$query->where('session_status', '=', $status);
}])->orderBy('last_name')->get();
This is requested via Ajax(axios)... Now this is how result is formatted:
Obj->data(array of objects)->[0]->name
->address
->session(array of objects)
->[0]->time
->fee
My issue is that my session parameter is array and there can only ever be (1) so I don't need to to be an array and I would like to have object (json) instead.
Thank you!
You might have more success if you change your client-side code to work with an array of sessions each session having its speaker, that means your original query would be like
$sessions = Sessions::with([
'speaker', 'audiancesession.audiances', 'subjectsession.subjects'
])->where('application_period_id', (int) ApplicationPeriod::orderBy('id','DESC')->first())->get();
Note the order by -> first in the ApplicationPeriod makes it so you don't have to get all application periods from the database to memory.
Then your client side should handle an array of sessions.
You can transform the above slightly using to get a similar result to what you need:
$speakers = $sessions->map(function ($session) {
$speaker = collect($session->speaker->toArray());
$speaker->put('session', collect($session->toArray())->except('speaker'));
return $speaker;
})->orderBy('last_name','DESC');
Though I wouldn't guarantee the result here as I've not tested it on your (complex looking) data.

Loop through query results without loading them all in array in Codeigniter [duplicate]

The normal result() method described in the documentation appears to load all records immediately. My application needs to load about 30,000 rows, and one at a time, submit them to a third-party search index API. Obviously loading everything into memory at once doesn't work well (errors out because of too much memory).
So my question is, how can I achieve the effect of the conventional MySQLi API method, in which you load one row at a time in a loop?
Here is something you can do.
while ($row = $result->_fetch_object()) {
$data = array(
'id' => $row->id
'some_value' => $row->some_field_name
);
// send row data to whatever api
$this->send_data_to_api($data);
}
This will get one row at the time. Check the CodeIgniter source code, and you will see that they will do this when you execute the result() method.
For those who want to save memory on large result-set:
Since CodeIgniter 3.0.0,
There is a unbuffered_row function,
All the methods above will load the whole result into memory (prefetching). Use unbuffered_row() for processing large result sets.
This method returns a single result row without prefetching the whole result in memory as row() does. If your query has more than one row, it returns the current row and moves the internal data pointer ahead.
$query = $this->db->query("YOUR QUERY");
while ($row = $query->unbuffered_row())
{
echo $row->title;
echo $row->name;
echo $row->body;
}
You can optionally pass ‘object’ (default) or ‘array’ in order to specify the returned value’s type:
$query->unbuffered_row(); // object
$query->unbuffered_row('object'); // object
$query->unbuffered_row('array'); // associative array
Official Document: https://www.codeigniter.com/userguide3/database/results.html#id2
Well, the thing is that result() gives away the entire reply of the query. row() simply fetches the first case and dumps the rest. However the query can still fetched 30 000 rows regardles of which function you use.
One design that would fit your cause would be:
$offset = (int)#$_GET['offset'];
$query = $this-db->query("SELECT * FROM table LIMIT ?, 1", array($offset));
$row = $query->row();
if ($row) {
/* Run api with values */
redirect(current_url().'?offset'.($offset + 1));
}
This would take one row, send it to api, update the page and use the next row. It will alos prevent the page from having a timeout. However it would most likely take a while with 30 000 records and refreshes, so you may wanna adjust your LIMIT ?, 1 to a higher number than 1 and go result() and foreach() multiple apis per pageload.
Well, there'se the row() method, which returns just one row as an object, or the row_array() method, which does the same but returns an array (of course).
So you could do something like
$sql = "SELECT * FROM yourtable";
$resultSet = $this->db->query($sql);
$total = $resultSet->num_rows();
for($i=0;$i<$total;$i++) {
$row = $resultSet->row_array($i);
}
This fetches in a loop each row from the whole result set.
Which is about the same as fetching everyting and looping over the $this->db->query($sql)->result() method calls I believe.
If you want a row at a time either you make 30.000 calls, or you select all the results and fetch them one at a time or you fetch all and walk over the array. I can't see any way out now.

zend framework automatically alter queries

My database (mysql) tables use TIMESTAMP columns, and whenever I want them returned in a query, I want them to be queried as "UNIX_TIMESTAMP(columnname)".
How do you easily modify queries in zend framework to achieve this?
For example, the current code is:
select = $this->select();
$select->where('user_id = ?',$user_id);
return $this->fetchAll($select);
This eventually becomes:
select * from tablename where user_id = 42;
I want something that automatically finds the TIMESTAMP column and changes the resulting query to:
select user_id,name,unix_timestamp(created) where user_id = 42;
I know I can use a MySQL view to achieve this, but I'd rather avoid that.
Thanks.
RR
You should be able to specify the fields you want in the select using the $select->from() object.
Zend_Db_Select
You should end up with something like this.
$select = $this->select();
$select->from(
array('t' => 'tablename'),
array('user_id', 'name', 'UNIX_TIMESTAMP(created)')
);
$select->where('user_id = ?',$user_id);
return $this->fetchAll($select);
If you wanted to run an expression that doesn't have parenthese in the function, Use the Zend_Db_Expr() method to escape the query properly.

Propel ORM - Custom where clause

I'm trying to match md5(ID) to an id.
SELECT *
FROM `user` u
WHERE
MD5(`user_id`) = '66f041e16a60928b05a7e228a89c3799'
this is ID = 58
I tried something like this. I know I'm close I just don't know what I'm missing
$criteria = new Criteria();
$criteria->addAnd('md5('.User::USER_ID.')', $_REQUEST['fs'], Criteria::CUSTOM);
$user = UserPeer::doSelectOne($criteria);
Any ideas?
First of all, directly using Criteria objects is deprecated not recommended. You should use Active Query classes.
Using these classes, you will be able to write stuff like this :
UserQuery::create()
->where('md5(User.Password) = ?', $_REQUEST['fs'], PDO::PARAM_STR)
->findOne();
You'll notice that I use the PhpName both of the table and the column in the query.
EDIT : For raw conditions, the parameter type has to be specified. You'll find more information on this issue.
After lenghty T&E process I managed to get it done like this
$c = new Criteria();
$c->add(UserPeer::USER_ID, "md5(user.user_id) = \"".$_REQUEST['fs']."\"", Criteria::CUSTOM); // risk of SQL injection!!
$saved_search = UserPeer::doSelectOne($c);
For some reason PropelORM though that $_REQUEST['fs'] was name of the table rather than the value. \"" solved the problem.

How to get Ruby MySQL returning PHP like DB SELECT result

So I use the PDO for a DB connection like this:
$this->dsn[$key] = array('mysql:host=' . $creds['SRVR'] . ';dbname=' . $db, $creds['USER'], $creds['PWD']);
$this->db[$key] = new PDO($this->dsn[$key]);
Using PDO I can then execute a MySQL SELECT using something like this:
$sql = "SELECT * FROM table WHERE id = ?";
$st = $db->prepare($sql);
$st->execute($id);
$result = $st->fetchAll();
The $result variable will then return an array of arrays where each row is given a incremental key - the first row having the array key 0. And then that data will have an array the DB data like this:
$result (array(2)
[0]=>[0=>1, "id"=>1, 1=>"stuff", "field1"=>"stuff", 2=>"more stuff", "field2"=>"more stuff" ...],
[1]=>[0=>2, "id"=>2, 1=>"yet more stuff", "field1"=>"yet more stuff", 2=>"even more stuff", "field2"=>"even more stuff"]);
In this example the DB table's field names would be id, field1 and field2. And the result allows you to spin through the array of data rows and then access the data using either a index (0, 1, 2) or the field name ("id", "field1", "field2"). Most of the time I prefer to access the data via the field names but access via both means is useful.
So I'm learning the ruby-mysql gem right now and I can retrieve the data from the DB. However, I cannot get the field names. I could probably extract it from the SQL statement given but that requires a fair bit of coding for error trapping and only works so long as I'm not using SELECT * FROM ... as my SELECT statement.
So I'm using a table full of State names and their abbreviations for my testing. When I use "SELECT State, Abbr FROM states" with the following code
st = #db.prepare(sql)
if empty(where)
st.execute()
else
st.execute(where)
end
rows = []
while row = st.fetch do
rows << row
end
st.close
return rows
I get a result like this:
[["Alabama", "AL"], ["Alaska", "AK"], ...]
And I'm wanting a result like this:
[[0=>"Alabama", "State"=>"Alabama", 1=>"AL", "Abbr"=>"AL"], ...]
I'm guessing I don't have the way inspect would display it quite right but I'm hoping you get the idea by now.
Anyway to do this? I've seen some reference to doing this type of thing but it appears to require the DBI module. I guess that isn't the end of the world but is that the only way? Or can I do it with ruby-mysql alone?
I've been digging into all the methods I can find without success. Hopefully you guys can help.
Thanks
Gabe
You can do this yourself without too much effort:
expanded_rows = rows.map do |r|
{ 0 => r[0], 'State' => r[0], 1 => r[1], 'Abbr' => r[1] }
end
Or a more general approach that you could wrap up in a method:
columns = ['State', 'Abbr']
expanded_rows = rows.map do |r|
0.upto(names.length - 1).each_with_object({}) do |i, h|
h[names[i]] = h[i] = r[i]
end
end
So you could collect up the rows as you are now and then pump that array of arrays through something like what's above and you should get the sort of data structure you're looking for out the other side.
There are other methods on the row you get from st.fetch as well:
http://rubydoc.info/gems/mysql/2.8.1/Mysql/Result
But you'll have to experiment a little to see what exactly they return as the documentation is, um, a little thin.
You should be able to get the column names out of row or st:
http://rubydoc.info/gems/mysql/2.8.1/Mysql/Stmt
but again, you'll have to experiment to figure out the API. Sorry, I don't have anything set up to play around with the MySQL API that you're using so I can't be more specific.
I realize that php programmers are all cowboys who think using a db layer is cheating, but you should really consider activerecord.