I have a class which extends from yii\rest\Controller, under this class is a function which returns an instance of ArrayDataProvider in JSON format.
public function actionIndex()
{
$user = User::find();
return $this->query($user->asArray()->all(), true);
}
protected function query($qry, $paginate=false)
{
$this->serializer['preserveKeys'] = false;
$dataProvider = new ArrayDataProvider([
'allModels' => $qry,
]);
if ($paginate) {
$dataProvider->setPagination(['pageSize' => 20]);
} else {
$dataProvider->setPagination(false);
}
return $dataProvider;
}
Assuming that the array returned is composed of 130k items (from actionIndex). Do everytime I hit this API with a page parameter on it, the ArrayDataProvider will process the 130k records and sliced it all over again?
Can someone tell what's the exact behavior an ArrayDataProvider is running? Is there any efficient way of handling such big array? What's the perks of using this as opposed with other providers?
Thank you!
Related
I would like to store every query I run inside my application. To do that, I've created a "loggers" table, a Logger modal and this function inside my boot AppServiceProvider
DB::listen(function($query) {
$logger = new Logger;
$logger->query = str_replace('"', '', $query->sql);
$logger->bindings = json_encode($query->bindings);
$logger->created_at = Carbon::now();
$logger->save();
});
Well, anytime I run an insert query, Laravel returns the loggers table ID instead of the model last inserted ID.
Why on earth does this happen?
public function store(CycleRequest $request) {
$appointment = new Appointment;
$appointment-> ... same data ...
$appointment->save();
if ( ! $errors ) ){
$msg['redirect'] = route('appointments.edit', $appointment);
// The page redirects to last logger id
}
}
I think you want to do triggers so I recommend use events for this, o maybe you can take the structure of you table "loggers" and do two differents querys each time that you do querys.
After searching a lot, I found a solution creating a Middleware. Hope this may help someone else.
I've created a QueryLogMiddleware and registered in 'web' middleware, but you may put it everywhere you want:
public function handle($request, Closure $next)
{
\DB::enableQueryLog();
return $next($request);
}
public function terminate($request, $response)
{
$queries = \DB::getQueryLog();
foreach ($queries as $query)
{
// Save $query to the database
// $query["query"]
// $query["bindings"]
...
}
\DB::disableQueryLog();
\DB::flushQueryLog();
}
I have two models in laravel project Item and ItemImgs
Item.php
class Item extends Model
{
protected $appends = [
'photo',
];
public function imgs()
{
return $this->hasMany(ItemImage::class);
}
public function getPhotoAttribute()
{
$img = $this->imgs->first();
return $img.src;
}
}
it's worked in views
dd(Item::all()); //worked
{{ $cane->photo}}; //worked
but when I try to get json
return response()->json([
'items' => Item::with('imgs')->get(),
]);
// not worked. Got timeout 500
You cannot use dot notation in PHP.
public function getPhotoAttribute()
{
$img = $this->imgs->first();
return $img.src; // Dot notation is not allowed
}
but you've to use:
public function getPhotoAttribute()
{
$img = $this->imgs->first();
return $img->src;
}
if what you're trying to do is to get the items that have imgs() then what you should do is query by relationship existence, as mentioned in the docs
https://laravel.com/docs/5.8/eloquent-relationships#querying-relationship-existence
'items' => Item::has('imgs')->get()
It is not possible to refer to the linked model tables in attributes. It works in views but gives out a memory error when outputting an array through json.
public function getPhotoAttribute(){
$img = ItemImage::where('item', $this->id)-
>first();
}
It works that way, but it's not elegant.
I'm working on a marketing application that allows users to message their contacts. When a message is sent, a new "processed_message" database entry is created. There is a list view that displays all campaigns and the number of messages sent, blocked and failed for each campaign. My problem is that this list view takes way too long to load after there are > 50 campaigns with lots of messages.
Currently each campaign has 3 computed attributes (messages_sent, messages_failed and messages_blocked) that are all in the Campaign model's "appends" array. Each attribute queries the count of processed_messages of the given type for the given campaign.
namespace App;
class Campaign
{
protected $appends = [
'messages_sent',
'messages_blocked',
'messages_failed'
];
/**
* #relationship
*/
public function processed_messages()
{
return $this->hasMany(ProcessedMessage::class);
}
public function getMessagesSentAttribute()
{
return $this->processed_messages()->where('status', 'sent')->count();
}
public function getMessagesFailedAttribute()
{
return $this->processed_messages()->where('status', 'failed')->count();
}
public function getMessagesBlockedAttribute()
{
return $this->processed_messages()->where('status', 'blocked')->count();
}
}
I also tried to query all of the messages at once or in chunks to reduce the number of queries but getting all of the processed_messages for a campaing at once will overflow memory and the chunking method is way too slow since it has to use offset. I considered using eager loading the campaigns with processed_messages but that would obviously use way too much memory as well.
namespace App\Http\Controllers;
class CampaignController extends Controller
{
public function index()
{
$start = now();
$campaigns = Campaign::where('user_id', Auth::user()->id)->orderBy('updated_at', 'desc')->get();
$ids = $campaigns->map(function($camp) {
return $camp->id;
});
$statistics = ProcessedMessage::whereIn('campaign_id', $ids)->select(['campaign_id', 'status'])->get();
foreach($statistics->groupBy('campaign_id') as $group) {
foreach($group->groupBy('status') as $messages) {
$status = $messages->first()->status;
$attr = "messages_$status";
$campaign = $campaigns->firstWhere('id', $messages->first()->campaign_id);
$campaign->getStatistics()->$attr = $status;
}
}
return view('campaign.index', [
'campaigns' => $campaigns
]);
}
}
My main goal is to reduce the current page load time considerably (which can take anywhere from 30 seconds to 5 minutes when there are a bunch of campaigns).
You could use the withCount method to count all the objects without loading the relation.
Reference:
If you want to count the number of results from a relationship without actually loading them you may use the withCount method, which will place a {relation}_count column on your resulting models.
In your controller you could do this:
$count = Campaign::withCount(['processed_messages' => function ($query) {
$query->where('content', 'sent');
}])->get();
You could do multiple counts in the same relationship too:
$campaigns = Campaign::withCount([
'processed_messages',
'processed_messages as sent_message_count' => function ($query) {
$query->where('content', 'sent');
}],
'processed_messages as failed_message_count' => function ($query) {
$query->where('status', 'failed');
}],
'processed_messages as blocked_message_count' => function ($query) {
$query->where('status', 'blocked');
}])->get();
You can access the count with this:
echo $campaigns[0]->sent_message_count
Docs
I'm worried about Yii2 Rate limiting api?
What is Rate limiting api, why this used?
Here are some methods from Yii2
Can a yii guru explain in simple words about these methods, where and when I should use rate limiting in my api?
public function getRateLimit($request, $action)
{
return [$this->rateLimit, 1]; // $rateLimit requests per second
}
public function loadAllowance($request, $action)
{
return [$this->allowance, $this->allowance_updated_at];
}
public function saveAllowance($request, $action, $allowance, $timestamp)
{
$this->allowance = $allowance;
$this->allowance_updated_at = $timestamp;
$this->save();
}
THE METHODS
getRateLimit(), loadAllowance() and saveAllowance() are three methods contained in the \yii\filters\RateLimitInterface Inteface that the user identity class should implement for enable rate limiting of your api.
getRateLimit() is the first method and it returns the maximum number of api calls that you can do in x seconds:
public function getRateLimit($request, $action) {
return [1,20]; // There can be 1 api call every 20 seconds
}
loadAllowance() return the number of the remaining allowed requests with the corresponding UNIX timestamp of the last time these where checked.
public function loadAllowance($request, $action)
{
return [$this->allowance, $this->allowance_updated_at];
}
saveAllowance() assign to $this->allowance the value of remaining allowed requests and save the timestamp in $this->allowance_updated_at.
public function saveAllowance($request, $action, $allowance, $timestamp)
{
$this->allowance = $allowance; //Saving Remaining Requests
$this->allowance_updated_at = $timestamp; // Saving Timestamp
$this->save(); //Save the model
}
IMPLEMENTATION
This is how implemented the Rate Limiting in my example application (using advanced template):
1 Set the user identity class
In the config/main.php of your api application set the user component.
'user' => [
'identityClass' => 'api\models\User', // User Model for your api
'enableSession' => false,
'loginUrl' => null,
],
2 Create a user model
This is model should implement the \yii\filters\RateLimitInterface:
This is mine:
class User extends \common\models\User implements \yii\filters\RateLimitInterface
{
public $rateLimit = 1;
public $allowance;
public $allowance_updated_at;
public function getRateLimit($request, $action) {
return [$this->rateLimit,1];
}
public function loadAllowance($request, $action)
{
return [$this->allowance, $this->allowance_updated_at];
}
public function saveAllowance($request, $action, $allowance, $timestamp)
{
$this->allowance = $allowance;
$this->allowance_updated_at = $timestamp;
$this->save();
}
}
After these two step Yii will automatically use yii\filters\RateLimiter configured as an action filter for yii\rest\Controller to perform rate limiting check (as cited in the documentation).
The last thing you have to do is disable the Rate limit header in your rest controller behaviors:
public function behaviors()
{
$behaviors = parent::behaviors();
$behaviors['rateLimiter']['enableRateLimitHeaders'] = false;
return $behaviors;
}
WHEN YOU SHOULD USE RATE LIMITING IN YOUR APPLICATION
Api calls return data (with your filters) from your database so when they're called the server execute queries. More are the calls more are also the number of queries that are execute,
You must limit the number of the calls in order to prevent Server heavy works and a resulting fallout of your system.
Hope this will help.
I'm not going far from the Yii2 Guide, but i don't think i can explain this in a simplier way.
Maybe Yii2 document could help you a lot,and link following,
http://www.yiiframework.com/doc-2.0/guide-rest-rate-limiting.html
you need to alter your user table in Database.The rate limiting takes effect for user loginned
I implemented each step but not show the headers
X-Rate-Limit-Limit, the maximum number of requests allowed with a time period
X-Rate-Limit-Remaining, the number of remaining requests in the current time period
X-Rate-Limit-Reset, the number of seconds to wait in order to get the maximum number of allowed requests
`
you can use rate limithing composer.
add :"ethercreative/yii2-ip-ratelimiter": "1.*"
public function behaviors()
{
$behaviors = parent::behaviors();
$behaviors['rateLimiter'] = [
// Use class
'class' => RateLimiter::className(),
'rateLimit' => 1,
'timePeriod' => 2,
'separateRates' => false,
'enableRateLimitHeaders' => false,
];
return $behaviors;
}
public function getRateLimit($request, $action) {
$id = $action->getUniqueId();
$limits = [
'user/login' => [20,10],
'article/index' => [100,10],
.......
'other' => [50,10]
];
if(!array_key_existe($id,$limits)) $id = 'other';
return $limits[$id];
}
I'm looking for a way to return a model as JSON including an association model after save (within a controller).
I know how to respond as JSON with associations by doing the following :
$objects = MyModel::with(['assocation1', 'association2.dependencies'])->get();
return response()->json($objects, 200);
But in a case of an object already found ? I've tried to use the same concept as above but it returns every rows.
$object = MyModel::first();
$object->with(['assocation1', 'association2.dependencies'])->get();
Laravel's documentation unfortunately does says much about it. What I'm trying to do is to return a JSON object including an association after save, within a controller :
class ExampleController extends Controller {
public function store()
{
$object = new MyModel($request->input('object'));
$response = DB::transaction(function () use ($object) {
if (object()->save()) {
// Here I want to return the object with association1 as JSON
return response()->json($object->with('association1')->get(), 201);
}
});
return $response;
}
}
Edit
More clarification about this case. Using either with or load seems to produce the same result: returning all rows from the Object object including associations. My goal here is to only return ONE object with it's association as JSON, not all of them.
I believe you aren't as far off as you think. In your second example, you shouldn't call get(). Try this instead:
if ( $object = $object->save() )
{
$object->load(['assocation1', 'association2.dependencies']);
return response()->json($object, 201);
}