Our client has a simple setup.
Page A has a form that submits to page B which displays the query results. Unfortunately, there is no other API or DB access to get the data.
Since we need to do this query often, we decided to automate this submission with Perl.
I've determined the form key value pairs of Page A with a sniffer and replicated the code. However, on running the program page B is throwing a HTTP 500 error with no additional meaningful explanation.
Any pointers to debug this code? Code in itself is simple:
use strict;
use warnings;
use LWP;
my $browser = LWP::UserAgent->new;
my $url = "targeturl.asp"
my $response = $browser->post( $url,
[
"HisSort" => "1",
"RTsort" => "",
"chkHisRun" => "on",
"chkRTRun" => "on",
"optAdHoc" => "on",
"optHist" => "",
"optServer" => "servername",
"optStatus" => "",
"optWhat" => "H",
"txtEnd" => "",
"txtFields" => "1,0,10,17,11,18,24,19,21,25,1",
"txtHEnd" => "11/3/2010",
"txtHStart" => "11/1/2010",
"txtServer" => "",
"txtStart" => "",
]
);
Note:
I don't have access to the source of page A or page B
Firstly, I suggest looking at WWW::Mechanize which is a friendlier wrapper around LWP. Secondly, if your HTTP client is getting 500 errors, then there should be something more meaningful in the web server error logs. And finally, as Matthew has mentioned, you need to closely examine the request being sent by the browser and work out how it differs from the request that your Perl program is sending.
Team,
This has been resolved.
It eventually turned out that the problem was not with the headers but with the key value pairs I was sending. Page B wasn't doing validations on the fields and was plugging them into a query directly.
I had to try some brute force combinations (by testing with Page A) to get to what exactly page B was expecting.
Thanks to all who volunteered to help.
Related
I'm looking for a way to save database entities changes for some entities. I mean I need to save in a database table all changes that are done on some tables (add, modify / delete) with ability to track user which did the change.
I'm working on NextJS with a custom ExpressJS server and MYSQL database were I use Prisma as ORM. I think it's maybe possible to write an ExpressJS middleware but I have yet no idea how to do it and asking myself if any library already exist.
Usually I work on PHP Symfony and used to manage this StofDoctrineExtensionsBundle which is great and works as expected. But my current project is a Typescript project only with Express/NextJS/React/Prisma/MYSQL.
Any feedback from your knowledge will be very appreciate.
Thank's in advance.
Regards,
Gulivert
EDIT: My current API which has to be moved to Express/NextJS is still running on Symfony and the table where all changes is logged looks like this :
{
"id": 59807,
"user": "ccba6ad2-0ae8-11ec-813f-0242c0a84005",
"patient": "84c3ef66-548a-11ea-8425-0242ac140002",
"action": "update",
"logged_at": "2021-11-02 17:55:09",
"object_id": "84c3ef66-548a-11ea-8425-0242ac140002",
"object_class": "App\\Entity\\Patient",
"version": 5,
"data": "a:2:{s:10:\"birth_name\";s:2:\"--\";s:10:\"profession\";s:2:\"--\";}",
"username": "johndoe",
"object_name": "patient",
"description": null
}
Explanation about database columns:
user => relation to user table
patient => relation to patient table
action => can be "create"/"update"/delete"
logged_at => date time where the change was done
object_id => entity row ID where an entity get a change
object_class => the entity updated
version => how many time the object was change
data => all data changed during the modification
username => the username of logged user did the change
object_name => a string to identify the object modified without
using the namespace of object_class
description => a value that can be update on some specific change * during usually the action delete to keep a trace what was deleted for instance
You might find prisma middleware useful for this.
Check out the example with session data middleware which is somewhat similar to what you're doing.
For your use-case the middleware might look like something like this:
const prisma = new PrismaClient()
const contextLanguage = 'en-us' // Session state
prisma.$use(async (params, next) => {
// you can find all possible params.action values in the `PrismaAction` type in `.prisma/client/index.d.ts`.
if (params.model == '_modelWhereChangeIsTracked_' && (params.action == 'create' || params.action == "update")) {
// business logic to create an entry into the change logging table using session data of the user.
}
return next(params)
})
// this will trigger the middleware
const create = await prisma._modelWhereChangeIsTracked_.create({
data: {
foo: "bar"
},
})
However, do note that there are some performance considerations when using Prisma middleware.
You can also create express middleware for the routes where you anticipate changes that need to be logged in the change table. Personally, I would prefer this approach in most cases, especially if the number of API routes where changes need to be logged is known in advance and limited in number.
I just set up Google Drive API Change Notifications, specifying the folderId of a certain shared folder that I have on My Drives. It sends notifications if I change the folder name, but doesn't send notifications if I add files.
I set up the channel successfully, as upon issuing these PHP commands
$client = new Google_Client();
$client->setApplicationName('Some Name');
$client->setAuthConfig( __DIR__ . '/service_account.json');
$client->setScopes(Google_Service_Drive::DRIVE);
$client->fetchAccessTokenWithAssertion();
$token = $client->getAccessToken();
$service = new Google\Service\Drive($client);
$httpclient = new \GuzzleHttp\Client();
$folderid = $someid;
$body = [
'kind' => "api#channel",
'id' => uniqid(),
'type' => 'web_hook',
'resourceId' => $folderid,
'resourceUri' => 'https://www.googleapis.com/drive/v3/files/'.$folderid,
'address' => 'myendpoint'
];
$apiendpoint = 'https://www.googleapis.com/drive/v3/files/' . $folderid . '/watch';
$result = $httpclient->post($apiendpoint, [
'headers' => ['Content-Type' => 'application/json', 'Authorization' => "Bearer {$token['access_token']}"],
'body' => json_encode($body),
"http_errors" => false]);
I see a successful return, and my channel immediately sends a sync message to my endpoint, where I have this
function myendpoint (WP_REST_Request $request) {
$headers= getallheaders();
write_log('in google drive webhook with headers '.print_r($headers, true));
$body = $request->get_body();
write_log('in google drive webhook with body '.print_r($body, true));
global $wpdb;
return http_response_code(200);
}
Via my logging, I see
[07-Nov-2021 12:52:35 UTC] in google drive webhook with headers Array
(
[Host] => courses-test.com
[User-Agent] => APIs-Google; (+https://developers.google.com/webmasters/APIs-Google.html)
[Content-Length] => 0
[Accept] => */*
[Accept-Encoding] => gzip, deflate, br
[X-Forwarded-For] => 66.102.8.121
[X-Forwarded-Proto] => https
[X-Goog-Channel-Expiration] => Sun, 07 Nov 2021 13:51:26 GMT
[X-Goog-Channel-Id] => 6187cbc82a08e
[X-Goog-Message-Number] => 1
[X-Goog-Resource-Id] => resid
[X-Goog-Resource-State] => sync
[X-Goog-Resource-Uri] => https://www.googleapis.com/drive/v3/files/folderid?acknowledgeAbuse=false&supportsAllDrives=false&supportsTeamDrives=false&alt=json
[X-Original-Host] => mydomain
)
I read here
https://developers.google.com/drive/api/v3/push
where it says
"To indicate success, you can return any of the following status codes: 200, 201, 202, 204, or 102."
and hence I am returning return http_response_code(200);
and also read here
https://developers.google.com/drive/api/v3/reference/files/watch
and I don't see anything obvious going on here.
Here
https://developers.google.com/search/docs/advanced/crawling/apis-user-agent?visit_id=637718909398077288-2640249237&rd=1
there is this remark
"Avoid unnecessary retry requests by ensuring that your application is well-designed and responds promptly to notification messages (within seconds)."
Now, when I first set up the channel, my endpoint returns right away, so that would seem to negate the above point.
Also, after I get channel set up, I also run these lines
$optParams = array(
'pageSize' => 10,
'fields' => 'nextPageToken, files(id, name)'
);
$results = $service->files->listFiles($optParams);
and indeed see a listing of files.
So I truly at a loss. When I go into the folder, whose folderId I specified to set up the watch channel, and add a file, or edit a file, I don't receive any notifications. The folder in question is shared, but as I understand, sharing only becomes an issue when dealing with shared drives, which is not my case.
And I read here,
https://developers.google.com/drive/api/v3/push#understanding-drive-api-notification-events
that watching for files being added should indeed work on a file watch, when the file is a folder, since add is one of the events for a resource, and it applies to Files.
**Update:
Ok, I seem to have things working, but not based on an add event. The only thing I changed, is that instead of using Guzzle to make the POST, as I show above, I use this
$channel->setId(uniqid());
$channel->setAddress($body['address']);
$channel->setType('web_hook');
$test = $service->files->watch($folderid, $channel);
Maybe adding some of those extra params in the $body like kind, resourceId, was messing things up when I was using POST via Guzzle. Or now that I know resourceId is not same as folderid, maybe that threw a flag on google's end so that a limited number of notifications were being sent. In any case, just using id', address and type seems to be the way to go, which I think is in the Google Drive Push docs. Just that in those docs, also mention of adding those other params in the request, like 'resourceId'. I suspect if I redo using Guzzle, but specify correct resourceId, or just leave it out and use id', address and type, that it would work.
So now, I get this sort of header in my endpoint when I add a file:
(
[Host] => courses-test.com
[User-Agent] => APIs-Google; (+https://developers.google.com/webmasters/APIs-Google.html)
[Content-Length] => 0
[Accept] => */*
[Accept-Encoding] => gzip, deflate, br
[X-Forwarded-For] => someip
[X-Forwarded-Proto] => https
[X-Goog-Changed] => children
[X-Goog-Channel-Expiration] => Mon, 08 Nov 2021 09:46:10 GMT
[X-Goog-Channel-Id] => chanid
[X-Goog-Message-Number] => num
[X-Goog-Resource-Id] => someid
[X-Goog-Resource-State] => update
[X-Goog-Resource-Uri] => https://www.googleapis.com/drive/v3/files/folderid?acknowledgeAbuse=false&supportsAllDrives=false&supportsTeamDrives=false&alt=json
[X-Original-Host] => mydomain
)
So I can detect [X-Goog-Changed] as being children and just keep track of my current uploaded files and see what the new file is. Or maybe using the fields query string somehow I can directly get the new file id?
Still seems odd that getting info about a new file is such a pain. Maybe it is easier in Google Cloud API and Google wants more people signing up for Cloud. In fact that add event only seems to apply to Cloud buckets.
In short: When watching a folder with files().watch() it is currently not possible to receive notifications about new files added to this folder
There are several workarounds that you can find along with explanations in answers to similar questions:
Use changes().watch():
https://stackoverflow.com/a/26551651/11599789 and https://stackoverflow.com/a/67787626/11599789
Pass the folderId to the token:
https://stackoverflow.com/a/33896300/11599789
Make copy of all file's metadata and query for changes:
https://stackoverflow.com/a/18151189/11599789
Also: There is a relevant feature request on Google's Issue Tracker.
It has been closed due to inactivity, but it is worth "starring" and commenting on it to increase visibility.
I'm hoping for a straight-forward solution to do this, but so far I've been coming up empty...
I have a front end vue app/form that sends data back to my laravel backend - I have a controller that validates and saves the request (not looking for feedback on this architecture at the moment unless it actually solves the problem - that's a task for another day...)
I've added a json column called "custom_redeem_fields"
For context, it's to support more flexibility and accepts key/val pairs to use in another field called "custom_redeem_instructions" that has text with delimiters for each of the keys from "custom_redeem_fields", although, I'd prefer to keep from defining these keys statically because the whole point is to be able to add new keys at will. So custom_redeem_instructions will read something like "please visit {•URL•} and enter code {•CODE•}..." and those values will come from the custom_redeem_fields json field.
In the model, I have "custom_redeem_fields" in the fillable array, as well as set as castable to json.
protected $fillable = ['custom_redeem_fields'];
protected $casts = ['custom_redeem_fields' => 'json'];
In the controller, I have ~20 additional columns (not really relevant here, so I've only included two) so I'm trying not to call them out individually beyond their validation rules. The request typically sends one field at a time, so the user can update and save each field as they go. This was working appropriately for all the other fields I had before I added the "custom_redeem_fields.xxxx" to the mix.
$validatedData = $request->validate([
'title' => 'sometimes|required|max:255',
'text' => 'sometimes|required_unless:redeem_type,9|max:255',
'custom_redeem_fields.email' => 'sometimes|email',
'custom_redeem_fields.phone' => ['sometimes', new ValidPhone],
'custom_redeem_fields.code' => 'sometimes',
'custom_redeem_fields.url' => 'sometimes|url'
]);
$ticket = Ticket::find($id)
$ticket->update($validatedData);
Now, with the "custom_redeem_fields.xxxxx" this falls apart - the entire json object stored in "custom_redeem_fields" is overwritten with the most recent update, rather than just updating the key included in the validatedData array. So if I save:
[
"title" => "Monty Pythons Flying Circus"
"text" => "Monty Pythons Flying Circus is a British surreal sketch comedy series created by and starring the comedy group Monty Python, consisting of Graham Chapman, ..."
"custom_redeem_fields" => [
"email" => "bob#example.com",
"phone" => "503.555.5555",
"code" => "1xoicvjq",
"url" => "https://example.com/"
]
]
and then I send:
"custom_redeem_fields" => ["email" => "pat#example.com"]
the custom redeem fields returns:
"custom_redeem_fields" => ["email" => "pat#example.com"]
rather than:
"custom_redeem_fields" => ["email" => "pat#example.com", "phone" => "503.555.5555", "code" => "1xoicvjq", "url" => "https://example.com/"]
It seems that validation rules need json keys to be notated with dot syntax (custom_redeem_fields.url), and eloquent needs arrow syntax (custom_redeem_fields->url), but I'm not sure what's the most straightforward way to transition between the two, which seems very not-laravel, and the documentation is certainly lacking in this department...
Any help would be appreciated.
Thanks!
Wouldn't array_merge() solve your problem, it would overwrite values you provide with the second parameter. If you give it the already existing ones as the first, it would combine the two as you want.
$customRedeemInput = [...];
$model->custom_redeem_fields = array_merge($model->custom_redeem_fields, $customRedeemInput);
$model->save();
I can't seem to use the batcher to copy an activity to notifications. I read from the docs
that there is a 100 limit to the TO field for copying activities, so i tried out using the batcher. but it does'nt seem to work. did I miss out something on the docs or on my code? if so, how do I get over the 100 limit?
$evtConvMention = array();
$evtConvMention[] = "event:{$event->id}";
$evtConvMention[] = "notification:1";
$evtConvMention[] = "notification:2";
$batcher = FeedManager::getClient()->batcher();
$batcher->addToMany([
"actor" => "App\User:{$user->id}",
"verb" => "eventpost",
"object" => "App\EventConversation:{$post->id}",
"foreign_id" => "App\EventConversation:{$post->id}"
], $evtConvMention);
The addToMany() call will have a similar limit. While I look into the PHP library a little more, it might be easier to use the To field in the activity payload itself.
$feed = FeedManager::getClient()->feed("event", $event->id);
$now = new \DateTime("now", new \DateTimeZone('Pacific/Nauru'));
$data = [
"actor" => "App\User:{$user->id}",
"verb" => "eventpost",
"object" => "App\EventConversation:{$post->id}",
"foreign_id" => "App\EventConversation:{$post->id}",
"time" => $now
];
$feed->addActivity($data);
We also HIGHLY recommend sending your own foreign_id and time fields in the payload as well (I've added an idea for the $now value in the code above, otherwise every feed will get its own unique record, which are a limited resource on your account.
If you have more than 100 notification feeds to write this into, it might be better to have the notification feeds for those users 'follow' the event feed. Then you don't need to use the to field at all.
I want to create a link on my error page to take user back to the previous link.
Suppose the current URL is http://example.com/site/product, and a user try to view
http://example.com/site/product?id=100 and a product with id =100 does not exit, the system should throw 404 error to the error page, now if i want to create a link to take the user back to http://example.com/site/product the previous URl how do I make this work. i can make this work by hardcoding this in my error views file, but i want it dynamically as i have many controller an action using the same view file.
I try this in my site conteoller
controller/site
public function actions()
{
$url = Url::remember();
return [
'error' => [
'class' => 'yii\web\ErrorAction',
],
$this->render('error',['url'=>$url]),
];
}
and try to get the value the in error view file like this
/views/site/error.php
<p>
<?= Html::a('go back', [$url)?>
</p>
but it has no vaule..
please any good idea on how to make this work, am also open to new solution
this is form Yii2 Guide http://www.yiiframework.com/doc-2.0/guide-helper-url.html#remember-urls
There are cases when you need to remember URL and afterwards use it
during processing of the one of sequential requests. It can be
achieved in the following way:
// Remember current URL Url::remember();
// Remember URL specified. See Url::to() for argument format.
Url::remember(['product/view', 'id' => 42]);
// Remember URL specified with a name given
Url::remember(['product/view', 'id' => 42], 'product');
In the next
request we can get URL remembered in the following way:
$url = Url::previous();
// or
$productUrl = Url::previous('product');