I'm looking for a "best practice" (besides "don't do that") when using Puppeteer with a page that may (but not always) reload when a radio button is clicked, a select option is selected, etc. The use case is I'm navigating an eCommerce page with options, and some of those options cause the page to reload, some of them don't.
I've tried hooking into the on("load") event to try and catch when this happens, and reset my page variable, but I can't get the "timing" correct, and still end up with execution context exceptions. What I've resorted to is try/catching the execution context exceptions in a loop so I can retry. Below is sort of what I'm talking about, and I'm in violent hatred of it (it's ugly, verbose, etc.):
async evaluate(fn: EvaluateFn, ...args: any[]): Promise<any> {
let result: ElementHandle | null = null;
let page: Page | undefined = undefined;
for(let attempt = 0; attempt < this.MAX_EXECUTION_RETRIES; attempt++) {
try {
if(page == undefined) {
// this.browser is an earlier instantiate instance of a Puppeteer Browser
const pages = await this.browser.pages();
page = pages[pages.length = 1];
}
result = await page.evaluate(fn, ...args);
break;
} catch(e) {
if(attempt < this.MAX_EXECUTION_RETRIES - 1 && (e.message && e.message.indexOf("Execution context") != -1)) {
await sleep(this.EXECUTION_RETRY_DELAY);
page = undefined;
} else {
throw e;
}
}
}
return result;
}
Is there a better way to get notified or check if my page variable's execution context is no longer valid?
Related
Using the looker embedsdk the connect() promise never resolves using an sso url.
Using the debugger in my dev environment I can see at this point in the #looker/chatty code there is a switch against evt.data.action which is always undefined. The debug in the line above fires many times but I just see:
looker:chatty:host window received +0ms undefined undefined
In the debugger I can see evt.data = {"type":"page:changed","page":{"type":"dashboard","absoluteUrl":"https://company.looker.com:19999/embed/dashboards/104?embed_domain=http://localhost:3100","url":"/embed/dashboards/104?embed_domain=http://localhost:3100"}}
So evt.data.data and evt.data.action are both undefined.
The iframe loads the dashboard fine. I’ve added localhost:3100 to the embed allow list and I’m including ?embed_domain=http://localhost:3100&sdk=2 in the target url when creating the sso url.
This is my code:
const [embed, setEmbed] = useState<LookerEmbedDashboard>();
const embedUrl = useLookerSSOEmbed(dashboard);
const embedCtrRef = useCallback(
(el) => {
if (el && embedUrl) {
el.innerHTML = '';
LookerEmbedSDK.init('https://company.looker.com:19999');
LookerEmbedSDK.createDashboardWithUrl(embedUrl)
.appendTo(el)
.build()
.connect()
.then((dashboard) => {
setEmbed(dashboard);
console.log('dashboard', dashboard);
})
.catch((error) => console.log('error', error));
}
},
[embedUrl]
);
return <EmbedContainer ref={embedCtrRef}></EmbedContainer>;
};
This is the part of the code in #looker/chatty host.js That it dies in. The single case in the switch never hits since evt.data.action is always undefinded:
var windowListener = function (evt) {
if (!_this.isValidMsg(evt)) {
// don't reject here, since that breaks the promise resolution chain
ChattyHost._debug('window received invalid', evt);
return;
}
ChattyHost._debug('window received', evt.data.action, evt.data.data);
switch (evt.data.action) {
case client_messages_1.ChattyClientMessages.Syn:
if (_this._port) {
// If targetOrigin is set and we receive another Syn, the frame has potentially
// navigated to another valid webpage and we should re-connect
if ((_this._targetOrigin && _this._targetOrigin === '*') ||
_this._targetOrigin === evt.origin) {
ChattyHost._debug('reconnecting to', evt.origin);
_this._port.close();
}
else {
ChattyHost._debug('rejected new connection from', evt.origin);
return;
}
}
_this._port = evt.ports[0];
_this._port.onmessage = eventListener;
_this.sendMsg(host_messages_1.ChattyHostMessages.SynAck);
_this._state = ChattyHostStates.SynAck;
break;
}
};
I'm learning Blazor.
I have created a Blazor WASM App with the "ASP.NET Core Hosted" option.
So I have 3 projects in the solution: Client, Server and Shared.
The following code is in the Client project and works perfectly when the endpoint is correct (obviously). But at some point I made a mistake and messed up the request URI, and then I noticed that the API returned an HTML page with code 200 OK (as you can see in the Postman screenshot below the code).
I expected one of my try-catches to get this, but the debugger jumps to the last line (return null) without throwing an exception.
My first question is why?
My second question is how can I catch this?
I know fixing the endpoint fixes everything, but would be nice to have a catch that alerts me when I have mistyped an URI.
Thanks.
private readonly HttpClient _httpClient;
public async Task<List<Collaborator>> GetCollaborators()
{
string requestUri = "api/non-existent-endpoint";
try
{
var response = await _httpClient.GetFromJsonAsync<CollaboratorsResponse>(requestUri);
if (response == null)
{
// It never enters here. Jumps to the last line of code.
}
return response.Collaborators;
}
catch (HttpRequestException)
{
Console.WriteLine("An error occurred.");
}
catch (NotSupportedException)
{
Console.WriteLine("The content type is not supported.");
}
catch (JsonException)
{
Console.WriteLine("Invalid JSON.");
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
return null;
}
it is a never good idea to use GetFromJsonAsync, You are not the first who are asking about the strange behavior. Try to use GetAsync. at least you will now what is going on.
var response = await client.GetAsync(requestUri);
if (response.IsSuccessStatusCode)
{
var stringData = await response.Content.ReadAsStringAsync();
var result = JsonConvert.DeserializeObject<CollaboratorsResponse>(stringData);
... your code
}
else
{
var statusCode = response.StatusCode.ToString(); // HERE is your error status code, when you have an error
}
I can't find simple answer, but my code is simple.
I tried something like that, but always when i try to console.log my testResult, then i always recieving null. How to save data from file correctly?
public getFile(
sourceFile: File
): string {
let testResult;
const file = sourceFile[0]
const fileReader = new FileReader();
fileReader.readAsText(file, "UTF-8")
fileReader.onloadend = (e) => {
testResult = fileReader.result.toString()
}
console.log(testResult)
return testResult
}
This problem is related to my other topics, main reason is i can't handle load json file, translate them and upload to user. If i can save this file outside onloadend, then i hope i can handle rest of them (other attempts failed, this one blocking me at beginning)
Your issue is quite classical and is related to the asynchronous operations. Function which you assign to the onloadend request is called only when loadend event fires, but the rest of code will not wait for that to happen and will continue execution. So console.log will be executed immediately and then return will actually return testResult while it is still empty.
Firstly, in order to understand what I just said, put the console.log(testResult) line inside of your onloadend handler:
fileReader.onloadend = (e) => {
testResult = fileReader.result.toString();
console.log(testResult);
}
At this point testResult is not empty and you may continue handling it inside this function. However, if you want your getFile method to be really reusable and want it to return the testResult and process it somewhere else, you need to wrap this method into a Promise, like this:
public getFile(
sourceFile: File
): Promise<string> {
return new Promise((resolve) => {
const file = sourceFile[0]
const fileReader = new FileReader();
fileReader.onloadend = (e) => {
const testResult = fileReader.result.toString();
resolve(testResult);
}
fileReader.readAsText(file, "UTF-8");
});
}
Now whereever you need a file you can use the yourInstance.getFile method as follows:
yourInstance.getFile().then(testResult => {
// do whatever you need here
console.log(testResult);
});
Or in the async/await way:
async function processResult() {
const testResult = await yourInstance.getFile();
// do whatever you need
console.log(testResult);
}
If you are now familiar with promises and/or async/await, please read more about here and here.
I'm trying to call an injected HttpClient during operations within a Razor Component. When I do so during OnInitialized, the return is as expected. When I do so on an event like an input change, the client call doesn't respond.
I'm using a mix of MVC Controllers/Views with Razor Components in .Net Core 3.1.
Startup.cs
services.AddControllersWithViews()...
services.AddRazorPages()...
services.AddHttpClient<IJiraService, JiraService>("jira", c =>
{
c.BaseAddress = new Uri(Configuration.GetSection("ApplicationSettings:Jira:Url").Value);
var auth =
$"{Configuration.GetSection("ApplicationSettings:Jira:UserName").Value}:{Configuration.GetSection("ApplicationSettings:Jira:Password").Value}";
var authHeaderValue = Convert.ToBase64String(Encoding.UTF8.GetBytes(auth));
c.DefaultRequestHeaders.AddAuthorization("Basic", authHeaderValue);
c.Timeout = TimeSpan.FromSeconds(20);
c.DefaultRequestHeaders.CacheControl = new CacheControlHeaderValue
{
NoCache = true
};
});
ChildComponent.razor
#inject IJiraService JiraService
#code {
public int SelectedReleaseId
{
get => ReleaseModel.SelectedReleaseId;
set
{
ReleaseModel.SelectedReleaseId = value;
ReleaseChanged().Wait();
}
}
}
#functions
{
private async Task ReleaseChanged()
{
if (ReleaseModel.SelectedReleaseId > 0)
{
var url = "...";
await JiraService.GetResponseAsync(url);
}
}
JiraService.cs
public async Task<string> GetResponseAsync(string url)
{
var resp = await httpClient.GetAsync(url); // <--- this is the call that never returns when invoked from an input control event
var respContentString = await resp.Content.ReadAsStringAsync();
if (resp.StatusCode != HttpStatusCode.OK)
{
throw new HttpOperationException(
$"Invalid response from Jira service: {resp.StatusCode}: {respContentString}");
}
return respContentString;
}
There's actually a bit of service classing in between, but this is the jist.
I've abstracted the call up to a parent component and implemented EventCallbacks all with the same result. The underlying call in the JiraService gets hit and i see a breakpoint stop on the await httpClient.GetAsync(url); but then execution just goes into the ether. There's not even an exception thrown or timeout.
It all seems so obvious now. The problem was a deadlock. This old post helped me realize that my property based #bind attribute was synchronously calling into an async/await graph. I refactored this into an #onchange function that enabled appropriate async/await behavior through the call stack and viola, await httpClient.GetAsync() behaved just like it should.
A little annoyed at the #bind behavior that takes the onchange event functionality in addition to the property value.
Using HttpClient in C#, I'm trying to verify that an image exists at a given URL without downloading the actual image. These images can come from any publicly accessible address. I've been using the HEAD HTTP verb which seems to work for many/most. Google drive images are proving difficult.
Given a public share link like so:
https://drive.google.com/file/d/1oCmOEJp0vk73uYhzDTr2QJeKZOkyIm6v/view?usp=sharing
I can happily use HEAD, get a 200 OK and it appears to be happy. But, that's not the image. It's a page where one can download the image.
With a bit of mucking around, you can change the URL to this to actually get at the image, which is what I really want to check:
https://drive.google.com/uc?export=download&id=1oCmOEJp0vk73uYhzDTr2QJeKZOkyIm6v
But, hitting that URL with HEAD results in a 405 MethodNotAllowed
Luckily, if the URL truly doesn't exist, you get back a 404 NotFound
So I'm stuck at the 405. What is my next step (NOT using Google APIs) when HEAD is not allowed? I can't assume it's a valid image if it simply doesn't 404. I check the Content-type to verify it's an image, which has issues outside the scope of this question.
HttpClient allows us to issue an http request where you can specify that you are interested about only the headers.
The trick is to pass an HttpCompletionOption enum value to the SendAsync or any other {HttpVerb}Async method:
| Enum name | Value | Description |
|---------------------|-------|---------------------------------------------------------------------------------------------------------------------|
| ResponseContentRead | 0 | The operation should complete after reading the entire response including the content. |
| ResponseHeadersRead | 1 | The operation should complete as soon as a response is available and headers are read. The content is not read yet. |
await client.GetAsync(targetUrlWhichDoesNotSupportHead, HttpCompletionOption.ResponseHeadersRead);
Here is an in-depth article that details how does this enum changes the behavior and performance of the HttpClient.
The related source code fragments:
in case of .NET Framework
in case of .NET Core
Brilliant, Peter! Thank you.
Here's my full method for anyone who may find it useful:
public async Task<bool> ImageExists(string urlOrPath)
{
try
{
var uri = new Uri(urlOrPath);
if (uri.IsFile)
{
if (File.Exists(urlOrPath)) return true;
_logger.LogError($"Cannot find image: [{urlOrPath}]");
return false;
}
using (var result = await Get(uri))
{
if (result.StatusCode == HttpStatusCode.NotFound)
{
_logger.LogError($"Cannot find image: [{urlOrPath}]");
return false;
}
if ((int)result.StatusCode >= 400)
{
_logger.LogError($"Error: {result.ReasonPhrase}. Image: [{urlOrPath}]");
return false;
}
if (result.Content.Headers.ContentType == null)
{
_logger.LogError($"No 'ContentType' header returned. Cannot validate image:[{urlOrPath}]");
return false;
}
if(new[] { "image", "binary"}.All(v => !result.Content.Headers.ContentType.MediaType.SafeTrim().Contains(v)))
{
_logger.LogError($"'ContentType' {result.Content.Headers.ContentType.MediaType} is not an image. The Url may point to an HTML download page instead of an actual image:[{urlOrPath}]");
return false;
}
var validTypes = new[] { "jpg", "jpeg", "gif", "png", "bmp", "binary" };
if(validTypes.All(v => !result.Content.Headers.ContentType.MediaType.SafeTrim().Contains(v)))
{
_logger.LogError($"'ContentType' {result.Content.Headers.ContentType.MediaType} is not a valid image. Only [{string.Join(", ", validTypes)}] accepted. Image:[{urlOrPath}]");
return false;
}
return true;
}
}
catch (Exception e)
{
_logger.LogError($"There was a problem checking the image: [{urlOrPath}] is not valid. Error: {e.Message}");
return false;
}
}
private async Task<HttpResponseMessage> Get(Uri uri)
{
var response = await _httpCli.SendAsync(new HttpRequestMessage(HttpMethod.Head, uri));
if (response.StatusCode != HttpStatusCode.MethodNotAllowed) return response;
return await _httpCli.SendAsync(new HttpRequestMessage() { RequestUri = uri }, HttpCompletionOption.ResponseHeadersRead);
}
Edit: added a Get() method which still uses HEAD and only uses ResponseHeadersRead if it encounters MethodNotAllowed. Using a live scenario I found it was much quicker. Not sure why. YMMV