Using HttpClient in C#, I'm trying to verify that an image exists at a given URL without downloading the actual image. These images can come from any publicly accessible address. I've been using the HEAD HTTP verb which seems to work for many/most. Google drive images are proving difficult.
Given a public share link like so:
https://drive.google.com/file/d/1oCmOEJp0vk73uYhzDTr2QJeKZOkyIm6v/view?usp=sharing
I can happily use HEAD, get a 200 OK and it appears to be happy. But, that's not the image. It's a page where one can download the image.
With a bit of mucking around, you can change the URL to this to actually get at the image, which is what I really want to check:
https://drive.google.com/uc?export=download&id=1oCmOEJp0vk73uYhzDTr2QJeKZOkyIm6v
But, hitting that URL with HEAD results in a 405 MethodNotAllowed
Luckily, if the URL truly doesn't exist, you get back a 404 NotFound
So I'm stuck at the 405. What is my next step (NOT using Google APIs) when HEAD is not allowed? I can't assume it's a valid image if it simply doesn't 404. I check the Content-type to verify it's an image, which has issues outside the scope of this question.
HttpClient allows us to issue an http request where you can specify that you are interested about only the headers.
The trick is to pass an HttpCompletionOption enum value to the SendAsync or any other {HttpVerb}Async method:
| Enum name | Value | Description |
|---------------------|-------|---------------------------------------------------------------------------------------------------------------------|
| ResponseContentRead | 0 | The operation should complete after reading the entire response including the content. |
| ResponseHeadersRead | 1 | The operation should complete as soon as a response is available and headers are read. The content is not read yet. |
await client.GetAsync(targetUrlWhichDoesNotSupportHead, HttpCompletionOption.ResponseHeadersRead);
Here is an in-depth article that details how does this enum changes the behavior and performance of the HttpClient.
The related source code fragments:
in case of .NET Framework
in case of .NET Core
Brilliant, Peter! Thank you.
Here's my full method for anyone who may find it useful:
public async Task<bool> ImageExists(string urlOrPath)
{
try
{
var uri = new Uri(urlOrPath);
if (uri.IsFile)
{
if (File.Exists(urlOrPath)) return true;
_logger.LogError($"Cannot find image: [{urlOrPath}]");
return false;
}
using (var result = await Get(uri))
{
if (result.StatusCode == HttpStatusCode.NotFound)
{
_logger.LogError($"Cannot find image: [{urlOrPath}]");
return false;
}
if ((int)result.StatusCode >= 400)
{
_logger.LogError($"Error: {result.ReasonPhrase}. Image: [{urlOrPath}]");
return false;
}
if (result.Content.Headers.ContentType == null)
{
_logger.LogError($"No 'ContentType' header returned. Cannot validate image:[{urlOrPath}]");
return false;
}
if(new[] { "image", "binary"}.All(v => !result.Content.Headers.ContentType.MediaType.SafeTrim().Contains(v)))
{
_logger.LogError($"'ContentType' {result.Content.Headers.ContentType.MediaType} is not an image. The Url may point to an HTML download page instead of an actual image:[{urlOrPath}]");
return false;
}
var validTypes = new[] { "jpg", "jpeg", "gif", "png", "bmp", "binary" };
if(validTypes.All(v => !result.Content.Headers.ContentType.MediaType.SafeTrim().Contains(v)))
{
_logger.LogError($"'ContentType' {result.Content.Headers.ContentType.MediaType} is not a valid image. Only [{string.Join(", ", validTypes)}] accepted. Image:[{urlOrPath}]");
return false;
}
return true;
}
}
catch (Exception e)
{
_logger.LogError($"There was a problem checking the image: [{urlOrPath}] is not valid. Error: {e.Message}");
return false;
}
}
private async Task<HttpResponseMessage> Get(Uri uri)
{
var response = await _httpCli.SendAsync(new HttpRequestMessage(HttpMethod.Head, uri));
if (response.StatusCode != HttpStatusCode.MethodNotAllowed) return response;
return await _httpCli.SendAsync(new HttpRequestMessage() { RequestUri = uri }, HttpCompletionOption.ResponseHeadersRead);
}
Edit: added a Get() method which still uses HEAD and only uses ResponseHeadersRead if it encounters MethodNotAllowed. Using a live scenario I found it was much quicker. Not sure why. YMMV
Related
I'm learning Blazor.
I have created a Blazor WASM App with the "ASP.NET Core Hosted" option.
So I have 3 projects in the solution: Client, Server and Shared.
The following code is in the Client project and works perfectly when the endpoint is correct (obviously). But at some point I made a mistake and messed up the request URI, and then I noticed that the API returned an HTML page with code 200 OK (as you can see in the Postman screenshot below the code).
I expected one of my try-catches to get this, but the debugger jumps to the last line (return null) without throwing an exception.
My first question is why?
My second question is how can I catch this?
I know fixing the endpoint fixes everything, but would be nice to have a catch that alerts me when I have mistyped an URI.
Thanks.
private readonly HttpClient _httpClient;
public async Task<List<Collaborator>> GetCollaborators()
{
string requestUri = "api/non-existent-endpoint";
try
{
var response = await _httpClient.GetFromJsonAsync<CollaboratorsResponse>(requestUri);
if (response == null)
{
// It never enters here. Jumps to the last line of code.
}
return response.Collaborators;
}
catch (HttpRequestException)
{
Console.WriteLine("An error occurred.");
}
catch (NotSupportedException)
{
Console.WriteLine("The content type is not supported.");
}
catch (JsonException)
{
Console.WriteLine("Invalid JSON.");
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
return null;
}
it is a never good idea to use GetFromJsonAsync, You are not the first who are asking about the strange behavior. Try to use GetAsync. at least you will now what is going on.
var response = await client.GetAsync(requestUri);
if (response.IsSuccessStatusCode)
{
var stringData = await response.Content.ReadAsStringAsync();
var result = JsonConvert.DeserializeObject<CollaboratorsResponse>(stringData);
... your code
}
else
{
var statusCode = response.StatusCode.ToString(); // HERE is your error status code, when you have an error
}
I have an application with both MVC and 'new' ApiController endpoints in ASP.NET Core 2.2 co-existing together.
Prior to adding the API endpoints, I have been using a global exception handler registered as middleware using app.UseExceptionHandler((x) => { ... } which would redirect to an error page.
Of course, that does not work for an API response and I would like to return an ObjectResult (negotiated) 500 result with a ProblemDetails formatted result.
The problem is, I'm not sure how to reliably determine in my 'UseExceptionHandler' lambda if I am dealing with an MVC or a API request. I could use some kind of request URL matching (eg. /api/... prefix) but I would like a more robust solution that won't come back to bite me in the future.
Rough psuedo-code version of what I'm trying to implement is:
app.UseExceptionHandler(x =>
{
x.Run(async context =>
{
// extract the exception that was thrown
var ex = context.Features.Get<IExceptionHandlerFeature>()?.Error;
try
{
// generically handle the exception regardless of what our response needs to look like by logging it
// NOTE: ExceptionHandlerMiddleware itself will log the exception
// TODO: need to find a way to see if we have run with negotiation turned on (in which case we are API not MVC!! see below extensions for clues?)
// TODO: ... could just use "/api/" prefix but that seems rubbish
if (true)
{
// return a 500 with object (in RFC 7807 form) negotiated to the right content type (eg. json)
}
else
{
// otherwise, we handle the response as a 500 error page redirect
}
}
catch (Exception exofex)
{
// NOTE: absolutely terrible if we get into here
log.Fatal($"Unhandled exception in global error handler!", exofex);
log.Fatal($"Handling exception: ", ex);
}
});
});
}
Any ideas?
Cheers!
This might be a bit different than what you expect, but you could just check if the request is an AJAX request.
You can use this extension:
public static class HttpRequestExtensions
{
public static bool IsAjaxRequest(this HttpRequest request)
{
if (request == null)
throw new ArgumentNullException(nameof(request));
if (request.Headers == null)
return false;
return request.Headers["X-Requested-With"] == "XMLHttpRequest";
}
}
And then middleware with an invoke method that looks like:
public async Task Invoke(HttpContext context)
{
if (context.Request.IsAjaxRequest())
{
try
{
await _next(context);
}
catch (Exception ex)
{
//Handle the exception
await HandleExceptionAsync(context, ex);
}
}
else
{
await _next(context);
}
}
private static Task HandleExceptionAsync(HttpContext context, Exception exception)
{
//you can do more complex logic here, but a basic example would be:
var result = JsonConvert.SerializeObject(new { error = "An unexpected error occurred." });
context.Response.ContentType = "application/json";
context.Response.StatusCode = 500;
return context.Response.WriteAsync(result);
}
see this SO answer for a more detailed version.
If you want to check whether the request is routed to ApiController, you could try IExceptionFilter to hanlde the exceptions.
public class CustomExceptionFilter : IExceptionFilter
{
public void OnException(ExceptionContext context)
{
if (IsApi(context))
{
HttpStatusCode status = HttpStatusCode.InternalServerError;
var message = context.Result;
//You can enable logging error
context.ExceptionHandled = true;
HttpResponse response = context.HttpContext.Response;
response.StatusCode = (int)status;
response.ContentType = "application/json";
context.Result = new ObjectResult(new { ErrorMsg = message });
}
else
{
}
}
private bool IsApi(ExceptionContext context)
{
var controllerActionDesc = context.ActionDescriptor as ControllerActionDescriptor;
var attribute = controllerActionDesc
.ControllerTypeInfo
.CustomAttributes
.FirstOrDefault(c => c.AttributeType == typeof(ApiControllerAttribute));
return attribute == null ? false : true;
}
}
Thanks to all of the advice from others, but I have realised after some more thought and ideas from here that my approach wasn't right in the first place - and that I should be handling most exceptions locally in the controller and responding from there.
I have basically kept my error handling middleware the same as if it was handling MVC unhandled exceptions. The client will get a 500 with a HTML response, but at that point there isn't much the client can do anyway so no harm.
Thanks for your help!
Trying to solve the problem referenced in this article: https://philna.sh/blog/2018/10/23/service-workers-beware-safaris-range-request/
and here:
PWA - cached video will not play in Mobile Safari (11.4)
The root problem is that we aren't able to show videos on Safari. The article says it has the fix for the issue but seems to cause another problem on Chrome. A difference in our solution is that we aren't using caching. Currently we just want to pass through the request in our service worker. Implementation looks like this:
self.addEventListener('fetch', function (event){
if (event.request.cache === 'only-if-cached' && event.request.mode !== 'same-origin') {
return;
}
if (event.request.headers.get('range')) {
event.respondWith(returnRangeRequest(event.request));
} else {
event.respondWith(fetch(event.request));
}
});
function returnRangeRequest(request) {
return fetch(request)
.then(res => {
return res.arrayBuffer();
})
.then(function(arrayBuffer) {
var bytes = /^bytes\=(\d+)\-(\d+)?$/g.exec(
request.headers.get('range')
);
if (bytes) {
var start = Number(bytes[1]);
var end = Number(bytes[2]) || arrayBuffer.byteLength - 1;
return new Response(arrayBuffer.slice(start, end + 1), {
status: 206,
statusText: 'Partial Content',
headers: [
['Content-Range', `bytes ${start}-${end}/${arrayBuffer.byteLength}`]
]
});
} else {
return new Response(null, {
status: 416,
statusText: 'Range Not Satisfiable',
headers: [['Content-Range', `*/${arrayBuffer.byteLength}`]]
});
}
});
}
We do get an array buffer returned on the range request fetch but it has a byteLength of zero and appears to be empty. The range header actually contains "bytes=0-" and subsequent requests have a start value but no end value.
Maybe there is some feature detection we can do to determine that it's chrome and we can just call fetch regularly? I'd rather have a solution that works everywhere though. Also res is showing type:"opaque" so maybe that has something to do with it? Not quite sure what to look at next. If we can't solve the problem for Chrome I might need a different solution for Safari.
It seems that it was the opaque response. I didn't realize that fetch was 'nocors' by default. Adding 'cors' mode and overwriting the range header seems to have allowed the rewrite to work on chrome. Sadly, it still doesn't work on Safari, but I was able to access the arrayBuffer after setting the cors values properly.
Here is the change I had to make:
var myHeaders = {};
return fetch(request, { headers: myHeaders, mode: 'cors', credentials: 'omit' })
.then(res => {
return res.arrayBuffer();
})
It's important that the server respond with allowed headers. e.g.
access-control-allow-methods: GET
access-control-allow-origin: *
can someone explain me how to implement caching of JsonResult actions in MVC 5 application?
I want to use caching of some ajax-called actions using [OutputCache()] attribute. Some of these actions return ActionResult with html-content, some JsonResult with serialized lists of {Id, Title} pairs which I'm going to use to construct dropdown lists.
My goal is to reduce amount of DB-queries (while building ViewModels) and server requests (when using ajax-calls for it).
So, my code looks like snippets below:
[OutputCache(Duration=60*60*24)]
public async Task<ActionResult> SearchCaseOrgDialog(){
//extract data return html page
return View();
}
[OutputCache(Duration=60*60*24)]
public async Task<JsonResult> AjaxOrgDepartments(){
//query database, serialize data, return json
var result = await ctx.OrgDepartments
.Select(d => new {
Id = d.Id,
Title = d.Title }
)
.ToListAsync();
return Json(result, JsonRequestBehavior.AllowGet);
}
When I look at FireFox tools-panel I see next picture for Html-content:
Here Firefox uses client-side cached version of ajax-requested page.
But situation differs with json-content:
It doesn't cache content, and seems to transfer data from server (server-side cache).
In both cases response headers look the same:
Cache-Control:"public, max-age=86400, s-maxage=0"
Content is requested using similar ajax-calls like
$.get(url, null, function(data){
//do something with data
});
So, how do I cache json-content? what is the right way to do it, and why default approach does not work?
If you want to avoid DB queries, you should consider caching the data at server side. You can use MemoryCache class to do that.
Quick sample
public class MyLookupDataCache
{
const string categoryCacheKey = "CATEGORYLIST";
public List<string> GetCategories()
{
var cache = MemoryCache.Default;
var items = cache.Get(categoryCacheKey);
if (items != null)
{
CacheItemPolicy policy = new CacheItemPolicy();
policy.AbsoluteExpiration = DateTime.Now.AddDays(7); //7 days
//Now query from db
List<string> newItems = repository.GetCategories();
cache.Set(categoryCacheKey, newItems, policy);
return newItems;
}
else
{
return (List<string>) items;
}
}
}
You can change the method signature to return the type you want. For simplicity, i am using List<String>
I have a JSON request, but it seems that it is not hitting the controller. Here's the jQuery code:
$("#ddlAdminLogsSelectLog").change(function() {
globalLogSelection = $("#ddlAdminLogsSelectLog").val();
alert(globalLogSelection);
$.getJSON("/Administrative/AdminLogsChangeLogSelection", { NewSelection: globalLogSelection }, function(data) {
if (data.Message == "Success") {
globalCurrentPage = 1;
} else if (data.Message == "Error") {
//Do Something
}
});
});
The alert is there to show me if it actually fired the change event, which it does.
Heres the method in the controller:
public ActionResult AdminLogsChangeLogSelection(String NewSelection)
{
String sMessage = String.Empty;
StringBuilder sbDataReturn = new StringBuilder();
try
{
if (NewSelection.Equals("Application Log"))
{
int i = 0;
}
else if (NewSelection.Equals("Email Log"))
{
int l = 0;
}
}
catch (Exception e)
{
//Do Something
sMessage = "Error";
}
return Json(new { Message = sMessage, DataReturn = sbDataReturn.ToString() }, JsonRequestBehavior.AllowGet);
}
I have a bunch of Json requests in my application, and it seems to only happen in this area. This is a separate area (I have 6 "areas" in the app, 5 of which work fine with JSON requests). This controller is named "AdministrativeController", if that matters.
Does anything jump out anyone as being incorrect or why the request would not pass to the server side?
Look at the GET in Firebug or Fiddler.
Either:
There is no GET, in which case your browser cached the results from last time (cough, IE, cough); change the cache policy on the response.
There is a GET, but it doesn't match your route; fix the routing or the JavaScript, as appropriate.
As it turns out, if the Area name and Controller name are the same, it looks like MVC gets a little confused. Im not sure if this is a bug on my side, or something witH MVC, but when I remove the "/" from the name in the Json request (ie. "Administrative/Action" instead of "/Administrative/Action") it works just fine. A colleague was the one to figure this one out for me, he found some forum response on it and showed me what they did. Once I removed the "/" it worked just fine.