How cookies are managed by networking code - function

I was recently migrating over from C# and looking to create some of my old applications. As such I have needed to find a way to manage sessions within Go web requests. I found a solution in the form of this code:
// Jar is session object struct - cookie jar including mutex for syncing
type Jar struct {
sync.Mutex
cookies map[string][]*http.Cookie
}
// NewJar is a function for creating cookie jar for use
func NewJar() *Jar {
jar := new(Jar)
jar.cookies = make(map[string][]*http.Cookie)
return jar
}
// SetCookies sets the cookies for the jar
func (jar *Jar) SetCookies(u *url.URL, cookies []*http.Cookie) {
jar.Lock()
if _, ok := jar.cookies[u.Host]; ok {
for _, c := range cookies {
jar.cookies[u.Host] = append(jar.cookies[u.Host], c)
}
} else {
jar.cookies[u.Host] = cookies
}
jar.Unlock()
}
// Cookies returns cookies for each host
func (jar *Jar) Cookies(u *url.URL) []*http.Cookie {
return jar.cookies[u.Host]
}
// NewJarClient creates new client, utilising a NewJar()
func NewJarClient() *http.Client {
proxyURL, _ := url.Parse("http://127.0.0.1:8888")
tr := &http.Transport{
MaxIdleConns: 10,
IdleConnTimeout: 30 * time.Second,
DisableCompression: true,
Proxy: http.ProxyURL(proxyURL),
}
return &http.Client{
Jar: NewJar(),
Transport: tr,
}
}
The problem I'm having is in understanding how this works. I create a client doing the following
client := NewJarClient()
but then when I issue networking fuctions using it such as a get request, the cookies automatically carry on and it all works as planned. The problem is Ihave no idea why. I see no mention of methods such as the Cookies one or the SetCookies one ever being called and it seems to just handle each one by magically running the functions. Could someone annotate or explain the given methods line by line or in a way so that they'd make better sense to me coming over from a C# background. Thanks :)

NewJar allocates and returns a new instance of type *Jar, now type *Jar, thanks to the methods defined on it, implements the interface called CookieJar, implicitly.
The http.Client type has a field called Jar which is defined as having the type CookieJar, that means that you can set http.Client.Jar to anything that implements the CookieJar interface, including the *Jar type. The NewJarClient function returns a new *http.Client instance with it's Jar field set to the *Jar instance returned by NewJar.
This allows the returned client value to use *Jar's methods without really knowing that it's a *Jar, it only knows that the value in its Jar field has the same set of methods as those defined by the CookieJar interface.
So the http.Client instance, when sending requests, uses your *Jar by calling its methods providing the parameters and handling the returned values. How your *Jar is used by the client is an implementation detail of the http.Client type and you don't have to worry about that. You just need to make sure that the methods that implement the CookieJar interface do what you want them to do, how and when they are called is up to the client.
But if you're interested in the implementation details of the client anyway, you can check out the source file of http.Client.

Due to misinformation in the form of a few dated blog posts, I came to the impression that I was unable to maintain cookies across requests in go - for some weird reason. Having thought that, I researched and looked into creating my own implementation which ca be seen above. It's been brought to my attention that my implementation is completely broken and flawed and that the standard http library itself can perfectly handle maintaining cookies, simply by including a value for the Jar when creating a client. For example:
jar, _ := cookiejar.New(nil)
proxyURL, _ := url.Parse("http://127.0.0.1:8888")
tr := &http.Transport{
MaxIdleConns: 10,
IdleConnTimeout: 30 * time.Second,
DisableCompression: true,
Proxy: http.ProxyURL(proxyURL),
}
c := &http.Client{
Jar: jar,
Transport: tr,
}

Related

Exception: '<' is an invalid start of a value

I have a Blazor Webassembly project with a controller method as follows:
[HttpGet]
public async Task<List<string>> GetStatesForProfile()
{
IConfigurationSection statesSection = configuration.GetSection("SiteSettings:States");
var sections = statesSection.GetChildren();
var states = statesSection.GetChildren().Select(s => s.Key).ToList<string>();
return states;
}
The razor page calls this method:
private async Task<bool> GetStatesModel()
{
try
{
States = await http.GetJsonAsync<List<string>>("api/account/getstatesforprofile");
...
}
catch (Exception ex)
{
Console.WriteLine($"Exception: {ex.Message}, Inner: {ex.InnerException.Message}");
}
I get this Exception:
Exception: '<' is an invalid start of a value.
I read these values from appsettings.json file, And there is no '<' in values.
{
"SiteSettings": {
"States": {
"New York": ["NYC"],
"California": ["Los Angeles", "San Francisco"]
}
}
Also I put a breakpoint in the controller method and it doesn't hit.
What is this error? Is it from parsing json? and how to resolve this?
I had a very similar problem.
In the end it turned out that my browser had cached the HTML error page (I guess I had some problems with the code when I first tried it). And no matter how I tried fixing the code I still only got the error from cache. Clearing my cache also cleared the problem.
It happens when you're trying to access an API that doesn't exist. You have to check your API project connectionstring under AppSettings and make sure it's correct and running. If it's a Blazor project, you can set it as your default project, execute and see if you get a json response.
Most probably the response you are receiving is html instead of actual JSON format for the endpoint you are requesting. Please check that.
An as HTML usually starts with <html> tag, the JSON validator fails on the very first character.
You should also clear any cache, that might be interfering with the returned data. (this has helped people resolve this same issue)
I know this is an old question, but it's one of the top results when Googling the error.
I've just spent more time than I care to admit to tracking down this error. I had a straightforward Blazor hosted app, basically unchanged from the template. It worked just fine when run locally, but when published to my web host API calls failed. I finally figured out that the problem was that I was running the publish from the Client project. When I changed to the Server project it worked properly.
Hopefully my long frustration and slight stupidity will save someone else making a similar mistake.
Seems like your api is not not accessible and its returning error HTML page by default.
You can try below solution:-
I think you are using httpclient to get data to blazor application.
If you have separate projects in solution for blazor and web api,
currently your startup application may set to run blazor project only.
Change startup projects to multiple (blazor and web api app) and give httpClient url in startup of blazor application, as webApi application url, that may solve your issue.
This error indicates a mismatch of the project targeting framework version and installed runtime on the machine. So make sure that the target framework for your project matches an installed runtime - this could be verified by multiple means; one of them is to check out the Individual Components tab of the Visual Studio Installer and lookup the target version.
E.g., there is the TargetFramework attribute in the proj file:
<PropertyGroup>
<TargetFramework>net5.0</TargetFramework>
</PropertyGroup>
Then launch the Visual Studio Installer, click Modify, and visit the Individual Components tab:
Install the missing runtime (.NET 5 Runtime in this case) and you're good to go.
I got the same error. Red herring. use your browser or postman to check your api endpoint is returning the json data and not some HTML. In my case my "api/companytypes" had a typo.
private CompanyType[] companytypesarray;
private List<CompanyType> CompanyTypeList;
private List<CompanyType> CompanyTypeList2;
public async Task<bool> LoadCompanyTypes()
{
//this works
CompanyTypeList = await Http.GetFromJsonAsync<List<CompanyType>>("api/companytype");
//this also works reading the json into an array first
companytypesarray = await Http.GetFromJsonAsync<CompanyType[]>("api/companytype");
CompanyTypeList2 = companytypesarray.ToList();
return true;
}
I know this is an old question, but I had the same problem. It took some searching, but I realized that the return data was in XML instead of JSON.
I'm assuming your "http" variable is of type HttpClient, so here's what I found worked for me.
By setting the "Accept" header to allow only JSON, you avoid a miscommunication between your app and the remote server.
http.DefaultRequestHeaders.Add("Accept", "application/json");
States = await http.GetJsonAsync<List<string>>("api/account/getstatesforprofile");
I had the same issue when passing in an empty string to a controller method. Creating a second controller method that doesn't accept any input variables, and just passing an empty string to the first method helped to fix my problem.
[HttpGet]
[ActionName("GetStuff")]
public async Task<IEnumerable<MyModel>> GetStuff()
{
return await GetStuff("");
}
[HttpGet("{search}")]
[ActionName("GetStuff")]
public async Task<IEnumerable<MyModel>> GetStuff(string search)
{
...
}
Versions of package
Try to update your packages to old or new version. In my case, system.net.http.json is updated from 6.0 to 5.0
Likely you are using an Asp.NetCore hosted WASM application. By default the client's App.razor has something similar to:
<CascadingAuthenticationState>
<Router AppAssembly="#typeof(Program).Assembly">
<Found Context="routeData">
<AuthorizeRouteView DefaultLayout="#typeof(MainLayout)"
RouteData="#routeData">
<NotAuthorized>
<RedirectToLogin />
</NotAuthorized>
<Authorizing>
<Loading Caption="Authorizing..."></Loading>
</Authorizing>
</AuthorizeRouteView>
</Found>
<NotFound>
<LayoutView Layout="#typeof(MainLayout)">
<p>Sorry, there's nothing at this address.</p>
</LayoutView>
</NotFound>
</Router>
</CascadingAuthenticationState>
Herein lies the problem. Since the Client and Server share the same base address, when the application cannot find "api/account/getstatesforprofile" it gives you the client's "Sorry, there's nothing at the address" page. Which is of course HTML.
I have not found the solution to this issue, but I am working on it and will reply once I find an issue.
I was having the same problem,
"JsonReaderException: '<' is an invalid start of a value."
In my case the url for the REST service was wrong.
I was using the URL from the client project. Then I looked at the Swagger screen,
https://localhost:44322/swagger/index.html
and noticed the right URL should start with "44322"...
Corrected, worked.
In my case, I had a comma (,) written mistakenly at the beginning of the appsettings.json file ...
Just check your file and verify
///////
my error details
//////
System.FormatException HResult=0x80131537 Message=Could not parse the JSON file.
Source=Microsoft.Extensions.Configuration.Json StackTrace: at line 16 This exception was originally thrown at this call stack: [External Code] Inner Exception 1: JsonReaderException: ',' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.
////
For me, most of the time it is the #lauri-peltonen answer above. However, now and again, depending on who wrote the controller I have found that this will work in Swagger but not when you call it via the client (at least in this Blazor project we are on.)
[HttpGet]
[Route("prog-map-formulations")]
public async Task<List<GetProgramMapFormulationsResult>> GetProgramMapFormulations(int formulationId)
{
...
}
It sends the request as:
api/formulation-performance-program-map/analytical-assoc-values?formulationId=1
And I get results in Swagger but failes with the '<' OP error.
When I change ONLY the route to:
[HttpGet]
[Route("prog-map-formulations/{formulationId:int}")]
public async Task<List<GetProgramMapFormulationsResult>> GetProgramMapFormulations(int formulationId)
{
...
}
It sends the request as:
api/formulation-performance-program-map/analytical-assoc-values/1
And this works in both Swagger as well as from the Client side in Blazor.
Of course, once updated, I did have to clear the cache!
If you delete "obj" folder in your directory then clean the solution and rebbuild it the exception will be resolved
In all these, there is two things that was my issue and realized, first off was that Route[("api/controller")] instead of Route[("api/[controller]")], that is missing square brackets. In the second exercise I was doing, with the first experience in mind, was from the name of the database. The database had a dot in the name (Stock.Inventory). When I change the database name to StockInventory it worked. The second one I am not so sure but it worked for me.

Server fails to launch in Google App Engine; OK in Localhost

I have a Flex App written in Go and React that is deployed to Google App engine. I would like it to interact with a MySql Database (2nd generation) on Google Cloud over a Unix socket. I believe the issue lies with the Go server not launching/responding to requests (see below for justification). The App is located at https://haveibeenexploited.appspot.com/
The project is simple. I have two routes in my Server:
server.go
package main
import (
"net/http"
"searchcontract"
)
func main() {
http.Handle("/", http.FileServer(http.Dir("./app/build")))
http.HandleFunc("/search", searchcontract.SearchContract)
http.ListenAndServe(":8080", nil)
}
The second route ("/search") is activated when a user hits the search button. Ideal behavior should return a row specifying the exploits available for the given "contract address" which React writes out to the screen.
searchcontract/searchcontract.go
//SearchContract is a handler that queries the DB for compromised contracts.
func SearchContract(w http.ResponseWriter, r *http.Request) {
var contractName contractID //Used for parsing in contractName
queryResult := getRow(&contractName.Name)
w.WriteHeader(200)
json.NewEncoder(w).Encode(queryResult)
}
//processRow queries the DB for a contract with ID value of name.
func getRow(contractName *string) *ContractVulnerabilityInfo {
var storage ContractVulnerabilityInfo //stores row to encode
//Login to database
...
scanErr := db.QueryRow("SELECT * FROM contracts WHERE ContractAddress=?;", &contractName).Scan(&storage.ContractAddress, &storage.IntegerOverflow, &storage.IntegerUnderflow, &storage.DOS, &storage.ExceptionState, &storage.ExternalCall, &storage.ExternalCallFixed, &storage.MultipleCalls, &storage.DelegateCall, &storage.PredictableEnv, &storage.TxOrigin, &storage.EtherWithdrawal, &storage.StateChange, &storage.UnprotectedSelfdestruct, &storage.UncheckedCall)
...
return &storage
}
My app.yaml file should allow me to deploy this flex app and does:
runtime: go1.12
env: flex
handlers:
- url: /.*
script: _server # my server.go file handles all endpoints
automatic_scaling:
max_num_instances: 1
resources:
cpu: 1
memory_gb: 0.5
disk_size_gb: 10
env_variables:
# user:password#unix(/cloudsql/INSTANCE_CONNECTION_NAME)/dbname
MYSQL_CONNECTION: root:root#unix(/cloudsql/haveibeenexploited:us-west1:hibe)/mythril
# https://cloud.google.com/sql/docs/mysql/connect-app-engine
beta_settings:
cloud_sql_instances: haveibeenexploited:us-west1:hibe
I am able to query the database successfully on localhost.Localhost correctly shows address
However, whenever I try to implement and push to AppEngine, when I query something that should be in the database, it does not show up in the remote App! App Engine does not show address in database. Furthermore, I get a status code of '0' returned, which indicates to me that the server function isn't even being called at all ('200' is what I expect if successful or some other error message.').
Summary
I can't wrap my head around this bug. What should work locally should work remotely. Also, I can't debug this app probably because Stackdriver does not support flex apps and the devserver Google Cloud provides does not support Go Apps.
I believe the primary issue is with Go not speaking to the React element correctly or the routing not being taken care of appropriately.
1) The problem does not lie with MySql connection/database access
- I changed my route to only be one page, turned off React, and included a hardcoded query. The result on localhost. The result on App Engine
2) There is an issue in either a) my routing or b) the interaction between React and Go.
3) Go seems to start correctly... at least when React is not started.
Any help is appreciated.
EDIT I believe that the go app indeed is still running, but the searchfunction is failing for whatever reason. The reason I believe this is because when I add another route for haveibeenexploited.com/hello, it works.

Which values can DocumentClientException.Error.Code have?

I want my data access layer to handle exceptions thrown by DocumentDB API provided via Microsoft.Azure.Documents.Client.DocumentClient class. For example, the optimistic concurrency check implemented using AccessCondition class, but others as well.
By looking at the exception thrown, the best way to recognize different DocumentClient-specific exceptions seems to be something like this:
try { ... }
catch (DocumentClientException exception)
when (exception.Error.Code == "Some magic here")
{
//let the user know how to recover from this..
}
I don't like such magic strings as they are not verifiable compile-time. It may contract a typo, or it may change on random moment with DocumentDB client/server changes, etc. Also, it is not clear which such magic codes I could/should be handling since I don't see the Microsoft.Azure.DocumentDB .net API containing any ErrorCodes enum or constants, nor find any list in documentation.
Where can I find a list of possible Error.Code values DocumentClient API can throw?
To make it even more confusing, the XmlDoc for DocumentClient.CreateDocumentAsync method suggest working instead on http status codes.
UPDATE: This question is not about Http status codes but DocumentClientException.Error.Code field as I assume the latter is more precise.
Where can I find a list of possible error codes values DocumentClient API can throw?
It's hard to find the completely list of error code that DocumentClinet API throw. The exception is depend on what your request.
For example, the optimistic concurrency check
Azure Cosmos DB uses ETags for handling optimistic concurrency.
When we retrieve a document from Azure Cosmos DB, it always contains an ETag property as apart of our document.
When we then want to send our request to replace a document, we can specify an AccessCondition with the ETag we received when we fetched out our document.
If the ETag we send is not current, the server will return a 412 Precondition Failed status code. In our .NET SDK, this is wrapped up in a DocumentClientException.
Here is an example that show the possible problems when the concurrency occurred.
By decompile the version 1.22.0 client, the code is set as a HttpStatusCode enum. I think all the possible values can be found here https://msdn.microsoft.com/en-us/library/system.net.httpstatuscode(v=vs.110).aspx then.
However, what really contains richer information for debug is the Error.Message. Might need to decompile the whole library to figure out, or wait for Microsoft to release the source codes, which is unlikely to happen since the latest update in github was 2 or 3 years ago.
public Error Error
{
get
{
if (this.error == null)
{
this.error = new Error()
{
Code = this.StatusCode.ToString(),
Message = this.Message
};
}
return this.error;
}
}
There is a list of the HTTP Status Codes for Azure Cosmos DB
I use the following code in my catch blocks
catch (DocumentClientException e)
{
var resp = new HttpResponseMessage
{
StatusCode = (HttpStatusCode) e.StatusCode,
Content = new StringContent(e.Message)
};
return resp;
}
Letting the user know how to handle the exception should be done on the client application.

How to fix cross-site origin policy for server and web-site

I'm using Dropwizard, which I'm hosting, along with a website, on the google cloud (GCE). This means that there are 2 locations currently active:
Some.IP.Address - UI
Some.IP.Address:8080 - Dropwizard server
When the UI tries to call anything from my dropwizard server, I get cross-site origin errors, which is understandable. However, this is posing a problem for me. How do I fix this? It would be great if I could somehow spoof the addresses so that I don't have to fully qualify the resource in the UI.
What I'm looking to do is this:
$.get('/provider/upload/display_information')
Or, if I have to fully qualify
$.get('http://Some.IP.Address:8080/provider/upload/display_information')
I tried setting Origin Filters in Dropwizard per this google groups thread (https://groups.google.com/forum/#!topic/dropwizard-user/ybDOTOxjlLI), but it doesn't seem to work.
In index.html that is served by the server at http://Some.IP.Address you might have a jQuery script that look as follows.
$.get('http://Some.IP.Address:8080/provider/upload/display_information', data, callback);
Of course your browser will not allow accessing http://Some.IP.Address:8080 due to the Same-Origin-Policy (SOP). The protocol (http, https) and the host as well as the port have to be the same.
To achieve Cross-Origin Resource Sharing (CORS) on Dropwizard, you have to add a CrossOriginFilter to the servlet environment. This filter will add some Access-Control-Headers to every response the server is sending. In the run method of your Dropwizard application write:
import org.eclipse.jetty.servlets.CrossOriginFilter;
public class SomeApplication extends Application<SomeConfiguration> {
#Override
public void run(TodoConfiguration config, Environment environment) throws Exception {
FilterRegistration.Dynamic filter = environment.servlets().addFilter("CORS", CrossOriginFilter.class);
filter.addMappingForUrlPatterns(EnumSet.allOf(DispatcherType.class), true, "/*");
filter.setInitParameter("allowedOrigins", "http://Some.IP.Address"); // allowed origins comma separated
filter.setInitParameter("allowedHeaders", "Content-Type,Authorization,X-Requested-With,Content-Length,Accept,Origin");
filter.setInitParameter("allowedMethods", "GET,PUT,POST,DELETE,OPTIONS");
filter.setInitParameter("preflightMaxAge", "5184000"); // 2 months
filter.setInitParameter("allowCredentials", "true");
// ...
}
// ...
}
This solution works for Dropwizard 0.7.0 and can be found on https://groups.google.com/d/msg/dropwizard-user/xl5dc_i8V24/gbspHyl4y5QJ.
This filter will add some Access-Control-Headers to every response. Have a look on http://www.eclipse.org/jetty/documentation/current/cross-origin-filter.html for a detailed description of the initialisation parameters of the CrossOriginFilter.

WebAPI and HTML5 SSE

was trying to encapsulate a partial view to show feedback that i can push back to the client.
This Article shows a method of pushing back data using HTML5 Server-Sent events (SSE).
I noticed that if i opened up several browser tabs and then closed one i got exceptions as the logic didn't remove the respective stream from the ConcurrentQueue. I amended the code as below
private static void TimerCallback(object state)
{
StreamWriter data;
Random randNum = new Random();
// foreach (var data in _streammessage)
for (int x = 0; x < _streammessage.Count; x++)
{
_streammessage.TryDequeue(out data);
data.WriteLine("data:" + randNum.Next(30, 100) + "\n");
try
{
data.Flush();
_streammessage.Enqueue(data);
}
catch (Exception ex)
{
// dont re-add the stream as an error ocurred presumable the client has lost connection
}
}
//To set timer with random interval
_timer.Value.Change(TimeSpan.FromMilliseconds(randNum.Next(1, 3) * 500), TimeSpan.FromMilliseconds(-1));
}
I also had to amend the OnStreamAvailable member as the framework syntax had changed to the second parameter being a HttpContent rather than HttpContentHeaders
public static void OnStreamAvailable(Stream stream, HttpContent headers, TransportContext context)
The problem now is i am still getting inconsistant behaviour if i add or remove clients i.e it times out when trying to initialise a new client. Does anyone have any ideas or more examples of using SSE with WinAPI and the correct "framework of methods" to handle disconnected clients
Cheers
Tim
This article is actually an adaptation of my original article from May - http://www.strathweb.com/2012/05/native-html5-push-notifications-with-asp-net-web-api-and-knockout-js/ (notice even variable names and port numbers are the same :-).
It is a very valid point that you are raising, and detecting a broken connection is something that's not very easy with this setup. The main reason is that while ASP.NET (the host) allows you to check a broken connection, there is no notification mechanism between ASP.NET (host) and Web API informing about that.
That is why in order to detect a broken connection (disconnected client) you should really try writing to the stream, and catch any error - this would mean the client has been disconnected.
I asked the same question to Brad Wilson/Marcin Dobosz/Damien Edwards at aspconf, and Damien suggested using HttpContext.Current.Response.IsClientConnected - so basically bypassing Web API and obtaining the connectivity info from the underlying host directly (however there is still a race condition involved anyway). That is really .NET 4. He also pointed an interesting way in which this problem could be avoided in .NET 4.5 using an async cancellation token. Frankly, I have never got around to test it, but perhaps this is something you should explore.
You can see their response to this problem in this video - http://channel9.msdn.com/Events/aspConf/aspConf/Ask-The-Experts - fast forward to 48:00