I'm currently creating pdf documents server side with wkhtmlpdf and nodejs. The client side sends the html to be rendered (which may include img tags with a source). When the user is previewing the html in the browser the images they uploaded to their account show fine because the user is authenticated via the browser and the node route can simply look up the image based on the user id (saved to session) and image id (passed in each image request).
The issue is when the images are attempting to be rendered in wkhtmltopdf webkit the renderer is not authenticated when it makes the request for images via node's exec of wkhtmltopdf in a separate process. A request to something like GET /user/images/<imageId> will fail due to the session not being set when the request is made inside the headless wkhtmltopdf renderer.
Is there a way to pass authentication via some wkhtmltopdf option or possibly a different way of authentication for images? The only restriction is not making images public.
I asked a similar question a while back that might help you:
Generate PDF Behind Authentication Wall
WKHTMLTOPDF has --cookie-jar which should get you what you need. Note that it didn't for me, and I wound up answering my own question with an alternate solution. In a nutshell, I wound up accessing the page via CURL - much more flexible - then writing a temporary that I converted to PDF, then deleted the temporary file.
A little round-a-bout, but it got the job done.
To implement authentication I allowed a cookie id flag ( with connect the key defaults to connect.sid ) as a query option in my image routes. The only "gotcha" is since images are requested from the server's perspective, you must ensure all your image paths are absolute domain paths rather than relative to your application ( unless those two are the same of course).
Steps for Expressjs:
Setup the id flag middleware which checks for say sid in the query via req.query (eg ?id=abc123 where abc123 is the req.cookies['connect.sid'], or req.signedCookies['connect.sid'] if your using a secret as you probably should )You may need to ensure the query middleware is setup first.
Ensure the req.headers contains this session id key and value prior to the cookie parser so the session is properly setup (eg if a cookie exists append a new one or if one does add it as the first req.headers.cookie = 'connect.sid=abc123;')
Ensure all image paths contain the full url (eg https://www.yourdomain.com/images/imageId?id=abc123)
Some extra tid bits: The image source replacement should probably happen at the server level to ensure the user does not copy/paste the image url with the session id and say email it to a friend which obviously leaves the door open for account hijacking.
Related
The first answer in this question: Session is lost and created as new in every servlet request is valid, however users in an intranet are very familiar no to type the whole domain in the URL. It will be very useful to configure the browser so they accept the URL without the domain and still keep the session.
According to this article: Cookie attributes- Domain and Path, you can know that the Domain attribute defines the scope of the cookie. It lets the browser know which website the cookie belongs to. If the server does not specify the domain of the cookie, it will default to the domain of the requested resource.
Therefore, I think your requirement cannot be implemented.
The decision was not to the browser, but to the Web.config file in the ASP.NET application. For a different issue we had to add the text cookieSameSite="None" to the SessionState tab in Web.config file on May 2020. Time came and apparently a new update of the browsers caused the application lost the session because it did not create the cookie with SessionID as ASP.NET need, unless we specify the whole domain in the URL.
That said we have 2 solutions: Remove the cookieSameSite="None" from Web.Config file or Type the whole domain in the address of the application.
From the 2 solutions the first one was not possible in our case, because the ASP.NET app is inside an iframe tab and without cookieSameSite="None" it does not keep the session, so we included the whole domain name inside the html page using iframe as src="myServer.MyDomain/MyApp/"
I'm workin' on a web project where performance is a very important issue.
EDIT:
The situation:
I wanna add some details about the user's workflow:
The user visits the welcome page of my website http://example.org/ .
He clicks a link in order to visit the page http://example.org/mypage
onclick-Handler of the link's executed.
The handler loads data usin' XHR.
The handler creates http://example.org/mypage dynamically.
The handler saves mypage locally usin' FileSystem API at filesystem:http://example.org/mypage. EDIT: ( filesystem:http://example.org/mypage is a local ressource stored in the FileSystem at the client side)
The handler extends the history and changes the URL of the location bar usin' History API from http://example.org/ (URL of the welcome page) to http://example.org/mypage (the page which the user wants to see) .
The user vists another page in the meantime.
Later on, the user types http://example.org/mypage directly into the location bar.
The browser shows/loads filesystem:http://example.org/mypage (which is the locally stored version of http://example.org/mypage) instead of http://example.org/mypage. That means: The browser doesn't create a new request, it uses the local stored copy of http://example.org/mypage .
How can I get the browser to use the locally stored version of the page instead of creating a new request? EDIT: - That's what I want to do in #10 of the list above.
EDIT:
My Question:
A client-side has already created/generated http://example.org/mypage in #2 to #7 of the list above. I don't need to create that page some other time. That's why I don't want the browser to create a request for http://example.org/mypage.
That's what I wanna do:
If filesystem:http://example.org/mypage has already been created (respectively if the user has already visited http://example.org/mypage):
Use filesystem:http://example.org/mypage instead of http://example.org/mypage.
Otherwise:
Send a request for http://example.org/mypage
Tries to solve:
I can't use the Fallback section of the manifest file to do something like: EDIT: (aside from the orgin)
FALLBACK:
http://example.org/mypage filesystem:http://example.org/mypage
In order to get the browser to use the local version stored in the FileSystem because Fallback directives are just used if the user is offline, otherwise they are ignored. EDIT: But I want to use filesystem:http://example.org/mypage instead of http://example.org/mypage, even if the user's online.
I know that I can use the Expire field in the response header of a server-generated page in order to not create a new request and to use the cached version.
But what if I create an page dynamically on the client side using JS and XHRs. EDIT: (I described that case in The situation) When create a page at the client side there's no way to get the client to cache that page. That's why I "cache" the page manually usin' FileSystem API to store it on the client side.
In order to improve the performance I'm trying to store any page which the user has already visited locally. When the user visits a page again then I show him the old, locally stored version of the page and my script creates an XHR to find out if the page changed in the meantime.
But how can I get the browser to use the local version of the page?
I can save the generated page locally on the client side using the FileSystem API and I can choose an URL for the generated page to display it at the browser's location bar using the History API.
When the user now visits another site and then presses the back button I can catch the onPopState event by an event handler.
And that event handler can load the dynamically created file using the FileSystem API.
But what should I do if the user doesn't use the back button and if he types the URL, which I have registered using the History API, directly into the location bar?
Then the browser wouldn't use the locally stored version of the page, the browser would create a request to load the page from the server.
Don't put dynamic data in the application cache. If you want to put dynamic data in your pages then get it from the server with AJAX, store the data in Local Storage, and populate the page with the data from storage through JavaScript (you can hook into the History API for this).
By the way, this won't work because fallback entries have to be on the same domain:
FALLBACK:
http://example.org/mypage filesystem:http://example.org/mypage
Once your page is in the Application Cache (ie. it is locally stored) the browser will always use the version from the Application Cache until the manifest is updated or the user deletes the cache. It doesn't really matter what expiry headers you put on the page, except if you put a long expiry and you frequently update the manifest then it's likely the Application Cache will be populated from the browser cache rather than refreshed from the server. This is why the stuff you put in the Application Cache should be static files. Get your dynamic stuff with AJAX.
You might use URLs that encode the actual link within your hierarchy, e.g. "mypage", in the anchor part of the URL, i.e. http://example.com/#mypage. Then you can use window.location.hash to obtain the string after the # and do whatever magic you want. Just make sure your root (or whatever you want in front of the #) is in AppCache.
I was wondering how to keep images secure on my website. We have a site that requires login then then user can view thousands of different images all named after their ID in the database.
Even though you need to login to view the images the proper way...nothing is stopping a user from browsing through the images by typing <website-director>/image-folder/11232.jpg or something.
this is not the end of the world but definitely not ideal. I see that to stop this facebook just names the images something much more complicated + stores them in hashed folders.
Gmail does a very interesting thing, their image tags looks like this:
<img src=/mail/?attid=0.1&disp=emb&view=att&th=12d7d49120a940e5>
I thought the src attribute has to contain a reference to an image??...how does gmail get around this?
This is more for educational purposes at this point, as I think this gmail scheme might be overkill for our implementation.
Thanks for your feedback in advance,
Andrew
I thought the src attribute has to contain a reference to an image?
GMail is referencing an image. It's just being pulled dynamically, probably based off of that th=12d7d49120a940e5 string.
Try browsing to http://mail.google.com/mail/?attid=0.1&disp=emb&view=att&th=12d7d49120a940e5
Instead of it being a direct path to its location on the server's filesystem, it uses a dynamic script (the images may even be in a database, who knows).
Besides serving up an image dynamically from your webapp, it's also possible to use a webapp to dynamically authorize access to static resources that the webserver will serve -- commonly by putting the files somewhere that the webserver has access to, but not mapped to any public URI, and then using something like X-Sendfile (lighttpd, Apache with mod_sendfile, others), X-Accel-Redirect (nginx), X-Reproxy-File (Perlbal), etc. etc. Or with FastCGI you can configure an application in a FastCGI "authorizer" role rather than a content provider.
Any of these will let you check the image being authorized, and the user's session, and make whatever decision you need to, without tying up a proceses of your backend application for the entire time that the image is being sent to the client. It's not universally true, but usually a connection to the backend app represents a lot more resources being reserved than a connection to the webserver, so freeing them up ASAP is smart.
The code that runs after this GET request is issued:
/mail/?attid=0.1&disp=emb&view=att&th=12d7d49120a940e5
outputs an image to the browser. Something doesn't have to be named with a .jpg or .png or whatever ending to be considered an image by a browser. This is how captcha algorithms are able to serve up different images depending on a value in the id. For example, this link:
http://www.google.com/recaptcha/api/image?c=03AHJ_VusfT0XgPXYUae-4RQX2qJ98iyf_N-LjX3sAwm2tv1cxWGe8pkNqGghQKBbRjM9wQpI1lFM-gJnK0Q8G3Nirwkec-nY8Jqtl9rwEvVZ2EoPlwZrmjkHT7SM32cCE8PLYXWMpEOZr5Uo6cIXz1mWFsz5Qad1iwA
Serves up this image:
So the answer really is to just obfuscate your image names/links a bit like Facebook does so that people can't easily guess them.
I'm thinking about this question for a long time. It is a big question, since it almost covers all corners related to web developing.
In my understanding, the process should be like:
enter the url to the address bar
a request will be sent to the DNS server based on your network configuration
DNS will route you to the real IP of the domain name
a request(with complete Http header) will be sent to the server(with 3's IP to identify)'s 80 port(suppose we don't specify another port)
server will search the listening ports and forward the request to the app which is listening to 80 port(let's say nginx here) or to another server(then 3's server will be like a load balancer)
nginx will try to match the url to its configuration and serve as an static page directly, or invoke the corresponding script intepreter(e.g PHP/Python) or other app to get the dynamic content(with DB query, or other logics)
a html will be sent back to browser with a complete Http response header
browser will parse the DOM of html using its parser
external resources(JS/CSS/images/flash/videos..) will be requested in sequence(or not?)
for JS, it will be executed by JS engine
for CSS, it will be rendered by CSS engine and HTML's display will be adjusted based on the CSS(also in sequence or not?)
if there's an iframe in the DOM, then a separate same process will be executed from step 1-12
The above is my understanding, but I don't know whether it's correct or not? How much precise? Did I miss something?
If it's correct(or almost correct), I hope:
Make the step's description more precise in your words, or write your steps if there is a big change
Make a deep explanation for each step which you are most familiar with.
One answer per step. Others can make supplement in each answer's comment.
And I hope this thread can help all web developers to have a better understanding about what we do everyday.
And I will update this question based on the answers.
Thanks.
As you say this is a broad question where it's possible to go into great detail on a number of topics. There's nothing wrong with the sequence you described, but you're leaving out a lot of detail. To mention a few:
The DNS layer can help direct clients to different servers based on geographical location to help with load balancing and latency minimization, and one server can respond to requests from many different DNS names.
A browser can make different types of requests (GET, POST, HEAD, etc), and usually includes several different headers including cookies, browser capabilities, language preferences, etc.
Most browsers usually maintain a cache in order to avoid downloading stuff many times, and use various techniques to determine whether the cached version of a file is valid.
In modern webpages there's often complex interaction between many different kinds of files (HTML, CSS, images, JavaScript, video, Flash, ...), and web developers often need detailed knowledge of differences among browsers in order to keep their pages working for everyone
Each of these topics, and many more, could be discussed at length. Perhaps it's more practical to ask more specific questions about the topics you're interested in?
You type maps.google.com(Uniform Resource Locator) into the address bar of your browser and press enter.
Every URL has a unique IP address associated with it. The mapping is stored in Name Servers and this procedure is called Domain Name System.
The browser checks its cache to find the IP Address for the URL.
If it doesn't find it, it checks its OS to find the IP address (gethostname);
It then Checks the router's cache.
It then checks the ISP's cache. If it is not available there the ISP makes a recursive request to different name servers.
It Checks the com name server (we have many name servers such as 'in', 'mil', 'us' etc) and it will redirect to google.com
google.com name server will find the matching IP address for maps.google.com in its’ DNS records and return it to your DNS recursor which will send it back to your browser.
Browser initiates a TCP connection with the server.It uses a three way handshake
Client machine sends a SYN packet to the server over the internet asking if it is open for new connections.
If the server has open ports that can accept and initiate new connections, it’ll respond with an ACKnowledgment of the SYN packet using a SYN/ACK packet.
The client will receive the SYN/ACK packet from the server and will acknowledge it by sending an ACK packet.
Then a TCP connection is established for data transmission!
The browser will send a GET request asking for maps.google.com web page. If you’re entering credentials or submitting a form this could be a POST request.
The server sends the response.
Once the server supplies the resources (HTML, CSS, JS, images, etc.) to the browser it undergoes the below process:
Parsing - HTML, CSS, JS
Rendering - Construct DOM Tree → Render Tree → Layout of Render Tree → Painting the render tree
The rendering engine starts getting the contents of the requested document from the networking layer. This will usually be done in 8kB chunks.
A DOM tree is built out of the broken response.
New requests are made to the server for each new resource that is found in the HTML source (typically images, style sheets, and JavaScript files).
At this stage the browser marks the document as interactive and starts parsing scripts that are in "deferred" mode: those that should be executed after the document is parsed. The document state is set to "complete" and a "load" event is fired.
Each CSS file is parsed into a StyleSheet object, where each object contains CSS rules with selectors and objects corresponding CSS grammar. The tree built is called CSSCOM.
On top of DOM and CSSOM, a rendering tree is created, which is a set of objects to be rendered. Each of the rendering objects contains its corresponding DOM object (or a text block) plus the calculated styles. In other words, the render tree describes the visual representation of a DOM.
After the construction of the render tree it goes through a "layout" process. This means giving each node the exact coordinates where it should appear on the screen.
The next stage is painting–the render tree will be traversed and each node will be painted using the UI backend layer.
Repaint: When changing element styles which don't affect the element's position on a page (such as background-color, border-color, visibility), the browser just repaints the element again with the new styles applied (that means a "repaint" or "restyle" is happening).
Reflow: When the changes affect document contents or structure, or element position, a reflow (or relayout) happens.
i was also searching for the same thing and found this awesome detailed answer being built collaboratively at github
I can describe one point here -
Determining which file/resource to execute, which language interpreter to load.
Pardon me if I am wrong in using interpreter here. There may be other mistakes in my answer, I will try to correct them later and include proper technical terms for things.
When the web server (e.g. apache) has received the URI it checks if there is any existing rewrite rule matching it. In that case the rewritten URI is taken. In either case, if there is no file name to end the URI, the default file is loaded, which is generally index.html or index.php etc. According to the extension of the file name, the appropriate apache module for server-side programming language support is loaded, e.g. mod_php for PHP, mod_python in case of python. The appropriate server side language interpreter (considering interpreted languages like PHP) then prepares the final HTML or output in some other form for the web server which finally sends it as the HTTP response.
I hope above image help you to understand whole process.
Full article is here
It's an important security issue and I'm sure this should be possible.
A simple example:
You run a community portal. Users are registered and upload their pictures.
Your application gives security rules whenever a picture is allowed to be displayed. For example users must be friends on each sides by the system, in order that you can view someone else's uploaded pictures.
Here comes the problem: it is possible that someone crawls the image directories of your server. But you want to protect your users from such attacks.
If it's possible to put the binary data of an image directly into the HTML markup, you can restrict the user access of your image dirs to the user and group your web application runs of and pass the image data to your Apache user and group directly in the HTML.
The only possible weakness then is the password of the user that your web app runs as.
Is there already a possibility?
There are other (better) ways, described in other answers, to secure your files, but yes it is possible to embed the image in your html.
Use the <img> tag this way:
<img src="data:image/gif;base64,xxxxxxxxxxxxx...">
Where the xxxxx... part is a base64 encoding of gif image data.
If I needed security on my images directory I wouldn't expose the directory at all. Instead my img src attributes would reference a page that would take a userid and an image id as a parameter.
The page would validate that that user did indeed have access to see that picture. If everythings good, send the binary back. Otherwise send nothing.
for example:
<img src="imgaccess.php?userid=1111&imgid=223423" />
Also, I wouldn't use guessable id's. Instead sticking to something like base 64 encoded guid's.
I'm not sure I understand, but here goes. Instead of serving up static images that reside in an images folder - why couldn't you, using your server side technology of choice, have the images dynamically sent down to the client? That way your server side code can get in the mix and allow or deny access programmatically?
<img src="/images/getImage.aspx?id=123353 />
You could move the pictures out of the document root into a private directory and deliver them through your application, which has access to that directory. Each time your app generates an image tag, it then also generates a short-lived security token which must be specified when accessing a particular image:
<img src="/app/getImage.xyz?image=12345&token=12a342e32b321" />
Chances are very rare that someone will brute force the right token at the right time with the right image.
There are at least to possibilities to verify the token in "getImage":
Track all image tags in your app and store records in a database which link the randomly generated tokens and image IDs to the requesting users. The "getImage" action then checks the supplied parameters against that database.
Generate the token as a checksum (MD5, CRC, whatever) over the user ID, the image ID and maybe the current day of the year, and be sure to mix in an unguessable salt. The "getImage" action will then recompute the checksum und check it against the specified one in order to verify the user's access. This method will produce less overhead than the first one.
PHP example:
$token = md5($_SESSION['user_id'].' '.$imageID.' '.$SECRET_SALT.' '.date('z'));
With HTML5 you could use the canvas tag and JavaScript to do this.
You could perhaps do something with either CSS or a table layout to draw a picture (probably really bad performance, resolution, portability).
Either way, there is no stopping people from taking your pics. They could take a screenshot and crop it out.
As Chris mentioned in his answer, having long picture id's so that the URL for each image is not easy to guess or brute force is important. And no directory listing on your webserver directories is also.
https://www.base64-image.de/
I used this website to generate base64 code fir given image, and then this website provide code to directly paste . It worked.