I am about to implement a HTML based Log file viewer. The update volume varies from 1-10 updates per second
The server is WebSocket based and will be developed by me as well - I have built a Fleck based prototype and this side looks fine.
Is there any other smart HTML field besides a plain text field which
I could use for updating?
Would you recommend me to collect updates and work with a fixed
update interval?
I guess it would be more efficient to add the update interval in the server then, right?I am new to Java Script and HTML 5, so please do not be too harsh if these questions are trivial.
I am about to build a similar application and I therefore played around a little bit, comparing the performance of 1.) attaching DOM elements for every log row, 2.) attaching a table row for every log row, and 3.) using a textarea tag:
http://jsfiddle.net/PBzg5/18/
While removing all rows from the viewer is fastest with the textarea it takes longest to fill it. Also, there seems to be no faster method than manual string concatenation for textarea. Attaching elements to DOM (i.e. one text element and one <br> element per log row) is definitely fastest, with the table-based version being close behind. Also, using DOM elements will allow you to do more advanced things like coloring individual words than when using textareas. However, I haven't tested the performance influence of this yet.
When you implement your viewer be sure to keep in mind that browsers will actually brake down pretty fast when you try to display an unlimited number of rows. Therefore, just keep a certain number of the newest rows in a buffer (like terminals usually do it) and only display them.
Related
I have a couple of questions about images, since I don't know what is better for my purposes. Also this might me helpful for other people because I couldn't find this info in other questions.
Well, although this is an asp.net core 2.0 application the first question could is a general question about images.
QUESTION 1
When I have images that I want to load everytime I usually add a query string so the explorers like Chrome or IE don't get the chached image they have. In my case I add the time ticks to the url of the image, this way it loads the image everytime since the query string is always different:
filePath += "?" + DateTime.Now.Ticks;
But in my case I have a panel where the administrators of the page can change a lot of images. The problem, when they change those images if there is no query string the users are going to see an old image they have stored in their explorer cache.
The question is, if I add the query string to many images is not bad for the performance? is there any other solution for this?
QUESTION 2
I also have photos of the users and other images stored in the site. When I saw a image all the visitors of the site can see the path (for example: www.site.com/user_files/user_001/photo001.jpg).
Is there a way to hide those paths or transform in another thing is asp.net core 2.0?
Thanks a lot.
Using something like ticks will get the job done, but in a very naive way. You're going to put more stress both on your server and the clients, since effectively the image will have to be refetched every single time, regardless of whether it has changed or not. If you will have any mobile users, the situation is far worse for them, as they'll be forced to redownload all these resources over and over, usually over limited (and costly) data plans.
A far better approach is to use a cryptographic digest, often called a "hash". Essentially, the same data encrypted in the same way will return the same hash. It's usually used to detect tampering with transmitted data, but since each message will (generally) have a unique hash and that hash will be the same each time for the same piece of data, you can also use this to generate a cache-busting query string that only changes when the image data itself changes.
Now, to be thorough, there's technically no guarantee that two messages won't result in the same hash. Instances where that occurs are called "collisions" and they can happen. However, if you use a sufficiently complex algorithm like SHA256, the likelihood of collisions is greatly reduced. Regardless, it should not be a real issue for concern for this particular use case of cache-busting images.
Simplistically, to create the hash, you simply do something like:
string hash;
using (var sha256 = SHA256.Create())
{
hash = Convert.ToBase64String(sha256.ComputeHash(imageBytes));
}
The value of hash then will be something like z1JZs/EwmDGW97RuXtRDjlt277kH+11EEBHtkbVsUhE=.
However, ASP.NET Core has an ImageTagHelper built-in that will handle this for you. Essentially, you just need to do:
<img src="/path/to/image.jpg" asp-append-version="true" />
As for your second question, about hiding or obfuscating the image path, that's not strictly possible, but can be worked around. The URL you use to reference the image uniquely identifies that resource. If you change it in any way, it's effectively not the same resource any more, and thus, would not locate the actual image you wanted to display. So, in a strict sense, no, you cannot change the URL. However, you can proxy the request through a different URL, effectively obfuscating the URL for the original image.
Simply, you'd just have an action on some controller that takes an image path (as part of the query string), loads that from the filesystem and returns it as a response. Care should be taken limit the scope of files that can be returned like this, both based on directory (only allow your image directory, for example, not C:\Windows\, etc.) and file type (only allow images to be returned, not random text files, config files, etc.). That portion is straight-forward enough, and you can find many examples online if you need them.
Ultimately, this doesn't really solve anything, though, because now your image path is simply in the query string instead. However, now that you've set this part up, you can encrypt that part of the query string using the Data Protection API. There's some basic getting started information available in the docs. Essentially, you're just going to encrypt the image path when creating the URL, and then in your action that returns the image, you decrypt the path first before running the rest of the code. For the encryption part, you can create a tag helper to do this for you without having to have a ton of logic in your views.
I have an optimization question.
Lets say that I'm making a website, and it has a JSON file with 5,000 pairs (about 582 kb) and through the combination of 3 sliders and some select tags it is possible to display every value. So the time to appear between every pair is in microseconds.
My question is: If the website is also made to run on mobile browsers, where is it more efficient to have the 5000 pairs of data - in a JSON file or in the data base? and why?
I am building a photo site with similar requirements and I can say after months of investigations and experimenting that there are no easy answer to that question. But I will try to give you some hints:
Try to divide the data in chunks, for example - if your sliders are selecting values between 1 through 100, instead of delivering exactly what the client selected, round up a bit maybe +-10 or maybe more, that way you can continue filtering on the client side without a server roundtrip. Save all data in client memory before querying.
Don't render more than what is visible on the screen, JSON storage and filtering is fast but DOM is very slow, minimize the visible elements.
Use 304 caching - meaning - whenever the client is requesting the same data twice; send a proper 304 response with etag. For example - a good rule of thumb here is to use something you know very easily, like the max ID in the database or so to see if any new data has been updated since the last call. If not, just send 304 and the client will use whatever he had the last time.
Use absolute positioning. Don't even try to use the CSS float or something like that, it will not work. Just calculate each position of each element. This will also help you to achieve tip nr 2 (by filtering out all elements that are outside of the visible screen). You can still use CSS transitions which gives nice animations when they change sliders.
You could experiment with IndexedDB to help with the client side querying but unfortunately the support in different browsers are still not good enough plus you hit the roof on storage, better to use the ordinary cache and with proper headings.
Good Luck!
A database like MongoDB would be good for this. It still uses the JSON syntax for storage so you can use the values from the JSON file. The querying is very fast too and you wouldn't have to parse the JSON file and store it in an object before using it.
Given the size of the data (just 582Kb) I will opt for the Json file.
The drawback is you will have a penalty starting the app and loading the data in memory, but then all queries will run very fast in memory as a good advantage.
You need to think about how much acceses will your app do to the database (how many queries) against load the file just once. And think if your main objective are mobile browsers or pcs.
For this volume of data I wouldn't try a database (another process consuming resources), just try how much resources (time, memory) are needed to load the JSON file.
If the data is going to grow... then you will need to rethink this, or maybe split your json file following some criteria.
I know I can check the response header's 'last-modified' value to determine when the web page was last modified, but in many instances that header is NOT provided. Also, in many instances the content itself hasn't changed, but the current time/date is displayed on the page, thus giving the appearance of a modification.
Any suggestions on how to overcome the above issues and determine if a web page has been (truly) modified?
Thanks.
Sure. Define for yourself what counts as a "modification" (for example, only things in the "content" div) and only look at that.
If you can't find a way to decide whether something's been changed, then you can't expect a computer to…
You are asking two question here:
When was it modified?
Was it modified?
To answer question #1, you'd have to check the page every so often to meet your granularity requirements e.g. every hour, every day, every week, etc. This could be quite resource intensive. This will depend on if you really need to know this.
To answer question #2, you need to compare something. You could do what #Paul Rosnia suggested, but if they as much as added a comma, it will be considered modified.
Then, you might also want to see what has been modifed. Then you you'd have to save the content and compare them to each other in order to highlight the changes.
You could use http://php.net/manual/en/function.file-get-contents.php and a CRON job to cache the page on your server and then perdiodically compare your cache. The comparing part will be the tricky part, since you have to write specific code to ignore the things that don't matter to you e.g. date/time stamp, header changes, menu changes, etc.
The sure-fire way to detect page changes is to download and checksum it. If the checksum changes, the page has been edited (with extremely high certainty).
Here's an example that works on the command line:
curl -s news.ycombinator.com | md5 #=> d86582bec138c051b0d8322f7823a23c
That was a few minutes ago. If you run it now you'll get a different answer!
i can't find any information inside the w3c html strict spec
http://www.w3.org/TR/html4/sgml/dtd.html
PHP USERS:
If you are using php to process your form take note that php has a max_input_vars setting in the php ini file. I believe the default is 1000.
I ran into this while building a cakephp app. To see if this is affecting your post, count how many rows of data you are putting into the form and then count how many rows are actually posted back.
So essentially your form would be limited by the number of input vars you have.
This isn't really an direct answer to the OP but I think it would be helpful for anyone searching for a limit on input fields.
To my knowledge, there is no upper limit to the number of form elements (<input> or otherwise) in an HTML document.
However, if you have a huge number of form elements on your page, you might end up with a POST request that is too large to be processed by the web server (the maximum size of POST requests depends on the server configuration).
I don’t think there’s a maximum number of input elements in a form given by the spec, if that’s what you are asking. If you have many inputs and want to make sure the form works, you’ll have to try on the clients you support. And of course, it would be much better to redesign the form, if that’s possible.
I don't think that there is a limitation on the number of unput fields in the standards.
There are two practical limitations that you need to consider:
Some browsers start to act up if there are too many input fields on a page. I haven't tried this with recent versions, but I remember testing this a few years back, and then I found that Internet Explorer behaved badly when the number of fields was closing to a hundred.
Too many input fields on a page is just inconvenient, and perhaps a bit scary, to the user. Split the input on several pages, or show placeholders and add input fields dynamically only where they are actually used.
Technically, there is no maximum number of input fields that can be put on a page. Practically, for a user, it is a bit inconvenient for a user to use/see all of the input fields if a web page has a large number of input fields.
I was encountering a similar problem in an old application.
I thought it was a limit to the number of HTML input elements or PHP ini configuration for max_input_vars.
But after more investigation, it turns out it was Suhosin that was installed on my production machine which over wrote the PHP.ini config value with a much lower limit.
suhosin.post.max_vars = 400
Please note that we cannot set this max_input_vars directive in run-time with function ini_set. We can change it directly in php.ini file.
; How many GET/POST/COOKIE input variables may be accepted
max_input_vars = 2000
Don't forget to restart web server. Sometimes we need to reboot the server to take effect.
For a major school project I am implementing a real-time collaborative editor. For a little background, basically what this means is that two(or more) users can type into a document at the same time, and their changes are automatically propagated to one another (similar to Etherpad).
Now my problem is as follows:
I want to be able to detect what changes a user carried out onto an HTML textfield. They could:
Insert a character
Delete a character
Paste a string of characters
Cut a string of characters
I want to be able to detect which of these changes happened and then notify other clients similar to "insert character 'c' at position 2" etc.
Anyway I was hoping to get some advice on how I would go about implementing the detection of these changes?
My first attempt was to consider the carot position before and after a change occurred, but this failed miserably.
For my second attempt I was thinking about doing a diff on the entire contents of the textfields old and new value. Am I missing anything obvious with this solution? Is there something simpler?
It is a really hard work make this working today, for several reasons, but
maybe you will need to restrict only to some browsers. read: https://developer.mozilla.org/en/XUL/Attribute/oninput the alternative to "oninput" is listen to all input events (keyboard, mouse, dragdrop) I suggest to use "oninput"
html is not perfect... even html5. input and textareas supports only single-range
selections. you can solve this using designmode/contenteditable instead of
textareas/textfield
detecting offsets of what changed is a hard work: read
-- https://developer.mozilla.org/en/Document_Object_Model_%28DOM%29/window.getSelection
-- http://www.quirksmode.org/dom/range_intro.html -- http://msdn.microsoft.com/en-us/library/ms535869%28v=VS.85%29.aspx -- http://msdn.microsoft.com/en-us/library/ms535872%28v=VS.85%29.aspx
you may need a "diff" algorithm written in javascript! http://ejohn.org/projects/javascript-diff-algorithm/
one personal note: detecting words, characters changes may be totally non-sense and not useful, detect instead paragraphs changes, or in case of an excel-like worksheet, the single cell
I hope this helps
feel free to correct my English!
My pseudocode/written out response would be (if I understand your question exactly) to use jQuery to detect keyup events and then save the input to the server via ajax, then also take the response and post it back to the input. This isn't very efficient, but basically the idea is that you're constantly posting and checking what else has been posted. If you want to see what someone else is doing in real time, you can ping the server every second or so and update with the response.
All of this of course can be optimized, but it still is kind of taxing for a server. You could also see if you can implement Google Topeka Wave for your project, or get in touch with Google Topeka to see how they do it :)