Given an HTML page I would like to get all the 'x' files that are embedded in the HTML file or are linked by it, where 'x' equals:
Images (JPG,PNG,GIF...)
Documents (Word, PowerPoint, PDF...)
Flash (.flv, .swf)
How do I do this?
So images are easy to extract because they are either linked to with a link ending in a (.png|.jpg|....) or they are embedded with an img tag.
Documents can not be embedded, they can only be linked to (with a link ending in a .doc|.ppt|.pdf|...). So they are also easy to get.
Here is my problem:
How do I get the flash files that are embedded in webpages?
Please give me a pseudo-algorithm or a regex pattern.
If I am wrong in my points above (1. and 2.) please tell me so too.
Thanks!
The Firefox extension DownThemAll lets you right-click a page and download all of the media of a specified extension. It's open source, so you might want to look at their code and see how they implemented it.
I'd use an event based XML parser (like SAX) and write the rules for the and tags to get the src and href attributes.
Related
I am trying to save a customized html file as a pdf.. normally I would press ctrl-P at my browser (chrome) and print as pdf..
But when I open the pdf file, there is no bookmark tab on the left side of the pdf reader (adobe)..
What I want is to save an html file as a pdf and the bookmark should appear in the left side of the pdf reader:
I created the html file.. I added links to some parts of it using id and hyperlink:
part1
...some codes here...
<div id="part1">
and it works, but I don't know how to create a bookmark in pdf from an html... normally ms word or libre office can convert their documents to pdf with a bookmark..
But how can I made a pdf with a bookmark using HTML?
Okay, so I ran into this problem and really wanted there to be a solution here that worked. When there wasn't, I figured I should add what I found so that hopefully the next developer can benefit from it.
First up: HTML conversion to PDF isn't really up to the HTML itself - it's up to whatever the conversion engine decides to do with your HTML. So for instance, if your approach is: Open it in IE/Chrome/Firefox/whatever > File > Print > Microsoft Print to PDF - well, your conversion engine is 'Microsoft Print to PDF'. Doesn't matter what browser you were using at that point - all its doing is creating a print stream to send to a printer. So if Microsoft Print to PDF isn't going to make bookmarks for you (which it doesn't) then it doesn't matter which web browser you use to open the PDF.
And this is the critical problem with any Ctrl-P / Print avenues. The web browser is ultimately creating a print stream, which the conversion library simply streams into a PDF. And all the web browsers I looked at do not have native support built in to convert to PDF (why would they? 99% of the use cases are covered with a 'Print to PDF' functionality.) And the print drivers I tried (Microsoft Print to PDF, Adobe PDF Print) didn't manage to suss out bookmarks from the raw print stream. Which makes sense.
So, at this point, what you're looking for is a standalone PDF Conversion engine - something that can actively open the HTML file and convert from there, instead of going through a web browser. Are there PDF Conversion engines that do this and add Header-Tag based bookmarks? Possibly. The ones we had at our disposal (ABCPdf, Neevia) weren't able to do it, but it's certainly possible there's one out there.
So what now?
There are a few different options I explored.
Option #1: Separate Files, Combined With Adobe
Adobe Acrobat (non-viewer version), when it's the conversion engine, will automatically add bookmarks for each file it converts. So you can submit the HTML contents, not as a single HTML file, but as HTML files for each section you want a bookmark over.
The good news is that if a section has a hyperlink that points to another document its merging, it's smart enough to have that hyperlink point to the spot within the internal PDF its creating (it's not an external hyperlink like I expected it would be). There are two bits of bad news, though:
Each section has to be the start of a PDF page. If your section is
two inches tall, the rest of the page will be blank, and the next
section will start on the following page.
The bookmarks aren't clean. When I did it, each file had 3
bookmarks. Which is pretty darned ugly and off-putting.
Option #2: Separate Files, Combined With Another Library
The first 'downside' of Option #1 might not be a problem. But the second is pretty ugly. And other libraries definitely can create the bookmarks without creating 3-per-file. The main obstacle here is: the library has to be smart enough to resolve those 'external' hyperlinks to within the PDF that's created. One thing that often hurts is that those conversion libraries often want to convert each separate file to a PDF internally first and then merge the PDFs together... but that means that it won't handle the cross-file hyperlinks correctly. I wasn't able to find a way to make this work with our existing PDF conversion libraries.
Option #3: Different Origination Method
Instead of having a 'Help.html', which is then converted to PDF somehow, start with a format other than HTML. And the easiest source to get into PDF+Bookmarks is MSWord+Headers. Generally, for each PDF help file you want, you can have a master .DOCX sitting somewhere behind the scenes. We've used this approach before, and while it's not the most elegant, it at least works pretty well.
Option #4: Programmatic with Library
This might not be applicable for the OP's use case... but if you're generating the help, there's nothing to say you can't use the PDF Conversion library programatically to add whatever bookmarks you want. Pretty much every PDF engine I've seen allows API access to bookmarks, so if this avenue is open to you, it's almost certainly the cleanest solution-wise.
Option #5: PDF Conversion Scouring
Like I mentioned, it's possible there's a PDF conversion engine out there that has a good HTML parsing engine and can handle bookmarks from various HTML tags (like H1, H2, etc.) However, it's probably going to take a bit to find it, because it's so much easier for a potential engine-writer to allow the file to be rendered with a native viewer. Think about it. If you were writing a PDF Conversion Service, which would you rather do:
Develop routines that can accurately render an HTML document fed
into it - aka, basically write your own web browser from scratch.
Have IE/Chrome/Whatever render it and simply take their print output
to convert to PDF.
... that second option is so ridiculously easier than the first, that it's no surprise most PDF Conversion engines don't have their own internal HTML parser (or for that matter, Word parser, Excel parser, etc.)
The bookmarks in html input document are set like this:
....
...
...
...
<h1 id="marcador1"> Chapter 1 </h1>
...
Don't use chrome, although it is simple to convert a web page to a PDF file. If you want pdf bookmarks, you can try microsoft word (2010). Just save the web pages to local, and open it with MS word 2010, then save it as pdf. The bookmark is there. see also: https://www.w3.org/TR/WCAG20-TECHS/PDF2.html
App comparison for converting PDF (regarding bookmark & internal hyperlink)
I did some tests for different app, (results may not be accurate due to personal settings / mis-used)
pdf bookmark
internal hyperlink
downloaded as .htm
file format looking
Chrome (print as PDF)
N
Y
N
looks same as the webpage
Calibre
Y
N/Y
Y
looks same as the webpage
Print Friendly & PDF 2.8.1 (Chrome Extension)
N
Y
N
syntax color is changed
WPS docx
N/Y
N
Y
format is changed a lot
Foxit PDF
N
N
Y
looks same as the webpage
Adobe PDF
N
N
Y
looks same as the webpage
MS Word docx
Adobe PDF (Chrome Extension)
annotation:
pdf bookmark = contains bookmark in PDF file
internal hyperlink =
Y = the web hyperlinks inside jumps to the position in the PDF internally
N = the web hyperlinks inside opens an external web link in your browser
downloaded as .htm =
Y = the webpage is downloaded as .htm then converted to PDF
N = the webpage is directly converted in Chrome browser
file format looking
(Though I said "looks same as the webpage", its not "exactly" same as the webpage -- you need to config the settings when you convert.
Also some minor parts / components of the webpage may or may not be contained in the PDF.)
Calibre Usage
To use Calibre (As shown, Calibre contains the bookmark. But it doesnt have internal hyperlink.)
webpage is downloaded as .htm (along with a folder)
drag the .htm into Calibre, it becomes a .zip file
use Convert books to convert .zip to .pdf
You may need to set up the bookmark detection mechanism in Convert books > Table of Contents if Calibre doesnt detect it.
Calibre is highly customizable on the conversion
(wish I know how to solve the issue of "not having internal hyperlink" directly inside Calibre, without going through HTTrack)
To use Calibre, with HTTrack to add internal hyperlink:
use HTTrack to download the webpage
(with depth of level of 1 (--ie: just current webpage), should be enough)
(you may need to config it so that it captures external files like images / syntax-format files)
drag the index.html into Calibre ... (proceed same as [2~4] above)
(you need to enable the option of creating the index.html)
WPS docx Usage (not recommend)
webpage is downloaded as .htm (along with a folder)
save as .docx
output as .pdf (enable the option convert title style format to bookmark)
(if no title style format is detected, that may due to the title are actually in the style format of hyperlink style format, you need to manually remove all those hyperlink style format.)
note
testing subject weblink is this ; (testing result PDF are not posted here)
Again, I could be wrong -- results may not be accurate due to personal settings / mis-used
Personally, I believe big companies like Adobe should have such functionality to include bookmarks in PDF. It just I dont know how to do it...
I'm developing a function (C#, ASP.NET 4) which creates and downloads a .ics file.
I'm trying to figure out what restrictions there are on the html within the X-ALT-DESC property. For example, if I send this:
X-ALT-DESC;FMTTYPE=text/html:<html><body><ul><li style="font-weight:bold">#1</li><li>#2</li></ul><table border=1><tr><td>table test</td></tr></table><span style="font-weight:bold">Site Visit Agenda</span><br/><span>8:00 AM</span><br/><span>Check in with management<br/>Facility Inspection<br/>Training and Meeting Setup</span></tr></table></body><html>
When I open the downloaded file, it opens in Outlook 2010, so that seems o.k. While the list stuff renders as I'd expect, the table border doesn't show, nor do either of the font-weight settings get rendered. (On the plus side, the html tags aren't rendered as text, it just doesn't format the alt desc like you'd see in a browser with the same HTML).
Of course, we're trying to figure out what HTML we can and cannot use to format the information we want to show up in the alt description.
Searching around, I can't find anything which talks about what's allowed and what isn't.
Thanks to anyone who has a pointer.
My web application has an feature wherein preformatted RTF documents are used as templates and the user can select the source of data and then merge with the RTF documents templates to create merged RTF files. The RTF templates have placeholders which get replaced with user selected content. The final doc can either be saved or opened directly if word/wordpad is available on the local users machine.
Now, I have a requirement to display the merged document to the user for confirmation. The user may either print or save the document to the system directly. The display should not be word/wordpad application but should be within the application itself, using textarea or something similar to render the document. Can you please let me know if its possible to render the RTF document in textarea or not. Along with the displayed content, there should be options to print and save the document.If I have to convert the RTF to Html and then display the html content in textarea , please let me know how i can do the conversion and then display the html in the page.
That's a very difficult requirement. First of all, let's dismiss the idea about a <textarea>, because it does not support any formatting at all. All the WYSIWYG editors you've seen out there are based on <iframe>s.
Secondly, no browser can directly display a RTF. You can embed it as an <object>, and some might show it (IE probably will), but I can't say which ones won't. Portable devices almost certainly won't. But you should test this though, maybe it works well enough after all.
Failing that, HTML conversion is also out of question, because RTF has very very many features that cannot be emulated in HTML. There are some converters out there (google), but but they will all come with serious limitations. If you want full support, you will have to do your own rendering via Canvas or Flash or something.
To this end I'd suggest checking out Google Docs. They've gone through all of this hassle and have a rather feature-full engine for displaying most possible documents. I think it was also possible to embed them in your own webapges, though I've never checked it out myself.
Use a <PRE> tag to Display/Render RTF doc in browser.
I teach GUI and use web pages rather than PP.
I am starting to add HTML5 and CSS instruction to the course.
for a file style_by_kind.html
CSS example style_by_kind.html
when student clicks on the link, obviously, runs the file and displays.
the students can use "save as" in their browser to get a copy for themselves.
I want to display the source as text so I can lecture about the html CSS code.
The link runs the code.
Physically including the file runs the html code.
does not applicable, output only
does not display, comment
Because of continuous updates, it is undesirable to copy the
file with another file type extension, and impossible to "quote" out the html.
Any easy syntax to put in the link or around a physically included copy,
so the student just sees plain text, would be greatly appreciated.
I would suggest changing the extension of the html file to txt.
So when u browse to the txt file, it will still get displayed in the browser with the exact same coding.
Depending on how the files are served up, you might be able to set a ContentType header as text/plain, but I suspect most browsers will still sniff the MIMETYPE of the file and render it as html.
Using the <pre> tag as #hamlin11 suggests is a good idea.
Use <plaintext></plaintext> tags
For example, "This is a lesson on bold tags: <plaintext><b>Bold Text</b></plaintext> will display as <b>Bold Text</b>"
I would like to know whether it a possible to detect whether a HTML page contains a video.
I know that one possible way is to look for ".swf" in the HTML source code. But most of the pages do not contain the file name.
For example, given following URL and possibly its source code, is it possible to find out whether it contains a video:
http://www.cnn.com/video/
There are many ways to embed Video into a HTML page - as Flash Video or instances of Platform-Specific players through <object> and <embed> tags (but not every one of those tags is a video! The same holds true for .swf - it's just the file extension of Flash files, Video or not), the new HTML 5 <video> tag... They are not impossible to find out but it's a lot of work to catch all possible player types, formats and embed codes, and will result in a lot of false positives / negatives.
Then, there are JavaScript libraries that initialize players after the containing page has loaded - those are almost impossible to detect.
It's still a very complex issue to get video into a web page reliably, and subsequently, it's even more complex to find it out. Depending on what you are trying to achieve, I would consider dropping it.
For your case (CNN site) you can parse Open Graph micro-markup for a video information.
Meta tags such as og:video:type, og:image will help you.
Video hosting services usually support micro-markup, e.g. open graph or scheme.org.
So you can parse these markups.
Check if an <object> tag exists in the DOM and check its content type and parameters. You will find the pattern by yourself.
You can also search for .flv, or .mp4 in the source code.