How to scrape a dynamic website without login and eleminating the CROS issue.
I tried to scrape an article, but since it is not given the full access, I had CROS problems.
Related
I have an eCommerce site where I have limited access to pages via a Site Manager. This gives me access to certain pages and some editable areas of those pages where I can update/modify html code.
I have a payment (https) page that I am trying to modify, (add a couple content statements at the bottom) but I believe the code is being intentionally blocked from displaying on this page.
I have tried to get the software vendor to modify this page for me, but I was told it would change all their customer's pages and they don't want that.
I am wondering if anyone out there would have any ideas or tricks with html code to force show a paragraph and bypass their built-in blocking?
Thanks for your time and help!
I think you won't be able to do it.
At your eCommerce system, you have a certain Role. That role belongs under some Permission. The Permission is set form your vendor.
Do you have any Domain or ID?
If yes, you could use JavaScript to manipulate the HTML but create DOM on Window Load.
http://juniorgoldreport.com/ - is the website I'm trying to create an RSS feed for and SEO better.
https://feedity.com/ - is what I'm using to see if I can create the feed, and then ultimately create an app.
A decent amount of my front page content is built via plugin that organize my post's making the maintenance of the website much easier for me, and allowing my employer to post content with ease as well.
Taking a look at the feed, it seems that the website cannot see anything.
From my understanding these post's are being displayed via a proper HTML format, the have h1 tags and proper div blocks and classes.
Do you guys have any suggestions as to how I should I could go about creating a proper feed? Is this a common issue within wordpress?
I'm currently building another website from scratch with a custom CMS (not via wordpress)
What are you suggestions?
RSS feeds are built into Wordpress. Take a look at this page from the Codex about feeds and how to access them.
As a primer you use http://juniorgoldreport.com/feed/ and get the RSS. If you want to access a specific category you'd use http://juniorgoldreport.com/category/[categoryslug]/feed. You can do the same thing with tags.
As for a non Wordpress site I can't say what the best way would be. There are many options.
I have to code a little html site in my informatic class and we have to give it to a client when finished. The thing is, he wants to be able to add posts regulary without having to go into the code. Is there a way to, like, translate raw text into an all-ready tag and automatically implement it on the website ?
you could use wordpress. Is it possible to convert your website to wordpress template? It have admin panel, also there is application for iOS. You could post through emails. It is quite complicated to make it by yourself as there are few security issues you have to be aware. Other way is to use other frameworks like http://get-simple.info which uses text databases in files.
I'm creating a Shopify store for a client who does not obviously know coding. He will need to update content on the website (and I have coded some pages with certain layouts (in the html part of the content) so when he types his content it often breaks our design)
As an alternative solution, I'm thinking of using the blog feature.
I know Shopify lets us create a page with only a certain blog, but what it does is show up in the URL - as a ...blog/cocktails.
That is not good for the end user, because I merely want it to be a page that calls the blog articles from this blog. I'm unable to find a way to do it.
I tried replacing blog.content with blog['cocktail'].content and etc..
but it didn't do anything.
Alternatively are there any suggestions how you can design a page and let clients add their own content just by typing (for instances where its a list of recipes for example)?
I think it would be best to create a custom page template same as blog page.
It will solve your all problems(i think) because first your url problem will be solved by it user has to just assign a template from backend.
Second one most important user can update its content from backend and it will not disturb your html.
If you still have problems just customise html and css.
I hope it will help you.
I have used .NET and ShDocVw for years to grab data off webpages without any issues I couldn't overcome. This website has me beat though. It seems like such as easy task to grab the titles and other information off a library search page, but I can't see the data to be able to grab it. Usually, I just look in the DOM, but the data wasn't there. I did a view source, but the data wasn't there. I am so confused.
I am learning Android right now and that is how I would like to solve my problem, but if .NET would be easier... Right now I will take any help, in any form.
The URL is http://catalog.kcls.org/opac/en-US/skin/kcls/xml/rresult.xml?if=&it=h&bl=&lf=&a=&la=&cl=&d=1&l=1&s=pubdate&sd=desc&adt=ml&tp=&t=bibcn%3ADVD%20FIC%20ON%20ORDER&av=&rt=multi
For this specific website, if you disable javascript in your browser, you will see they give you a link to a plain html search portal
http://catalog.kcls.org/opac/en-US/extras/slimpac/start.html