I have added this tag to my head tag in html I can see the image when I have inspected the code but I can't get the icon displayed.
<link rel = "shortcut icon" type = "image" href = "{% static 'images/logo.png' %}">
I even tried
<link rel = "shortcut icon" type = "image/png" href = "{% static 'images/logo.png' %}">
and
<link rel = "shortcut icon" type = "image/x-icon" href = "{% static 'images/logo.png' %}">
but It's of no use
I have referred this link add logo meta to html
my code goes as this
<head>
<meta charset="UTF-8">
<meta name="viewport"
content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<link rel = "shortcut icon" type = "image" href = "{% static 'images/logo.png' %}">
</head>
Everything is fine. You just need to clear cache/cookies for your localhost.
Related
I am working on a project in which i have to scrape images related to a keyword from a image site. When i search for any keyword on imgur(My choice for the image site), the results are shown as small thumbnails which when clicked open the main article with various images on it. My program for now makes a list of various links in the thumbnails and opens them one by one to download all images in it.
My problem is that when i inspect image element on the article it shows that the image is in class ".image-placeholder" but when i download the html by request method it does not show any such class available.
One of the examples is that when i request response object of article https://imgur.com/gallery/rfEAUDW.
I get below html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<meta content="width=device-width,initial-scale=1" name="viewport"/>
<meta content="funny, image, gif, gifs, memes, jokes, image upload, upload image, lol, humor, vote, comment, share, imgur, imgur.com, wallpaper" name="keywords">
<meta content="Discover the magic of the internet at Imgur, a community powered entertainment destination. Lift your spirits with funny jokes, trending memes, entertaining gifs, inspiring stories, viral videos, and so much more." name="description">
<meta content="Copyright 2020 Imgur, Inc." name="copyright"/>
<link href="https://s.imgur.com/images/favicon-32x32.png" rel="icon" sizes="32x32" type="image/png"/>
<link href="https://s.imgur.com/images/favicon-96x96.png" rel="icon" sizes="96x96" type="image/png"/>
<link href="https://s.imgur.com/images/favicon-16x16.png" rel="icon" sizes="16x16" type="image/png"/>
<link href="https://s.imgur.com/images/favicon-152.png" rel="apple-touch-icon-precomposed"/>
<meta content="#2cd63c" name="msapplication-TileColor"/>
<meta content="https://s.imgur.com/images/favicon-144.png" name="msapplication-TileImage"/>
<link href="https://m.imgur.com/" media="only screen and (max-width: 640px)" rel="alternate"/>
<meta content="834554521765408b9effdc758b69c5ee" name="p:domain_verify">
<meta content="Imgur" property="og:site_name">
<meta content="12331492" property="fb:admins"/>
<meta content="12301369" property="fb:admins"/>
<meta content="127621437303857" property="fb:app_id"/>
<meta content="imgur://imgur.com/?from=fbreferral" property="al:android:url"/>
<meta content="Imgur" property="al:android:app_name"/>
<meta content="com.imgur.mobile" property="al:android:package"/>
<meta content="imgur://imgur.com/?from=fbreferral" property="al:ios:url"/>
<meta content="639881495" property="al:ios:app_store_id"/>
<meta content="Imgur" property="al:ios:app_name"/>
<meta content="https://imgur.com/" property="al:web:url"/>
<meta content="#imgur" name="twitter:site"/>
<meta content="imgur.com" name="twitter:domain"/>
<meta content="com.imgur.mobile" name="twitter:app:id:googleplay"/>
<meta content="Imgur" property="author"/>
<meta content="Imgur" property="article:author"/>
<meta content="https://www.facebook.com/imgur" property="article:publisher"/>
<title>
Drawing Model (for you all naruto fans) - Imgur
</title>
<meta content="https://imgur.com/gallery/rfEAUDW" data-react-helmet="true" property="og:url"/>
<meta content="https://i.imgur.com/rfEAUDWh.jpg" data-react-helmet="true" name="twitter:image"/>
<link href="https://api.imgur.com/oembed.json?url=https://imgur.com/gallery/rfEAUDW" rel="alternate" title="Drawing Model (for you all naruto fans) - Imgur" type="application/json+oembed"/>
<link href="https://api.imgur.com/oembed.xml?url=https://imgur.com/gallery/rfEAUDW" rel="alternate" title="Drawing Model (for you all naruto fans) - Imgur" type="application/xml+oembed"/>
<meta content="funny" data-react-helmet="true" property="article:tag"/>
<meta content="" data-react-helmet="true" property="article:tag"/>
<meta content="" data-react-helmet="true" property="article:tag"/>
<meta content="" data-react-helmet="true" property="article:tag"/>
<meta content="" data-react-helmet="true" property="article:tag"/>
<meta href="https://imgur.com/gallery/rfEAUDW" rel="canonical"/>
<meta content="none" name="robots"/>
<meta content="600" data-react-helmet="true" property="og:image:width"/>
<meta content="315" data-react-helmet="true" property="og:image:height"/>
<meta content="https://i.imgur.com/rfEAUDW.jpg?fb" data-react-helmet="true" property="og:image"/>
<meta content="article" data-react-helmet="true" property="og:type"/>
<meta content="summary_large_image" data-react-helmet="true" name="twitter:card"/>
<script>
dataLayer=[];var pbjs=pbjs||{};pbjs.que=pbjs.que||[]
</script>
<script>
!function(e,t,a,n,g){e[n]=e[n]||[],e[n].push({"gtm.start":(new Date).getTime(),event:"gtm.js"});var m=t.getElementsByTagName(a)[0],r=t.createElement(a);r.async=!0,r.src="//www.googletagmanager.com/gtm.js?id=GTM-M6N38SF",m.parentNode.insertBefore(r,m)}(window,document,"script","dataLayer")
</script>
<link href="https://s.imgur.com/desktop-assets/css/styles.ebc99cf807f6b7c8c39c.css" rel="stylesheet"/>
</meta>
</meta>
</meta>
</meta>
</head>
<body>
<noscript>
<iframe height="0" src="https://www.googletagmanager.com/ns.html?id=GTM-M6N38SF" style="display:none;visibility:hidden" width="0">
</iframe>
</noscript>
<noscript>
If you're seeing this message, that means
<strong>
JavaScript has been disabled on your browser
</strong>
, please
<strong>
enable JS
</strong>
to make Imgur work.
</noscript>
<div id="root">
</div>
<script async="" src="https://www.googletagmanager.com/gtag/js?id=UA-6671908-15">
</script>
<script>
function gtag(){dataLayer.push(arguments)}window.dataLayer=window.dataLayer||[],gtag("js",new Date),gtag("config","UA-6671908-15",{send_page_view:!1})
</script>
<script class="abp" src="https://s.imgur.com/min/px.js?ch=1">
</script>
<script class="abp" src="https://s.imgur.com/min/px.js?ch=2">
</script>
<script src="https://s.imgur.com/desktop-assets/js/main.2e34f379cd8d1a3ca8b1.js">
</script>
</body>
</html>
Thanks in advance for all your help.
My code is:
import requests
from bs4 import BeautifulSoup
from pathlib import Path
import os
#UserInput for phrase to search.
phrase = input('Enter text to search: ')
finalphrase = phrase.replace(" ", "+")
print('Searching for ' + finalphrase + '.....')
#makes soup of the search page and collect all results as galleryElem
site = 'https://imgur.com'
res = requests.get(site + '/search?q=' + phrase)
soup = BeautifulSoup(res.text, 'html.parser')
galleryElem = soup.select('.image-list-link')
#For each thumbnail will open it and download its content
for n in range(len(galleryElem)):
imageLink = galleryElem[n].get('href')
res1 = requests.get(site+imageLink)
soup1 = BeautifulSoup(res1.text, 'html.parser')
imageElem = soup.select('.image-placeholder')
for m in range(len(imageElem)):
image = imageElem[m].get('src')
img = open(os.path.join(finalphrase, number, os.path.basename(image), 'wb'))
number += 1
for a in image.iter_content(100000):
img.write(a)
img.close()
You didn't use selenium. You used requests and bs4. If you have used selenium, you wouldn't have met this problem, because selenium executes javascript for you, while requests doesn't. And the website seems to load images with javascript.
An example with selenium can be found here.
You can check documentations of selenium for more information.
I'm trying to create a static blog using Hugo and these guides (here and here).
I've installed Hugo and created a minimal place-holder blog post for testing. The page renders correctly when I run hugo server -D and go to localhost:1313 in a browser. However, the page doesn't render correctly when I just open public/index.html with a browser.
I'm using the config file that came with the theme that I'm using. In the config file, I've changed the theme attribute to the name of the theme and the baseURL to '/', as well as other smaller fields like author, description, and copyright.
Why is index.html being rendered correctly when it's being served from the local Hugo server but not when I'm just accessing it as a file through the browser?
Here is my blog being correctly rendered through Hugo server
Here is my blog post being rendered incorrectly rendered through direct file access
This is my config file
theme = "soho"
baseURL = "/"
title = "Personal Blog" # "The World's Okayest Programmer"
languageCode = "en"
enableInlineShortcodes = true
summarylength = 10
enableEmoji = true
[params]
author = "My Name"
description = "My Blog"
## Set one of:
# gravatar = "soho#example.com"
profilePicture = "images/profile.png"
copyright = "My Name"
license = "CC BY-SA 4.0"
licenseURL = "https://creativecommons.org/licenses/by-sa/4.0"
## Set custom theme color.
# themeColor = "#fc2803"
## Set custom CSS and/or JS to override site defaults.
customCss = ["css/blog.css"]
customJs = ["js/blog.js"]
## Set as many as you want.
[[params.socialIcons]]
icon = "fa-linkedin"
title = "Linkedin"
url = "#"
[[params.socialIcons]]
icon = "fa-github"
title = "GitHub"
url = "#"
[[params.socialIcons]]
icon = "fa-twitter"
title = "Twitter"
url = "#"
[menu]
[[menu.main]]
name = "Posts"
weight = 100
identifier = "posts"
url = "/posts/"
[[menu.main]]
name = "About"
identifier = "about"
weight = 300
url = "/about/"
[taxonomies]
category = "categories"
series = "series"
tag = "tags"
[markup]
[markup.highlight]
codeFences = true
guessSyntax = false
hl_Lines = ""
lineNoStart = 1
lineNos = false
lineNumbersInTable = true
noClasses = true # if false, you need to provide you own custom CSS
style = "monokai"
tabWidth = 4
And this is the public/index.html that is generated by Hugo
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Hugo 0.72.0" />
<title>Personal Blog</title>
<meta name="description" content="My Blog" />
<meta itemprop="name" content="Personal Blog">
<meta itemprop="description" content="My Blog">
<meta name="twitter:card" content="summary" />
<meta name="twitter:title" content="Personal Blog" />
<meta name="twitter:description" content="My Blog" />
<meta property="og:title" content="Personal Blog" />
<meta property="og:description" content="My Blog" />
<meta property="og:type" content="website" />
<meta property="og:url" content="/" />
<link type="text/css" rel="stylesheet" href="/css/print.css" media="print">
<link type="text/css" rel="stylesheet" href="/css/poole.css">
<link type="text/css" rel="stylesheet" href="/css/hyde.css">
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:400,400i,700&display=swap">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/all.min.css"
integrity="sha256-mmgLkCYLUQbXn0B1SRqzHar6dCnv9oZFPEC1g1cwlkk=" crossorigin="anonymous" />
<link rel="apple-touch-icon-precomposed" sizes="144x144" href="/apple-touch-icon-144-precomposed.png">
<link rel="shortcut icon" href="/favicon.png">
<link href="/index.xml" rel="alternate" type="application/rss+xml" title="Personal Blog" />
</head>
<body>
<aside class="sidebar">
<div class="container">
<div class="sidebar-about">
<div class="author-image">
<img src="/images/profile.png" class="img-circle img-headshot center" alt="Profile Picture">
</div>
<h1>Personal Blog</h1>
<p class="lead">My Blog</p>
</div>
<nav>
<ul class="sidebar-nav">
<li>
Home
</li>
<li>
Posts
</li>
<li>
About
</li>
</ul>
</nav>
<section class="social-icons">
<a href="#" rel="me" title="Linkedin" target="_blank">
<i class="fab fa-linkedin" aria-hidden="true"></i>
</a>
<a href="#" rel="me" title="GitHub" target="_blank">
<i class="fab fa-github" aria-hidden="true"></i>
</a>
<a href="#" rel="me" title="Twitter" target="_blank">
<i class="fab fa-twitter" aria-hidden="true"></i>
</a>
</section>
</div>
</aside>
<main class="content container">
<div class="posts">
</div>
</main>
<footer>
<div>
© My Name 2020
· CC BY-SA 4.0
</div>
</footer>
<script src="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/js/all.min.js"
integrity="sha256-MAgcygDRahs+F/Nk5Vz387whB4kSK9NXlDN3w58LLq0=" crossorigin="anonymous"></script>
</body>
</html>
Assuming your layout files use relURL,¹ put the following in your config.toml.
relativeURLs = true
Make sure...
this line is above all TOML tables
the string relativeURLs uses these upper- and lower-case characters
This is documented at https://gohugo.io/content-management/urls/#relative-urls:
By default, all relative URLs are left unchanged by Hugo, which can be problematic when you want to make your site browsable from a local file system. Setting relativeURLs to true in your site configuration will cause Hugo to rewrite all relative URLs to be relative to the current content. For example, if your /posts/first/ page contains a link to /about/, Hugo will rewrite the URL to ../../about/.
¹ relURL is documented at https://gohugo.io/functions/relurl/ and your theme, which I think is https://github.com/alexandrevicenzi/soho, uses it.
It looks like the CSS files are not loaded in your page. Most likely they are not referenced properly in the HTML file, i.e. in the theme or layouts.
In your config.toml I see that you've set the baseURL to /, but it's supposed to include the hostname, so something like https://yoursite.com/. Check Hugo's config reference.
When running locally with hugo server, you can override that setting with the --baseURL flag.
I am looking to scrape player prices on https://www.fanteam.com/participate/138905/new/e30= using Python and Selenium libraries. I have used the following code:
url = 'https://www.fanteam.com/participate/138905/new/e30='
options = webdriver.ChromeOptions()
options.add_argument('--lang=en')
driver = webdriver.Chrome(chrome_options=options)
driver.get(url)
But I can't get all the players with prices, because I can't find any element on the page(see the picture below
players with prices).
There is HTML of this site:
<!DOCTYPE html>
<html lang="en">
<head>
<script type='text/javascript'>
</script>
<meta charset="UTF-8">
<link rel="shortcut icon" type="image/x-icon" href="/assets/favicon.ico">
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no, minimal-ui">
<meta name="mobile-web-app-capable" content="yes">
<meta property="og:title" content="FanTeam: The home of Fantasy Sports">
<meta property="og:description" content="Create Your Daily Fantasy Team, Play & Win Cash!">
<meta property="og:site_name" content="FanTeam">
<meta property="og:image:width" content="300">
<meta property="og:image:height" content="300">
<meta property="og:url" content="https://www.fanteam.com/participate/138905/new/e30=">
<meta property="og:image" content="https://www.fanteam.com/assets/og-banner.png">
<link href="https://fonts.googleapis.com/css?family=Open+Sans:400,300,600,700,800&subset=latin,cyrillic,cyrillic-ext,latin-ext" rel="stylesheet" type="text/css">
<link rel="manifest" href="/manifest.json">
<script>
(function(getDescriptor) {
Object.getOwnPropertyDescriptor = function(obj, key) {
var descriptor = getDescriptor.apply(this, arguments)
if (!descriptor && obj === window && key == "showModalDialog") {
return {}
}
return descriptor
}
}(Object.getOwnPropertyDescriptor));
</script>
<style>
</style>
<title>FanTeam - Daily Fantasy & Betting</title>
</head>
<body>
<ft-cookie-warning></ft-cookie-warning>
<main>
<ft-header logo="fanteam-logo.svg" logosmall="logosmall.svg"></ft-header>
<section class="ft-view-port-wrapper">
<view-port></view-port>
</section>
<ft-footer tabindex="-1" logo="fanteam-logo.svg"></ft-footer>
<ft-push-receiver></ft-push-receiver>
<ft-olark></ft-olark>
</main>
<script src="https://cdnjs.cloudflare.com/ajax/libs/webcomponentsjs/1.0.6/webcomponents-lite.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/babel-polyfill/6.26.0/polyfill.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/fetch/2.0.3/fetch.min.js"></script>
<script src="/build/application-b8ab977b2a.js" data-root="https://fanteam-game.api.scoutgg.net" data-ws="https://fanteam-game.ws.scoutgg.net" data-auth-url="" data-white-label="fanteam" data-olark="8903-397-10-7512" data-google-analytics="UA-55860585-1"
data-asset-host="https://d34h6ikdffho99.cloudfront.net" data-vapid-public-key="BH8zySo8DKTd9EY0koPSAmA7fo58QTVuFjcB4hTp95WDu21l4dwjckigl0hpYBgeS-6h2kbMtfbXw4u4097wK3w" data-scoutcc="https://scoutcc.scoutgg.net" data-payment-url="https://globpay.fantasy.solutions/v1"
data-projection-url="https://betflex-projection.api.scoutgg.net//api/v1" data-sportsbook-path="https://stage.fenixplayground.es/apuestas/mobilegoto.aspx" data-service-worker="sw.js"></script>
</body>
</html>
Any code like
el = driver.find_element_by_xpath("//div[#class='player-list']")
return me the error:
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[#class='player-list']"}
But when I inspect an element I can see it in the browser.
How to click any element on the page?
The website you are trying to scrape has a shadow-DOM in its html and any html present inside it cannot be accessed and that is the reason you are getting NoSuchElementException.
Currently, selenium does not support the shadow DOM automation, so you need to use javascript in this case to scrape the data.
To get the data using javascript, you can use:
JavascriptExecutor js = (JavascriptExecutor) driver;
String return_value = (String) js.execute_script("return document.getElementByXpath('xpath').innerHTML");
References for the shadow DOM:
https://medium.com/rate-engineering/a-guide-to-working-with-shadow-dom-using-selenium-b124992559f
https://www.seleniumeasy.com/selenium-tutorials/accessing-shadow-dom-elements-with-webdriver
I read a lot of posts about this but none of the solutions seems to work in my case. I learned a lot of stuff reading all these but my problem is still there.
I always get this error message :
The 'og:image' property should be explicitly provided, even if a value can be inferred from other tags.
But in my source code there is :
<meta property="og:image" content="http://www.shootmeagain.com/photo/Y2013/3769/saintvitus-oslo_30-03-2013_17.jpg" />
All my header is here :
<!DOCTYPE html><html xmlns:og="http://ogp.me/ns#"><head><title>PHOTOS : Saint Vitus à Oslo (Rockefeller) le 30-03-2013 | Shoot Me Again Webzine.</title>
<meta charset="UTF-8">
<link rel=canonical href='http://www.shootmeagain.com/photos/3769_saintvitus_oslo_30-03-2013'>
<meta property="og:description" content="PHOTOS : Saint Vitus à Oslo (Rockefeller) le 30-03-2013 | Shoot Me Again Webzine." />
<meta property="og:title" content="PHOTOS : Saint Vitus à Oslo (Rockefeller) le 30-03-2013 | Shoot Me Again Webzine." />
<meta property="og:url" content="http://www.shootmeagain.com/photos/3769_saintvitus_oslo_30-03-2013" />
<meta property="og:type" content="website" />
<meta property="og:image" content="http://www.shootmeagain.com/photo/Y2013/3769/saintvitus-oslo_30-03-2013_17.jpg" />
<meta property='fb:admins' content='erikshootmeagain' />
<meta property='fb:app_id' content='146373435406736' />
<link rel=stylesheet href='http://www.shootmeagain.com/layout.css' type='text/css'>
<link rel=stylesheet href='http://www.shootmeagain.com/layout600.css' type='text/css'>
<link rel=stylesheet href='http://www.shootmeagain.com/layout900.css' type='text/css'>
<link rel=stylesheet href='http://www.shootmeagain.com/layout1200.css' type='text/css'>
<link rel=stylesheet href='http://www.shootmeagain.com/layout1600.css' type='text/css'>
<link rel=stylesheet href='http://www.shootmeagain.com/layout1900.css' type='text/css'>
<link rel=stylesheet href='http://www.shootmeagain.com/layoutheight.css' type='text/css'>
<link rel=stylesheet href='http://www.shootmeagain.com/js/jquery-ui-1.11.3.custom/jquery-ui.css' type='text/css'>
<link rel=stylesheet href='http://www.shootmeagain.com/js/jquery-ui-1.11.3.custom/jquery-ui.theme.css' type='text/css'>
<link rel="shortcut icon" href="http://www.shootmeagain.com/images/favicon.ico">
<link rel="icon" type="image/gif" href="http://www.shootmeagain.com/images/animated_favicon1.gif">
<meta name='keywords' content='webzine, chroniques, photos, interview, concours, mp3, video, hardcore, rock, emo, screamo, metal, punk, garage'>
<meta name='author' content='ChamO'>
<meta name='publisher' content='ChamO'>
<meta name='viewport' content='width=device-width'>
<meta http-equiv="content-language" content="fr">
<link href='http://fonts.googleapis.com/css?family=Lato:100,200,300,400,500,600,700,800,900' rel='stylesheet' type='text/css'>
<script language="javascript" type="text/javascript" src="http://www.shootmeagain.com/js/jquery-1.11.2.min.js"></script>
<script language="javascript" type="text/javascript" src="http://www.shootmeagain.com/js/jquery-ui-1.11.3.custom/jquery-ui.min.js"></script>
<script language="javascript" type="text/javascript" src="http://www.shootmeagain.com/js/editPost.js"></script>
</head>
Here is an URL for example : http://www.shootmeagain.com/photos/3769_saintvitus_oslo_30-03-2013
Does anybody have an idea ?
Thanks a lot !
Erik
You can fix this issue by adding the og:image width, height, alt and secure_url properties on separate lines:
<meta property="og:image:width" content="1200">
<meta property="og:image:height" content="630">
<meta property="og:image:alt" content="description_of_site">
<meta property="og:image:secure_url" content="https://your_domain/my_og_image.png">
That should pass.
FB also complains if the fb:app_id is missing too, but you can leave that out.
<head>
<title>Free Jokes!</title>
<meta charset="utf-8">
<style type = "text/css" src = "final project.css"></style>
</head>
I have a separate css file that is validated. But the main reason I think this isn't working is that when I validated the html, it says,"Attribute src not allowed on element style at this point."
Any suggestions? Thanks in advance!
Also any other syntax errors would be much appreciated...
replace
<style type = "text/css" src = "final project.css"></style>
with:
<link href="final project.css" rel="stylesheet">
have a look at this on how to include css : http://www.w3schools.com/css/css_howto.asp
<head>
<title>Free Jokes!</title>
<meta charset="utf-8" />
<link type="text/css" href="final project.css" rel="stylesheet" />
</head>
Link element uses "href" not "src"
try
<link rel="stylesheet" href="final project.css">
add only rel attribute:
<link type="text/css" href="final project.css" rel="stylesheet" />
and also avoid using space in file name.