I'm trying to create a static blog using Hugo and these guides (here and here).
I've installed Hugo and created a minimal place-holder blog post for testing. The page renders correctly when I run hugo server -D and go to localhost:1313 in a browser. However, the page doesn't render correctly when I just open public/index.html with a browser.
I'm using the config file that came with the theme that I'm using. In the config file, I've changed the theme attribute to the name of the theme and the baseURL to '/', as well as other smaller fields like author, description, and copyright.
Why is index.html being rendered correctly when it's being served from the local Hugo server but not when I'm just accessing it as a file through the browser?
Here is my blog being correctly rendered through Hugo server
Here is my blog post being rendered incorrectly rendered through direct file access
This is my config file
theme = "soho"
baseURL = "/"
title = "Personal Blog" # "The World's Okayest Programmer"
languageCode = "en"
enableInlineShortcodes = true
summarylength = 10
enableEmoji = true
[params]
author = "My Name"
description = "My Blog"
## Set one of:
# gravatar = "soho#example.com"
profilePicture = "images/profile.png"
copyright = "My Name"
license = "CC BY-SA 4.0"
licenseURL = "https://creativecommons.org/licenses/by-sa/4.0"
## Set custom theme color.
# themeColor = "#fc2803"
## Set custom CSS and/or JS to override site defaults.
customCss = ["css/blog.css"]
customJs = ["js/blog.js"]
## Set as many as you want.
[[params.socialIcons]]
icon = "fa-linkedin"
title = "Linkedin"
url = "#"
[[params.socialIcons]]
icon = "fa-github"
title = "GitHub"
url = "#"
[[params.socialIcons]]
icon = "fa-twitter"
title = "Twitter"
url = "#"
[menu]
[[menu.main]]
name = "Posts"
weight = 100
identifier = "posts"
url = "/posts/"
[[menu.main]]
name = "About"
identifier = "about"
weight = 300
url = "/about/"
[taxonomies]
category = "categories"
series = "series"
tag = "tags"
[markup]
[markup.highlight]
codeFences = true
guessSyntax = false
hl_Lines = ""
lineNoStart = 1
lineNos = false
lineNumbersInTable = true
noClasses = true # if false, you need to provide you own custom CSS
style = "monokai"
tabWidth = 4
And this is the public/index.html that is generated by Hugo
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Hugo 0.72.0" />
<title>Personal Blog</title>
<meta name="description" content="My Blog" />
<meta itemprop="name" content="Personal Blog">
<meta itemprop="description" content="My Blog">
<meta name="twitter:card" content="summary" />
<meta name="twitter:title" content="Personal Blog" />
<meta name="twitter:description" content="My Blog" />
<meta property="og:title" content="Personal Blog" />
<meta property="og:description" content="My Blog" />
<meta property="og:type" content="website" />
<meta property="og:url" content="/" />
<link type="text/css" rel="stylesheet" href="/css/print.css" media="print">
<link type="text/css" rel="stylesheet" href="/css/poole.css">
<link type="text/css" rel="stylesheet" href="/css/hyde.css">
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:400,400i,700&display=swap">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/css/all.min.css"
integrity="sha256-mmgLkCYLUQbXn0B1SRqzHar6dCnv9oZFPEC1g1cwlkk=" crossorigin="anonymous" />
<link rel="apple-touch-icon-precomposed" sizes="144x144" href="/apple-touch-icon-144-precomposed.png">
<link rel="shortcut icon" href="/favicon.png">
<link href="/index.xml" rel="alternate" type="application/rss+xml" title="Personal Blog" />
</head>
<body>
<aside class="sidebar">
<div class="container">
<div class="sidebar-about">
<div class="author-image">
<img src="/images/profile.png" class="img-circle img-headshot center" alt="Profile Picture">
</div>
<h1>Personal Blog</h1>
<p class="lead">My Blog</p>
</div>
<nav>
<ul class="sidebar-nav">
<li>
Home
</li>
<li>
Posts
</li>
<li>
About
</li>
</ul>
</nav>
<section class="social-icons">
<a href="#" rel="me" title="Linkedin" target="_blank">
<i class="fab fa-linkedin" aria-hidden="true"></i>
</a>
<a href="#" rel="me" title="GitHub" target="_blank">
<i class="fab fa-github" aria-hidden="true"></i>
</a>
<a href="#" rel="me" title="Twitter" target="_blank">
<i class="fab fa-twitter" aria-hidden="true"></i>
</a>
</section>
</div>
</aside>
<main class="content container">
<div class="posts">
</div>
</main>
<footer>
<div>
© My Name 2020
· CC BY-SA 4.0
</div>
</footer>
<script src="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.12.1/js/all.min.js"
integrity="sha256-MAgcygDRahs+F/Nk5Vz387whB4kSK9NXlDN3w58LLq0=" crossorigin="anonymous"></script>
</body>
</html>
Assuming your layout files use relURL,¹ put the following in your config.toml.
relativeURLs = true
Make sure...
this line is above all TOML tables
the string relativeURLs uses these upper- and lower-case characters
This is documented at https://gohugo.io/content-management/urls/#relative-urls:
By default, all relative URLs are left unchanged by Hugo, which can be problematic when you want to make your site browsable from a local file system. Setting relativeURLs to true in your site configuration will cause Hugo to rewrite all relative URLs to be relative to the current content. For example, if your /posts/first/ page contains a link to /about/, Hugo will rewrite the URL to ../../about/.
¹ relURL is documented at https://gohugo.io/functions/relurl/ and your theme, which I think is https://github.com/alexandrevicenzi/soho, uses it.
It looks like the CSS files are not loaded in your page. Most likely they are not referenced properly in the HTML file, i.e. in the theme or layouts.
In your config.toml I see that you've set the baseURL to /, but it's supposed to include the hostname, so something like https://yoursite.com/. Check Hugo's config reference.
When running locally with hugo server, you can override that setting with the --baseURL flag.
Related
I've been very confused, i am using .json file as a Database for my HTML Contents (<p>example</p>)
When i am trying to put the JSON Value from main.json to my HTML Document index2.html using Flask (Python), but it doesn't seem to work, and just returning <p>Here begins the chat</p> on my website.
Here are the codes
main.json
{
"posts": {
"divs": "<p>Here begins the chat!</p>"
}
}
index2.html
<html>
<head>
<title>Hit-Git | Social Media Platform</title>
<link rel="shortcut icon" href="{{ url_for('static', filename='michellelogo.ico') }}">
<link rel= "stylesheet" type= "text/css" href= "{{url_for('static', filename='styles/style.css') }}">
<meta name="viewport" content="width=device-width, initial-scale=1">
<ul>
<button type="button" class="topbutton" onclick="window.location.href='/'">Home</button>
<button type="button" class="topbutton active" onclick="window.location.href='/login'">News Feed</button>
<button style="float:right" type="button" class="books" onclick="window.location.href='/'">Hit-Git</button>
</ul>
</head>
<body>
<div style="text-align: center;">
<h3>News Feed is Under Construction!</h3>
<p>News Feed is currently on under testing and building, trial and errors are still happening, but rest assured, soon you will be able to use most of Hit-Git! Thank you for your patience.</p>
</div>
{{posts}}
</body>
</html>
and the main.py
#app.route("/posts", methods=['GET', 'POST'])
def posts():
allpost = data1()
daposts = allpost['posts']['divs']
daposts = str(daposts)
return render_template('posts.html', posts=daposts)
Sorry for my bad english
I have added this tag to my head tag in html I can see the image when I have inspected the code but I can't get the icon displayed.
<link rel = "shortcut icon" type = "image" href = "{% static 'images/logo.png' %}">
I even tried
<link rel = "shortcut icon" type = "image/png" href = "{% static 'images/logo.png' %}">
and
<link rel = "shortcut icon" type = "image/x-icon" href = "{% static 'images/logo.png' %}">
but It's of no use
I have referred this link add logo meta to html
my code goes as this
<head>
<meta charset="UTF-8">
<meta name="viewport"
content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<link rel = "shortcut icon" type = "image" href = "{% static 'images/logo.png' %}">
</head>
Everything is fine. You just need to clear cache/cookies for your localhost.
I am working on a project in which i have to scrape images related to a keyword from a image site. When i search for any keyword on imgur(My choice for the image site), the results are shown as small thumbnails which when clicked open the main article with various images on it. My program for now makes a list of various links in the thumbnails and opens them one by one to download all images in it.
My problem is that when i inspect image element on the article it shows that the image is in class ".image-placeholder" but when i download the html by request method it does not show any such class available.
One of the examples is that when i request response object of article https://imgur.com/gallery/rfEAUDW.
I get below html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<meta content="width=device-width,initial-scale=1" name="viewport"/>
<meta content="funny, image, gif, gifs, memes, jokes, image upload, upload image, lol, humor, vote, comment, share, imgur, imgur.com, wallpaper" name="keywords">
<meta content="Discover the magic of the internet at Imgur, a community powered entertainment destination. Lift your spirits with funny jokes, trending memes, entertaining gifs, inspiring stories, viral videos, and so much more." name="description">
<meta content="Copyright 2020 Imgur, Inc." name="copyright"/>
<link href="https://s.imgur.com/images/favicon-32x32.png" rel="icon" sizes="32x32" type="image/png"/>
<link href="https://s.imgur.com/images/favicon-96x96.png" rel="icon" sizes="96x96" type="image/png"/>
<link href="https://s.imgur.com/images/favicon-16x16.png" rel="icon" sizes="16x16" type="image/png"/>
<link href="https://s.imgur.com/images/favicon-152.png" rel="apple-touch-icon-precomposed"/>
<meta content="#2cd63c" name="msapplication-TileColor"/>
<meta content="https://s.imgur.com/images/favicon-144.png" name="msapplication-TileImage"/>
<link href="https://m.imgur.com/" media="only screen and (max-width: 640px)" rel="alternate"/>
<meta content="834554521765408b9effdc758b69c5ee" name="p:domain_verify">
<meta content="Imgur" property="og:site_name">
<meta content="12331492" property="fb:admins"/>
<meta content="12301369" property="fb:admins"/>
<meta content="127621437303857" property="fb:app_id"/>
<meta content="imgur://imgur.com/?from=fbreferral" property="al:android:url"/>
<meta content="Imgur" property="al:android:app_name"/>
<meta content="com.imgur.mobile" property="al:android:package"/>
<meta content="imgur://imgur.com/?from=fbreferral" property="al:ios:url"/>
<meta content="639881495" property="al:ios:app_store_id"/>
<meta content="Imgur" property="al:ios:app_name"/>
<meta content="https://imgur.com/" property="al:web:url"/>
<meta content="#imgur" name="twitter:site"/>
<meta content="imgur.com" name="twitter:domain"/>
<meta content="com.imgur.mobile" name="twitter:app:id:googleplay"/>
<meta content="Imgur" property="author"/>
<meta content="Imgur" property="article:author"/>
<meta content="https://www.facebook.com/imgur" property="article:publisher"/>
<title>
Drawing Model (for you all naruto fans) - Imgur
</title>
<meta content="https://imgur.com/gallery/rfEAUDW" data-react-helmet="true" property="og:url"/>
<meta content="https://i.imgur.com/rfEAUDWh.jpg" data-react-helmet="true" name="twitter:image"/>
<link href="https://api.imgur.com/oembed.json?url=https://imgur.com/gallery/rfEAUDW" rel="alternate" title="Drawing Model (for you all naruto fans) - Imgur" type="application/json+oembed"/>
<link href="https://api.imgur.com/oembed.xml?url=https://imgur.com/gallery/rfEAUDW" rel="alternate" title="Drawing Model (for you all naruto fans) - Imgur" type="application/xml+oembed"/>
<meta content="funny" data-react-helmet="true" property="article:tag"/>
<meta content="" data-react-helmet="true" property="article:tag"/>
<meta content="" data-react-helmet="true" property="article:tag"/>
<meta content="" data-react-helmet="true" property="article:tag"/>
<meta content="" data-react-helmet="true" property="article:tag"/>
<meta href="https://imgur.com/gallery/rfEAUDW" rel="canonical"/>
<meta content="none" name="robots"/>
<meta content="600" data-react-helmet="true" property="og:image:width"/>
<meta content="315" data-react-helmet="true" property="og:image:height"/>
<meta content="https://i.imgur.com/rfEAUDW.jpg?fb" data-react-helmet="true" property="og:image"/>
<meta content="article" data-react-helmet="true" property="og:type"/>
<meta content="summary_large_image" data-react-helmet="true" name="twitter:card"/>
<script>
dataLayer=[];var pbjs=pbjs||{};pbjs.que=pbjs.que||[]
</script>
<script>
!function(e,t,a,n,g){e[n]=e[n]||[],e[n].push({"gtm.start":(new Date).getTime(),event:"gtm.js"});var m=t.getElementsByTagName(a)[0],r=t.createElement(a);r.async=!0,r.src="//www.googletagmanager.com/gtm.js?id=GTM-M6N38SF",m.parentNode.insertBefore(r,m)}(window,document,"script","dataLayer")
</script>
<link href="https://s.imgur.com/desktop-assets/css/styles.ebc99cf807f6b7c8c39c.css" rel="stylesheet"/>
</meta>
</meta>
</meta>
</meta>
</head>
<body>
<noscript>
<iframe height="0" src="https://www.googletagmanager.com/ns.html?id=GTM-M6N38SF" style="display:none;visibility:hidden" width="0">
</iframe>
</noscript>
<noscript>
If you're seeing this message, that means
<strong>
JavaScript has been disabled on your browser
</strong>
, please
<strong>
enable JS
</strong>
to make Imgur work.
</noscript>
<div id="root">
</div>
<script async="" src="https://www.googletagmanager.com/gtag/js?id=UA-6671908-15">
</script>
<script>
function gtag(){dataLayer.push(arguments)}window.dataLayer=window.dataLayer||[],gtag("js",new Date),gtag("config","UA-6671908-15",{send_page_view:!1})
</script>
<script class="abp" src="https://s.imgur.com/min/px.js?ch=1">
</script>
<script class="abp" src="https://s.imgur.com/min/px.js?ch=2">
</script>
<script src="https://s.imgur.com/desktop-assets/js/main.2e34f379cd8d1a3ca8b1.js">
</script>
</body>
</html>
Thanks in advance for all your help.
My code is:
import requests
from bs4 import BeautifulSoup
from pathlib import Path
import os
#UserInput for phrase to search.
phrase = input('Enter text to search: ')
finalphrase = phrase.replace(" ", "+")
print('Searching for ' + finalphrase + '.....')
#makes soup of the search page and collect all results as galleryElem
site = 'https://imgur.com'
res = requests.get(site + '/search?q=' + phrase)
soup = BeautifulSoup(res.text, 'html.parser')
galleryElem = soup.select('.image-list-link')
#For each thumbnail will open it and download its content
for n in range(len(galleryElem)):
imageLink = galleryElem[n].get('href')
res1 = requests.get(site+imageLink)
soup1 = BeautifulSoup(res1.text, 'html.parser')
imageElem = soup.select('.image-placeholder')
for m in range(len(imageElem)):
image = imageElem[m].get('src')
img = open(os.path.join(finalphrase, number, os.path.basename(image), 'wb'))
number += 1
for a in image.iter_content(100000):
img.write(a)
img.close()
You didn't use selenium. You used requests and bs4. If you have used selenium, you wouldn't have met this problem, because selenium executes javascript for you, while requests doesn't. And the website seems to load images with javascript.
An example with selenium can be found here.
You can check documentations of selenium for more information.
I am looking to scrape player prices on https://www.fanteam.com/participate/138905/new/e30= using Python and Selenium libraries. I have used the following code:
url = 'https://www.fanteam.com/participate/138905/new/e30='
options = webdriver.ChromeOptions()
options.add_argument('--lang=en')
driver = webdriver.Chrome(chrome_options=options)
driver.get(url)
But I can't get all the players with prices, because I can't find any element on the page(see the picture below
players with prices).
There is HTML of this site:
<!DOCTYPE html>
<html lang="en">
<head>
<script type='text/javascript'>
</script>
<meta charset="UTF-8">
<link rel="shortcut icon" type="image/x-icon" href="/assets/favicon.ico">
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no, minimal-ui">
<meta name="mobile-web-app-capable" content="yes">
<meta property="og:title" content="FanTeam: The home of Fantasy Sports">
<meta property="og:description" content="Create Your Daily Fantasy Team, Play & Win Cash!">
<meta property="og:site_name" content="FanTeam">
<meta property="og:image:width" content="300">
<meta property="og:image:height" content="300">
<meta property="og:url" content="https://www.fanteam.com/participate/138905/new/e30=">
<meta property="og:image" content="https://www.fanteam.com/assets/og-banner.png">
<link href="https://fonts.googleapis.com/css?family=Open+Sans:400,300,600,700,800&subset=latin,cyrillic,cyrillic-ext,latin-ext" rel="stylesheet" type="text/css">
<link rel="manifest" href="/manifest.json">
<script>
(function(getDescriptor) {
Object.getOwnPropertyDescriptor = function(obj, key) {
var descriptor = getDescriptor.apply(this, arguments)
if (!descriptor && obj === window && key == "showModalDialog") {
return {}
}
return descriptor
}
}(Object.getOwnPropertyDescriptor));
</script>
<style>
</style>
<title>FanTeam - Daily Fantasy & Betting</title>
</head>
<body>
<ft-cookie-warning></ft-cookie-warning>
<main>
<ft-header logo="fanteam-logo.svg" logosmall="logosmall.svg"></ft-header>
<section class="ft-view-port-wrapper">
<view-port></view-port>
</section>
<ft-footer tabindex="-1" logo="fanteam-logo.svg"></ft-footer>
<ft-push-receiver></ft-push-receiver>
<ft-olark></ft-olark>
</main>
<script src="https://cdnjs.cloudflare.com/ajax/libs/webcomponentsjs/1.0.6/webcomponents-lite.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/babel-polyfill/6.26.0/polyfill.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/fetch/2.0.3/fetch.min.js"></script>
<script src="/build/application-b8ab977b2a.js" data-root="https://fanteam-game.api.scoutgg.net" data-ws="https://fanteam-game.ws.scoutgg.net" data-auth-url="" data-white-label="fanteam" data-olark="8903-397-10-7512" data-google-analytics="UA-55860585-1"
data-asset-host="https://d34h6ikdffho99.cloudfront.net" data-vapid-public-key="BH8zySo8DKTd9EY0koPSAmA7fo58QTVuFjcB4hTp95WDu21l4dwjckigl0hpYBgeS-6h2kbMtfbXw4u4097wK3w" data-scoutcc="https://scoutcc.scoutgg.net" data-payment-url="https://globpay.fantasy.solutions/v1"
data-projection-url="https://betflex-projection.api.scoutgg.net//api/v1" data-sportsbook-path="https://stage.fenixplayground.es/apuestas/mobilegoto.aspx" data-service-worker="sw.js"></script>
</body>
</html>
Any code like
el = driver.find_element_by_xpath("//div[#class='player-list']")
return me the error:
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[#class='player-list']"}
But when I inspect an element I can see it in the browser.
How to click any element on the page?
The website you are trying to scrape has a shadow-DOM in its html and any html present inside it cannot be accessed and that is the reason you are getting NoSuchElementException.
Currently, selenium does not support the shadow DOM automation, so you need to use javascript in this case to scrape the data.
To get the data using javascript, you can use:
JavascriptExecutor js = (JavascriptExecutor) driver;
String return_value = (String) js.execute_script("return document.getElementByXpath('xpath').innerHTML");
References for the shadow DOM:
https://medium.com/rate-engineering/a-guide-to-working-with-shadow-dom-using-selenium-b124992559f
https://www.seleniumeasy.com/selenium-tutorials/accessing-shadow-dom-elements-with-webdriver
I have an Angular page www.title.org running via AWS Cloudfront.
This project has some meta information in the index.html file, like:
<meta property="og:title" content="Title" />
<meta property="og:description" content="Bla bla" />
<meta property="og:image" content="image.png" />
<meta property="og:url" content="www.title.org" />
However when I post www.title.org/quiz in social media I want there to be a different thumbnail than image.png in the preview.
I have attempted using Angular Meta in my quiz.component.ts file:
this.meta.updateTag({ name: 'og:title', content: 'Title' })
this.meta.updateTag({ name: 'og:description', content: 'New description' })
this.meta.updateTag({ name: 'og:image', content: 'image_2.png' })
this.meta.updateTag({ name: 'og:url', content: 'www.title.org/quiz' })
but this doesn't change the thumbnail or anything else for that matter.
How do you change thumbnails depending on what route you are linking to for an Angular page?
Update: I attempted create a new index file, with different meta data and a redirect. So in quiz/index.html I have:
<head>
<meta charset="utf-8">
<meta property="og:title" content="My quiz" />
<meta property="og:description" content="Do the quiz" />
<meta property="og:image" content="image_2.png" />
<meta property="og:url" content="https://www.title.org/quiz" />
...
</head>
<body>
<script language="javascript" type="text/javascript">
window.location.href = "https://www.styreskolen.org/quiz";
</script>
</body>
So if I post https://www.styreskolen.org/quiz/index.html I should get the new meta data, but no such luck. It does however work on Skype, so it almost seems like LinkedIn just doesn't care what I do, and just wants to look at my root code.
Okay, in the end my way to solve this was to create a separate bucket with a Cloudfront distribution with its own domain name www.quiz.title.org that only contained an index.html file redirecting you to www.title.org/quiz and the meta data that I wanted.