I'm using ajax to update a value without refreshing the page.
When I use this html code the website gives an 404 not found error. But when I leave the script tag out of the html, the site works fine and show the p tag. What am i doing wrong in the script tag?
<!DOCTYPE html>
<html lang="nl">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Smart_home</title>
</head>
<body>
<div id="verbruik">
<p></p>
</div>
<script>
function loadDoc() {
const xhttp = new XMLHttpRequest();
xhttp.onload = function () {
document.getElementById("verbruik").innerHTML =
this.responseText;
}
xhttp.open("GET", "/home/pi/smart_home/ajax_info.txt", true);
xhttp.send();
setTimeout(loadDoc, 2000);
}
loadDoc();
</script>
</body>
</html>
I would like to get the parameters in a URL and use them to generate an og:image meta tag in my SPA. The specific purpose is to have a dynamic thumbnail for a given url. The idea is for crawlers to be able to find the appropriate thumbnail.
example URL:
https://my.app/#/post?uid=abc&pid=123
These two parameters will not necessarily always be included. I hope it won't cause an issue.
My understanding is that crawlers generally only check the html for metadata. How could I include a bit of code in my html before the metadata? (I am relatively new to HTML)
Would I be able to put a script in the head tag? Are variable in the script available outside of the script? Can I use variable in the URL address of my og:image tag?
<!DOCTYPE html>
<html>
<head>
<meta property="og:image" content="https://my.app/{uid}/{pid}/thumb.png" />
<meta charset="UTF-8">
<meta content="IE=Edge" http-equiv="X-UA-Compatible">
<meta name="description" content="Get Dressed. Better than you ever had">
<!-- iOS meta tags & icons -->
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black">
<meta name="apple-mobile-web-app-title" content="vestiqweb">
<link rel="apple-touch-icon" href="icons/Icon-192.png">
<!-- Favicon -->
<link rel="shortcut icon" type="image/png" href="favicon.png"/>
<title>VESTIQ</title>
<link rel="manifest" href="manifest.json">
</head>
<body id="app-container">
<script>
if ('serviceWorker' in navigator) {
window.addEventListener('load', function () {
navigator.serviceWorker.register('flutter_service_worker.js');
});
}
</script>
<script src="https://www.gstatic.com/firebasejs/7.17.1/firebase-app.js"></script>
<script src="https://www.gstatic.com/firebasejs/7.14.4/firebase-firestore.js"></script>
<script src="https://www.gstatic.com/firebasejs/7.17.1/firebase-auth.js"></script>
<script src="https://www.gstatic.com/firebasejs/7.17.1/firebase-analytics.js"></script>
<script>
var firebaseConfig = {
//config info
};
firebase.initializeApp(firebaseConfig);
firebase.analytics();
</script>
<script src="main.dart.js?version=14" type="application/javascript"></script>
</body>
</html>
I was able to put a script before the meta tags and used window.location.href to get the URL. I used URLSearchParams to extract the parameters. However, I can only get the second parameter and not the first. If I add an '&' before the uid it works, but makes the url look weird... "?&uid"
<script>
const queryString = window.location.href;
const urlParams = new URLSearchParams(queryString);
const uid = urlParams.get('uid')
const pid = urlParams.get('pid')
if (uid != null && pid != null)
document.getElementById('urlThumb').content = `https://my.app/posts%2F${uid}%2F${pid}%2Furl_thumb.jpg?alt=media`;
</script>
I am looking to scrape player prices on https://www.fanteam.com/participate/138905/new/e30= using Python and Selenium libraries. I have used the following code:
url = 'https://www.fanteam.com/participate/138905/new/e30='
options = webdriver.ChromeOptions()
options.add_argument('--lang=en')
driver = webdriver.Chrome(chrome_options=options)
driver.get(url)
But I can't get all the players with prices, because I can't find any element on the page(see the picture below
players with prices).
There is HTML of this site:
<!DOCTYPE html>
<html lang="en">
<head>
<script type='text/javascript'>
</script>
<meta charset="UTF-8">
<link rel="shortcut icon" type="image/x-icon" href="/assets/favicon.ico">
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no, minimal-ui">
<meta name="mobile-web-app-capable" content="yes">
<meta property="og:title" content="FanTeam: The home of Fantasy Sports">
<meta property="og:description" content="Create Your Daily Fantasy Team, Play & Win Cash!">
<meta property="og:site_name" content="FanTeam">
<meta property="og:image:width" content="300">
<meta property="og:image:height" content="300">
<meta property="og:url" content="https://www.fanteam.com/participate/138905/new/e30=">
<meta property="og:image" content="https://www.fanteam.com/assets/og-banner.png">
<link href="https://fonts.googleapis.com/css?family=Open+Sans:400,300,600,700,800&subset=latin,cyrillic,cyrillic-ext,latin-ext" rel="stylesheet" type="text/css">
<link rel="manifest" href="/manifest.json">
<script>
(function(getDescriptor) {
Object.getOwnPropertyDescriptor = function(obj, key) {
var descriptor = getDescriptor.apply(this, arguments)
if (!descriptor && obj === window && key == "showModalDialog") {
return {}
}
return descriptor
}
}(Object.getOwnPropertyDescriptor));
</script>
<style>
</style>
<title>FanTeam - Daily Fantasy & Betting</title>
</head>
<body>
<ft-cookie-warning></ft-cookie-warning>
<main>
<ft-header logo="fanteam-logo.svg" logosmall="logosmall.svg"></ft-header>
<section class="ft-view-port-wrapper">
<view-port></view-port>
</section>
<ft-footer tabindex="-1" logo="fanteam-logo.svg"></ft-footer>
<ft-push-receiver></ft-push-receiver>
<ft-olark></ft-olark>
</main>
<script src="https://cdnjs.cloudflare.com/ajax/libs/webcomponentsjs/1.0.6/webcomponents-lite.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/babel-polyfill/6.26.0/polyfill.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/fetch/2.0.3/fetch.min.js"></script>
<script src="/build/application-b8ab977b2a.js" data-root="https://fanteam-game.api.scoutgg.net" data-ws="https://fanteam-game.ws.scoutgg.net" data-auth-url="" data-white-label="fanteam" data-olark="8903-397-10-7512" data-google-analytics="UA-55860585-1"
data-asset-host="https://d34h6ikdffho99.cloudfront.net" data-vapid-public-key="BH8zySo8DKTd9EY0koPSAmA7fo58QTVuFjcB4hTp95WDu21l4dwjckigl0hpYBgeS-6h2kbMtfbXw4u4097wK3w" data-scoutcc="https://scoutcc.scoutgg.net" data-payment-url="https://globpay.fantasy.solutions/v1"
data-projection-url="https://betflex-projection.api.scoutgg.net//api/v1" data-sportsbook-path="https://stage.fenixplayground.es/apuestas/mobilegoto.aspx" data-service-worker="sw.js"></script>
</body>
</html>
Any code like
el = driver.find_element_by_xpath("//div[#class='player-list']")
return me the error:
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[#class='player-list']"}
But when I inspect an element I can see it in the browser.
How to click any element on the page?
The website you are trying to scrape has a shadow-DOM in its html and any html present inside it cannot be accessed and that is the reason you are getting NoSuchElementException.
Currently, selenium does not support the shadow DOM automation, so you need to use javascript in this case to scrape the data.
To get the data using javascript, you can use:
JavascriptExecutor js = (JavascriptExecutor) driver;
String return_value = (String) js.execute_script("return document.getElementByXpath('xpath').innerHTML");
References for the shadow DOM:
https://medium.com/rate-engineering/a-guide-to-working-with-shadow-dom-using-selenium-b124992559f
https://www.seleniumeasy.com/selenium-tutorials/accessing-shadow-dom-elements-with-webdriver
I am using google fonts and it generates following error for below link
<link href="http://fonts.googleapis.com/css?family=Lato:100,300,400,700,900,100italic,300italic,400italic,700italic,900italic|Montserrat:700|Merriweather:400italic|Roboto+Condensed|Source+Sans+Pro|Droid+Serif|Open+Sans+Condensed|Oswald|Molengo|PT Sans|Droid Sans')" rel="stylesheet" />
ERROR MESSAGE
Line 35, Column 289: Bad value for attribute href on element link: Illegal character in query: not a URL code point.
…if|Open+Sans+Condensed|Oswald|Molengo|PT Sans|Droid Sans')" rel="stylesheet" />
Syntax of URL:
Any URL. For example: /hello, #canvas, or http://example.org/. Characters should be represented in NFC and spaces should be escaped as %20.
SAMPLE HTML
<html>
<head>
<title>Title</title>
<meta charset="utf-8" />
<meta name="viewport" content="user-scalable=no, width=device-width, initial-scale=1, maximum-scale=1" />
<link href="http://fonts.googleapis.com/css?family=Lato:100,300,400,700,900,100italic,300italic,400italic,700italic,900italic|Montserrat:700|Merriweather:400italic|Roboto+Condensed|Source+Sans+Pro|Droid+Serif|Open+Sans+Condensed|Oswald|Molengo|PT+Sans|Droid+Sans')" rel="stylesheet" />
</head>
<body>
</body>
</html>
UPDATE
This generates error
<link rel="stylesheet" href="http://fonts.googleapis.com/css?family=Roboto%20Condensed|Source%20Sans%20Pro" />
This Works
<link rel="stylesheet" href="http://fonts.googleapis.com/css?family=Lato" />
<link rel="stylesheet" href="http://fonts.googleapis.com/css?family=Roboto%20Condensed" />
When i add | to add multiple fonts it generates error so should i use multiple <link> tag to add fonts or ?
Confused about this is as below links is generate by on Google fonts font use on website
https://www.google.com/fonts#UsePlace:use/Collection:Open+Sans|Roboto:400,700,400italic|Roboto+Condensed:400,300|Lato
Your example code working with JAVASCRIPT NOTATION
LINK and IMPORT may not help to eliminate the VALIDATION error - so please try with JAVASCRIPT notation it works well without any error.
<!DOCTYPE html>
<html>
<head>
<title>Title</title>
<meta charset="utf-8" />
<script type="text/javascript">
WebFontConfig = {google: { families: [ Lato:100,300,400,700,900,100italic,300italic,400italic,700italic,900italic|Montserrat:700|Merriweather:400italic|Roboto+Condensed|Source+Sans+Pro|Droid+Serif|Open+Sans+Condensed|Oswald|Molengo|PT+Sans|Droid+Sans ] }};
(function() {
var wf = document.createElement('script');
wf.src = ('https:' == document.location.protocol ? 'https' : 'http') +
'://ajax.googleapis.com/ajax/libs/webfont/1/webfont.js';
wf.type = 'text/javascript';
wf.async = 'true';
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(wf, s);
})();
</script>
</head>
<body>
</body>
</html>
You will need to substiture & sign with &
<!DOCTYPE html>
<html>
<head>
<title>Title</title>
<meta charset="utf-8" />
<meta name="viewport" content="user-scalable=no, width=device-width, initial-scale=1, maximum-scale=1" />
<link href='http://fonts.googleapis.com/css?family=Open+Sans&subset=latin,cyrillic-ext,greek-ext,greek,vietnamese,latin-ext,cyrillic' rel='stylesheet' type='text/css'>
</head>
<body>
</body>
</html>
You may please use JAVASCRIPT notation for including the fonts from google
<!DOCTYPE html>
<html>
<head>
<title>Title</title>
<meta charset="utf-8" />
<meta name="viewport" content="user-scalable=no, width=device-width, initial-scale=1, maximum-scale=1" />
<script type="text/javascript">
WebFontConfig = {
google: { families: [ 'Open+Sans::cyrillic-ext,latin,greek-ext,greek,vietnamese,latin-ext,cyrillic' ] }
};
(function() {
var wf = document.createElement('script');
wf.src = ('https:' == document.location.protocol ? 'https' : 'http') +
'://ajax.googleapis.com/ajax/libs/webfont/1/webfont.js';
wf.type = 'text/javascript';
wf.async = 'true';
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(wf, s);
})(); </script>
</head>
<body>
</body>
</html>
Few more suggestions
Always include doctype at the top of HTML page
Try the IMPORT and JAVASCRIPT alternatives to include the fonts.
Please use your own google font - to avoid typos I tried with new fonts from google.
The character | is not allowed in the query component (nor anywhere else in a URI). It would have to be percent-encoded with %7C.
So
http://fonts.googleapis.com/css?family=Lato:100,300,400,700,900,100italic,300italic,400italic,700italic,900italic|Montserrat:700|Merriweather:400italic|Roboto+Condensed|Source+Sans+Pro|Droid+Serif|Open+Sans+Condensed|Oswald|Molengo|PT+Sans|Droid+Sans')
should be this URI instead
http://fonts.googleapis.com/css?family=Lato:100,300,400,700,900,100italic,300italic,400italic,700italic,900italic%7CMontserrat:700%7CMerriweather:400italic%7CRoboto+Condensed%7CSource+Sans+Pro%7CDroid+Serif%7COpen+Sans+Condensed%7COswald%7CMolengo%7CPT+Sans%7CDroid+Sans')
There is a space in the string near the end
PT Sans|Droid Sans')"
should be escaped as:
PT%20Sans|Droid%20Sans')"
I'm trying to make a audio player in Chrome with HTML5 and Javascript. But I got an error(file not readable) at the very beginning...
Below is my HTML code:
<head>
<meta charset="utf-8" />
<!-- Always force latest IE rendering engine (even in intranet) & Chrome Frame
Remove this if you use the .htaccess -->
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
<title>index</title>
<meta name="description" content="" />
<meta name="generator" content="Studio 3 http://aptana.com/" />
<meta name="author" content="liuuzyan" />
<!--<meta name="viewport" content="width=device-width; initial-scale=1.0" />-->
<!-- Replace favicon.ico & apple-touch-icon.png in the root of your domain and delete these references -->
<link rel="shortcut icon" href="/favicon.ico" />
<link rel="apple-touch-icon" href="/apple-touch-icon.png" />
<script src="js/jquery-1.6.1.min.js" type="text/javascript"></script>
<script src="js/fun.js" type="text/javascript"></script>
</head>
<body>
<audio id="player" controls="controls">
Please use Firefox/Chrome/IE9 to browse this page.
</audio>
<input type="file" multiple="multiple" id="fileChose" />
</body>
And below is my Javascript code:
$(document).ready( function() {
$('#fileChose').change( function() {
var fileList=this.files;
for(var i=0;i<fileList.length;i++) {
var reader=new FileReader();
reader.onloaded=function(e){
$('#player').attr('src',e.target.result)
}
reader.onerror=function(e){
switch(e.target.error.code) {
case e.target.error.NOT_FOUND_ERR:
alert("file not found");
break;
case e.target.error.NOT_READABLE_ERR:
alert("file not readable");
break;
case e.target.error.ABORT_ERR:
alert("aborted");
break;
default:
alert('generic error?');
}
}
reader.readAsDataURL(fileList[i]);
}
});
});
Can anyone help me with this problem? Thanks a lot!
In Chrome you can add: --allow-file-access-from-files to the Chrome command line to allow local file to be read from local HTML pages.
You can also bundle your code up as Chrome Extension and get local file access that way.
Or upload the HTML files to a Server as already mentioned.