How to access the #text of an HTML element with Cheerio? - cheerio

I am trying to scape data from this site and it has this structure
<div>
<b>insert bold here</b>
important text
link here
</div>
and I need to access the "important text" which chrome dev tools shows #text.
I've tried to remove the and but I always end up getting a result of undefined when I did the .text() method.
I've tried looping over the children, contents, etc.

here is the code:
const $ = cheerio.load(html);
const result = [...$("div").contents()]
.filter(e => e.type === "text" && $(e).text().trim())
.map(e => $(e).text().trim());
console.log(result[0]);
See how it's working against your test input in cheerio sandbox:
https://scrapeninja.net/cheerio-sandbox?slug=2a11aaa1eb1198fb1fbe55a51b0dd4bc67dcb3db

Related

find url in a string using typescript : anchor tag displayed as plain text

I am trying to find the URL in a given text using typescript and if its the URL, then I have to make it as hyperlink.
I used the code from one of the stackoverflow links.
I have tried the below code :
Urlify(text) {
var exp = /(\b((https?|ftp|file):\/\/|(www))[-A-Z0-9+&##\/%?=~_|!:,.;]*[-A-Z0-9+&##\/%=~_|]*)/ig;
var res = text.replace(exp,"<a href='$1'>$1</a>");
console.log("res : ", res);
return res;
}
However, I am not able to see it as an URL in browser. It looks like below :
Hi the line is <a href='www.google.com.'>www.google.com.</a> End of line.
Instead of
Hi the line is www.google.om. End of line.
Why am I seeing the "a href" in browser ? Am I missing anything ?
Please help.

.innerText of an element is not showing up

I have a div that is contenteditable and grabbing the div using useRef(), which is a reactjs hook.
When I try to display the text inside the contenteditable div, the alert shows nothing but the log shows the text.
Is there something I am missing?
this is just a snippet I created
export default function Input() {
const inputRef = useRef();
const showText = () => {
console.log("text: ", inputRef.current.innerText);
alert("text: ", inputRef.current.innerText);
}
return (
<>
<div ref={inputRef} contentEditable="true" supressContentEditableWarning={true} />
<button onClick={showText}>Show text</button>
</>
)
}
It also does't work when I use it as a value inside an object eg.
const obj = {
text: inputRef.current.innerText
}
I will be thankful if someone can help me understand what is going on here!!
UPDATE
just don't use alert to debug lol.
Is there anything stopping you from getting the innerText using DOM like this-
var innerText = document.getElementById('elementName').innerText
then passing the value to your reactJS?
window.alert only takes a single parameter, so only the first string is shown. If you pass in too many arguments to a javascript function, the extra parameters will simply be ignored. This is different from console.log, which is a variadic function, meaning it will take any number of parameters and display all of them.
Try alert("text: " + inputRef.current.innerText) instead.

How to create a string map "location" and "location Value Text" of this string text?

I developed project using anguler with ngRx framework. I used TypeScript with HTML for developing front-end.My db have saved 'HTML' format texts like below.
"<html><body>A.txt
B.txt
D.txt
www.facebook.com
</body></html>"
This text priviouly , I drectly render in html file using <dev INNERHTML ={{stringText }} \> like wise.
But my project using JXBrowser and as it's configuration , this can't be directly open in default browser clicking just link.
For that work ,I need to take href location as URL and when click it passed to .ts file.
I thought ,it change as like this <a role="button" click='getLink(myText)'> {{getLink(value}} </a>'. so ,create this ,I need that text put a array with contain 'location' and value.Next ,I though ,Iterate that array in HTML file.
I need some expert help to do this ? I am struggle with map above text to such kind of string array (eg :array[hrfeLink][value]). Hope some expert help me.
------------Updated---------------
According to the comment, I will try this way, and I can take the link location. But still couldn't take value.
let parser = new DOMParser();
let doc = parser.parseFromString(info, "text/html");
let x = doc.getElementsByTagName('a');
for (let i = 0; i < x.length ; i++) {
console.log(x[i].getAttribute('href'));
}
What is the value that you want? Is it the anchor text of the link?
We create an interface Link with the properties that we want from each link
interface Link {
location: string;
value: string;
}
Then we create a function that extracts all links from an html string and converts them to an array of Link objects.
function parseLinks( stringHTML: string ): Link[] {
// create a parser object
const parser = new DOMParser();
// turn the string into a Document
const doc = parser.parseFromString( stringHTML, "text/html" );
// get all links
const linkNodes = doc.getElementsByTagName('a');
// convert from HTMLCollection to array to use .map()
const linksArray = [...linkNodes];
// map from HTMLAnchorElement to Link object
return linksArray.map( element => ({
location: element.href,
value: element.innerText,
}))
}
Now you can do whatever with the links from your text
const text = `<html><body>A.txt
B.txt
D.txt
www.facebook.com
</body></html>`;
const links: Link[] = parseLinks( text );
// can use like this
links.map( ({location, value}) => {
// do something here
})
Typescript Playground Link

HTML.TextAreaFor - removing html tags for display only

In an MVC application I have to use #HTML.TextAreaFor to display some text from a database, the trouble is sometimes that text may have HTML tags within it and I can't see a way to remove those for display only.
Is it possible to do this in the view (maybe with CSS?) without having to strip the tags in the controller first?
EDIT
The data coming from the controller contains html tags which I do not want to remove, I just don't want to display them
Normally I would use #HTML.Raw but it has to work in a #HTML.TextAreaFor control.
If you want to decode Html returned from the Controller you can use the following JavaScript method:
This method decodes "Chris&apos; corner" to "Chris' corner".
var decodeEntities = (function () {
// this prevents any overhead from creating the object each time
var element = document.createElement('div');
function decodeHTMLEntities(str) {
if (str && typeof str === 'string') {
// strip script/html tags
str = str.replace(/<script[^>]*>([\S\s]*?)<\/script>/gmi, '');
str = str.replace(/<\/?\w(?:[^"'>]|"[^"]*"|'[^']*')*>/gmi, '');
element.innerHTML = str;
str = element.textContent;
element.textContent = '';
}
return str;
}
return decodeHTMLEntities;
})();
You can do this by using a razor code in your view.
#Html.Raw(HttpUtility.HtmlDecode(Model.Content))
if I set Model.Content to this string "<strong>This is me</strong><button>click</button>", the code above will render it like HTML code and will have a strong text next to a button as an output like the image below:
There's some nice rich text editors libraries like CK Editor, Quill, or TinyMCE that can display HTML while still maintaining the editor capabilities of being a text editor. All of these libraries have capabilities of being read-only as well if that's necessary.
Example from Quill -
Sorted this by changing TextAreaFor toTextBoxFor and setting a formatted value.
#Html.TextBoxFor(x => Model.MyItem, new { #class = "form-control", #required = "true", Value = Regex.Replace(Model.MyItem, "<.*?>", String.Empty) })

How to display openweathermap weather icon

I am using openweathermap to display weather reports. Everything is working fine but there is a problem with the icon.
The JSON response code is:
Array
(
[city] => Array
(
[id] => 1271476
[name] => Guwahati
[coord] => Array
(
[lon] => 91.751
[lat] => 26.1862
)
[country] => IN
[population] => 899094
)
[cod] => 200
[message] => 0.0630711
[cnt] => 1
[list] => Array
(
[0] => Array
(
[dt] => 1495688400
[temp] => Array
(
[day] => 33
[min] => 24.89
[max] => 33.82
[night] => 24.89
[eve] => 30.6
[morn] => 33
)
[pressure] => 1013.02
[humidity] => 90
[weather] => Array
(
[0] => Array
(
[id] => 500
[main] => Rain
[description] => light rain
[icon] => 10d
)
)
[speed] => 3.92
[deg] => 88
[clouds] => 24
[rain] => 2.73
)
)
)
Now how can I display the icon: [weather][0][icon] => 10d?
What is 10d & how can I get the URL of the icon?
Well, I know a way using jQuery.
<div id="icon"><img id="wicon" src="" alt="Weather icon"></div>
At the HTML above you see the unique thing missing is the src attribute, so let's fill it with some jQuery and JavaScript.
You may create a variable to hold the icon code provided by the API like:
var iconcode = a.weather[0].icon;
After it you should concatenate this var iconcode with the url that contains the icons, like:
var iconurl = "http://openweathermap.org/img/w/" + iconcode + ".png";
Finally just change src attribute in the DOM by doing this:
$('#wicon').attr('src', iconurl);
You can get OpenWeatherMap API icons through this link. All you need to do is that moderate the icon id given in bold below in this link. You can change 10d with any icon id that you need.
http://openweathermap.org/img/w/10d.png
For more information, You can read here OpenWeatherMap Icons
So I spent a lot of time solving this problem. This answer is for pure HTML and JavaScript and if you don't want to use jquery.
1- Include the "icons" file in your program: openweatherAPI Icons integration
2- In your index.html :
<div class="weather-icon"><img src="icons/unknown.png" /></div>
3- In your JavScript file(follow these 3 steps in your JS code) :
1st Step: let locationIcon = document.querySelector('.weather-icon');
2nd Step: const {icon} = data.weather[0];
3rd Step(not in code format, as it was making thebackticks part disappear):
locationIcon.innerHTML = <img src="icons/${icon}.png">;
Worked just fine for me.
Happy building.
the src of the icon would be like this:
http://openweathermap.org/img/wn/10d#2x.png
see Weather icons
This code works for me in React Native:
const icon = wInfo.weather[0].icon; // For instance "09d"
<Image source={{ uri: ``http://openweathermap.org/img/w/${icon}.png`` }} />
Thank you all very much! I am a very beginning Flutter programmer and wanted to display the Icon in the Weatherapp, we made on course with Angela Yu.
I did this in Flutter:
String weerImageString;
weerImageString = weatherData['weather'][0]['icon'];
and then were I wanted it to display, I did:
Image.network('http://openweathermap.org/img/w/$weerImageString.png',),
I hope that I can someday helping someone with this. And... if there is an easier way, I would love to hear!
For react, you can use like this:
Step 1: initialize blank state
constructor(){
super()
this.state={
icon:''
}
}
Step 2: api call
async componentDidMount(){
const url = 'http://api.openweathermap.org/data/2.5/'
const key = 'your api key'
const response = await fetch(`${url}weather?q=Guwahati&units=metric&APPID=${key}`)
const data = await response.json() //this will return the json response
const iconName = data.weather[0].icon // this will hold the icon
const iconApi = await fetch('http://openweathermap.org/img/w/' + iconName + '.png')
this.setState({
icon : iconApi.url
})
}
Step 3: Display icon
<div className="weather-icon">
<img style={{width:'70px'}} src= {this.state.icon} />
</div>
Here d refers to day, like n refers to night.
And based on weather status it will change, e.g. 03d for scattered clouds, 01d for clear sky etc.
Here you will get a full list of these icons https://openweathermap.org/weather-conditions#How-to-get-icon-URL
This answer is in reference to the Android,
so after struggling for few hours I finally figured out how to display icons from openweathermap api.
The URL is https://openweathermap.org/img/w/${icon_id}.png
Just put the icon Id you are getting from the API and you will get the result.
Common mistakes which I faced were :
Not using https, as I was using http and it was not working
Also you can get bigger size image by using :
url - https://openweathermap.org/img/wn/${icon_id}#4x.png
icon_id examples : 04d, 10d
working response : https://openweathermap.org/img/wn/04d#4x.png
This is how i solved it. Totally works. no need for JQuery or any of those.
First thing is you realize that the api route is in an array in the weather tab so u have to parse it correctly. use console.log to make sure the get the "10n" kind of result so u know you are getting the right output.
then you use this img tag
<img
alt="icon"
src={http://openweathermap.org/img/w/${icon}.png}
width="120"
height="100"
/>
where icon is the string "04n" that you get from the parsed data.
then it should work perfectly.
Below is my parsed data example. you can see i am geting the temp, humidity, dt_txt is one of the results and icon is the icon
(
{ main: { temp, humidity }, dt_txt, weather: [{ icon }] },
index
)
Firstly, 10d is the icon id that changes according to the forecast data.
I was doing a react weather-App project, and I did the following in my component:
<div>
<img
id="wicon"
src={`http://openweathermap.org/img/wn/${a.weather[0].icon}#2x.png`}
alt="weather icon">
</img>
</div>
This way if the value of the icon change it automatically changes in the Dom.
This worked for me!
create variable to access data between javascript and HTML.
var var1 = document.querySelector('idhere') // you have to use parent class/id
Get icon from JASON
var tempvariable = data['weather'][0]['icon'];
pass link along with html tag to html
var1.innerHTML = "http://openweathermap.org/img/w/" +tempvariable+ ".png' alt='Icon depicting current weather.'>"
or
var1.innerHTML = "http://openweathermap.org/img/w/" +data['weather'][0]['icon']+ ".png' alt='Icon depicting current weather.'>"// step 2 is not required if you use this method.
http://openweathermap.org/img/wn/$weatherData.weather[0].icon#2x.png
To display this on a webpage, you could just add it to the src attribute of an img tag.
This is what the URL for getting the icon will look like....
Where weatherData is the data that you get from the API call you make to the OPENWEATHERMAP. It comes in JSON format. You need to parse.
I think you are asking about IconCode
https://openweathermap.org/
to get converted to an image
For example 02n =>img
So, if thats what you want, then:
You need to use this link
https://openweathermap.org/img/wn/02n#2x.png
Replace the '02n' with the image codes you get as a response
Done
For more image codes information Go to link
const icon = `https://openweathermap.org/img/wn/${weather[0]["icon"]}#2x.png`;
const li = document.createElement("li");
li.classList.add("city");
const markup = `
<figure>
<img class="city-icon" src=${icon} alt=${weather[0]["main"]}>
<figcaption>${weather[0]["description"]}</figcaption>
</figure>
`;
li.innerHTML = markup;
This is what I did and it worked for me. You see, from the response object map through it and with the result access icon as follows
<img src="http://openweathermap.org/img/w/${result.icon}.png" alt="img">
Your html file:
<div class="weather">
<form class="weather-form">
<input type="text" class="city-name">
<input type="submit">
</form>
<img src="" class="iconurl">
</div>
Your JS file:
// hit API
const getWeather = async (cityname) => {
let response = await fetch('https://api.openweathermap.org/data/2.5/weather?q=' + cityname + '&appid=${API KEY}') //you get your api key once you sign up for openweathermap.org
return response.json()
}
// DOM
const weatherContainer = document.querySelector('.weather')
const weatherForm = document.querySelector('.weather-form')
const iconurl = document.querySelector('.iconurl')
// Event Listener
weatherForm.addEventListener('submit', (e) => {
e.preventDefault()
const cityInput = document.querySelector('.city-name')
getWeather(cityInput.value).then(c => {
c.weather.forEach(ww => {
let url = "http://openweathermap.org/img/w/" + ww.icon + ".png"
iconurl.src = url
})
})
})
I had a same problem in Flutter/Dart but not only can't display weather icon but also icon string doesn't have a "http:" in response. I'm working on weatherapi.com.
Here's how You do it in FLUTTER/DART and I'm using Dio package(not http):
In model that You build You need to add:
icon_url = "http:" + json['current']['condition']['icon'] - I did it with "http:" first cause as I said, response doesn't have it in icon url, otherwise just use rest of the code if You're using openweathermap, parameters in [] are just parameters from response, change them, depended on your API. To display icon use Image.network(
nameofyourModel.icon_url). Hope it helps somebody !
ExpressJS:
First Get Icon:
const icon = weather.weather[0].icon;
Second:
iconurl= http://openweathermap.org/img/wn/${icon}.png;
Third:
" alt="">
A little late I must admit. I'm at University right now and I had the same experience as you, where I didn't know how to get the icon to work.
However, a guy at University was kind enough to give me help, which I appreciate very much.
So, in your Javascript, you put:
document.getElementById("icon").src = "https://openweathermap.org/img/wn/" + response.weather[0].icon + "#2x.png";
And in your HTML, you put:
<img id="icon" src = "" alt = "Weather icon">
Hope this helps anyone who was in the same situation as me :)
Thanks!