async def meme(ctx):
content = get("https://meme-api.herokuapp.com/gimme").text
data = json.loads(content,)
meme = discord.Embed(title=f"{data['title']}", Color = discord.Color.random()).set_image(url=f"{data['url']}")
await ctx.reply(embed=meme)
msg = await ctx.reply()
await msg.add_reaction(reaction1)
await msg.add_reaction(reaction2)
um i was trying to make a meme command everything works fine idk why it doesnt add reaction to that message please help me get thru like i want it to react on the embed the bot just sent like on that message
You're not storing the right message
...
await ctx.reply(embed=meme) # <- This is the message you send, put this in the variable instead
msg = await ctx.reply() # <- This will fail because it's empty, you're not sending any content
...
Related
I am using pupteer to select a field input with the name of website. When i select it and then use the click method i keep getting an error message that says "website_input.click is not a function "
But when i try to do this
page.type("input[name='website']","test");
it works fine which i find very weird. Below is the code i am using. Any help would be really appreciated
const website_input = page.$("input[name='website']");
await website_input.click({clickCount: 3});
await website_input.press('Backspace');
The problem might be related to promises. You should await it and then work with the element:
const website_input = await page.$("input[name='website']");
await website_input.click({clickCount: 3});
await website_input.press('Backspace');
That's because page.$() method returns Promise<ElementHandle<T> | null>, not just element handle. You can check the docs.
const context = await browser.createIncognitoBrowserContext();
const [page] = await context.pages();
Why isn't this code working? The array returned by pages() is always empty.
The only way I know to actually open a page is to call context.newPage(), which means having an about:blank empty page. I would like to avoid that.
I have retrieved all elements of the class (".col.double.societe.u2d.hasLogo" ), i would like to loop each of them to retrieve the class ('.adresse') texts
My code is:
const societeHandles = await page.$$('.col.double.societe.u2d.hasLogo');
for(const societeHandle of societeHandles){
const adresse = await societeHandle.$eval(".adresse");
But i got run error Error: failed to find element matching selector ".adresse"
the classe is there:
<p class="adresse">
325 rue Pasteur
<br>33200 Bordeaux
<br>Gironde
<br>Aquitaine
</p>
Am i allow to do it ? or is it a syntax error ?
Thanks
The puppeteer can't locate the address element because of the selector you chose to locate all of them, it's probably too specific. Try .col.double.societe.
require('puppeteer').launch().then(async browser => {
const page = await browser.newPage();
await page.goto('http://www.dollmedia-btp.com/annuaire/aquitaine/gironde/bordeaux');
const societeHandles = await page.$$('.col.double.societe');
// Have we got any addresses?
console.log("Total count: " + societeHandles.length)
for(const societeHandle of societeHandles){
const adresse = await societeHandle.$eval(".adresse", el => el.textContent.trim());
console.log(adresse + "\n")
}
await browser.close();
});
It is also a handy practice to log if anything found during the request at all, so I logged the number of found nodes.
Also note the correct use of handle.$eval: you pass the function a selector and also a function to execute in the browser context. (In my example I just return the text from found elements)
I am trying to write a test (using jest-puppeteer) for an input in my React application that handles autocomplete or copy/pasted strings in a unique way.
I was hoping by using Puppeteer, I could paste text into the input and then validate that the page is updated correctly. Unfortunately, I can't find any working example of how to do this.
I've tried using page.keyboard to simulate CMD+C & CMD+V but it does not appear that these sorts of commands work in Puppeteer.
I've also tried using a library such as clipboardy to write and read to the OS clipboard. While clipboardy does work for write (copy), it seems read (paste) does not affect the page run by Puppeteer.
I have successfully copied the text using a variety of methods but have no way to paste into the input. I've validated this assumption by adding event listeners for "copy" and "paste" to the document. The "copy" events fire, but no method has resulted in the "paste" event firing.
Here are a few approaches I have tried:
await clipboardy.write('1234'); // writes "1234" to clipboard
await page.focus("input");
await clipboardy.read(); // Supposedly pastes from clipboard
// assert input has updated
await clipboardy.write('1234');
await page.focus("input");
await page.keyboard.down('Meta');
await page.keyboard.press('KeyV');
await page.keyboard.up('Meta');
// assert input has updated
await page.evaluate(() => {
const input = document.createElement('input');
document.body.appendChild(input);
input.value = '1234';
input.focus();
input.select();
document.execCommand('copy');
document.body.removeChild(input);
});
wait page.focus("input");
await page.keyboard.down('Meta');
await page.keyboard.press('KeyV');
await page.keyboard.up('Meta');
I think the only missing piece here is pasting the text; but how do you paste text using Puppeteer?
This works for me with clipboardy, but not when I launch it in headless :
await clipboardy.write('foo')
const input= await puppeteerPage.$(inputSelector)
await input.focus()
await puppeteerPage.keyboard.down('Control')
await puppeteerPage.keyboard.press('V')
await puppeteerPage.keyboard.up('Control')
If you make it works in headless tell me.
I tried it the clipBoard API too but I couldn t make it compile:
const browser = await getBrowser()
const context = browser.defaultBrowserContext();
// set clipBoard API permissions
context.clearPermissionOverrides()
context.overridePermissions(config.APPLICATION_URL, ['clipboard-write'])
puppeteerPage = await browser.newPage()
await puppeteerPage.evaluate((textToCopy) =>{
navigator.clipboard.writeText(textToCopy)
}, 'bar')
const input= await puppeteerPage.$(inputSelector)
await input.focus()
await puppeteerPage.evaluate(() =>{
navigator.clipboard.readText()
})
I came up with a funny workaround how to paste a long text into React component in a way that the change would be registered by the component and it would not take insanely long time to type as it normally does with type command:
For text copying I use approach from Puppeteer docs (assume I want to select text from first 2 paragraphs on a page for example). I assume you already know how to set the permissions for clipboard reading and writing (for example one of the answers above shows how to do it).
const fromJSHandle = await page.evaluateHandle(() =>Array.from(document.querySelectorAll('p'))[0])
const toJSHandle = await page.evaluateHandle(() =>Array.from(document.querySelectorAll('p'))[1])
// from puppeteer docs
await page.evaluate((from, to) => {
const selection = from.getRootNode().getSelection();
const range = document.createRange();
range.setStartBefore(from);
range.setEndAfter(to);
selection.removeAllRanges();
selection.addRange(range);
}, fromJSHandle, toJSHandle);
await page.bringToFront();
await page.evaluate(() => {
document.execCommand('copy') // Copy the selected content to the clipboard
return navigator.clipboard.readText() // Obtain the content of the clipboard as a string
})
This approach does not work for pasting (on Mac at least): document.execCommand('paste')
So for pasting I use this:
await page.$eval('#myInput', (el, value) =>{ el.value = value }, myLongText)
await page.type(`#myInput`,' ') // this assumes your app trims the input value in the end so the whitespace doesn't bother you
Without the last typing step (the white space) React does not register change/input event. So after submitting the form (of which the input is part of for example) the input value would still be "".
This is where typing the whitespace comes in - it triggers the change event and we can submit the form.
It seems that one needs to develop quite a bit of ingenuity with Puppeteer to figure out how to work around all the limitations and maintaining some level of developer comfort at the same time.
Im trying to scrape some data from Twitter using BeautifulSoup as a part of a project. To scrape the ‘following’ section I need to first login, so I tried doing so using MechanicalSoup. I know the login is successful as I received an email saying so, but when I go to a different page in the same website to scrape data it again redirects me to the login page.
import mechanicalsoup
browser = mechanicalsoup.StatefulBrowser(soup_config={'features': 'lxml'},
raise_on_404=True,
user_agent='MyBot/0.1: mysite.example.com/bot_info',)
login_page = browser.get("https://twitter.com/login")
login_form = login_page.soup.findAll("form")
login_form = login_form[2]
login_form.find("input", {"name": "session[username_or_email]"})["value"] = "puturusername"
login_form.find("input", {"name": "session[password]"})["value"] = "puturpassword"
login_response = browser.submit(login_form, login_page.url)
login_response.soup()
This sent me a successful login email, upon which I tried:
page_stml = browser.open('https://twitter.com/MKBHD/following').text
page_soup = soup(page_html,"html.parser")
page_soup
I received the page containing https://twitter.com/login?redirect_after_login=%2FMKBHD%2Ffollowing& instead of the actual ‘following’ page.
And if I try the code given below instead of 'browser.open('https://twitter.com/MKBHD/following').text':
# verify we are now logged in
page = browser.get_current_page()
print(page)
messages = page.find("div", class_="flash-messages")
if messages:
print(messages.text)
assert page.select(".logout-form")
print(page.title.text)
# verify we remain logged in (thanks to cookies) as we browse the rest of
# the site
page3 = browser.open("https://github.com/MechanicalSoup/MechanicalSoup")
assert page3.soup.select(".logout-form”)
I get the output:
----> 4 messages = page.find("div", class_="flash-messages")
AttributeError: 'NoneType' object has no attribute ‘find’
update:
the login_response.soup()
gives me the following:
</style>, <body>
<noscript>
<center>If you’re not redirected soon, please use this link.</center>
</noscript>
<script nonce="O1gf092z/sXmKkH64mLOzQ==">
document.cookie = "app_shell_visited=1;path=/;max-age=5";
location.replace(location.href.split("#")[0]);
</script>
</body>, <noscript>
<center>If you’re not redirected soon, please use this link.</center>
</noscript>, <center>If you’re not redirected soon, please use this link.</center>, use this link, <script nonce="O1gf092z/sXmKkH64mLOzQ==">
document.cookie = "app_shell_visited=1;path=/;max-age=5";
location.replace(location.href.split("#")[0]);
</script>]
To avoid to get the redirection page, you can use StatefulBrowser() object instead of Browser().
I wrote a short post about it : https://piratefache.ch/python-3-mechanize-and-beautifulsoup
import mechanicalsoup
if __name__ == "__main__":
URL = "https://twitter.com/login"
LOGIN = "your_login"
PASSWORD = "your_password"
TWITTER_NAME = "displayed_name" # Displayed username on Twitter
# Create a browser object
browser = mechanicalsoup.StatefulBrowser()
# request Twitter login page
browser.open(URL)
# we grab the login form
browser.select_form('form[action="https://twitter.com/sessions"]')
# print form inputs
browser.get_current_form().print_summary()
# specify username and password
browser["session[username_or_email]"] = LOGIN
browser["session[password]"] = PASSWORD
# submit form
response = browser.submit_selected()
# get current page output
response_after_login = browser.get_current_page()
# verify we are now logged in ( get img alt element containing username )
# if you found a better way to check, let me know. Since twitter generate dynamically all theirs classes, its
# pretty complicated to get better information
user_element = response_after_login.select_one("img[alt="+TWITTER_NAME+"]")
# if username is in the img field, it means the user is successfully connected
if TWITTER_NAME in str(user_element):
print("You're connected as " + TWITTER_NAME)
else:
print("Not connected")
Sources:
StatefulBrowser
Example with StatefulBrowser