I need to scrape 29 images of this hotel. With the code below, the output is each link separately every time I run the cell. Even though I used links.append(), I need to re-run the cell in order to get another image.
r = rq.get("https://uk.hotels.com/ho177101/?q-check-out=2020-04-18&FPQ=3&q-check-in=2020-04-17&WOE=6&WOD=5&q-room-0-children=0&pa=1&tab=description&JHR=2&q-room-0-adults=2&YGF=2&MGT=1&ZSX=0&SYE=3#:WO")
soup = BeautifulSoup(r.text, "html.parser")
links = []
x = soup.select('img[src^="https://exp.cdn-hotels.com/hotels/1000000/560000/558400/558353"]')
for img in x:
links.append(img['src'])
#os.mkdir("hotel_photos")
for l in links:
print(l)
Thank you in advance!
Try this:
links = []
x = soup.select('a[href^="https://exp.cdn-hotels.com/hotels/1000000/560000/558400/558353"]')
for img in x:
links.append(img['href'])
But, this will only work for that specific link, if you need a code that would scrape any link you provide, because this "https://exp.cdn-hotels.com/hotels/1000000/560000/558400/558353" changes depending on the hotel, so this would be a better approach:
x = soup.select("li[id^='thumb-']")
for i in x:
links.append((next(i.children, None)["href"]))
for l in links:
print(l)
PS: If you need after that to download the pictures, make sure to replace "&w=82&h=82" with "&w=773&h=530" to match the picture displayed.
Related
I have four goals:
Connect to a Postgresql database and pull some data
Gloss up a table with some colour and formatting
Include an image (company logo) above it
Export as PDF
1 and 2 are easy enough and 4 seems possible even if not convenient, but I don't think R was designed to add and position images. I've attached some sample code of how I envision creating the table, and then a mockup of what I think the final version might look like. Can anyone advise on the best way to accomplish this?
Sample data:
data(mtcars)
df <- head(mtcars)
HTML approach: flexible and portable to other apps
library(tableHTML)
html_table <- df %>%
tableHTML(rownames = FALSE, border = 0) %>%
add_css_row(css = list(c('font-family', 'text-align'), c('sans-serif', 'center'))) %>%
add_css_header(css = list(c('background-color', 'color'), c('#173ACC', 'white')), headers = 1:ncol(df))
Grob approach: Creating a ggplot-like image. I've seen recommendations to use grid.arrange to place an image on top and export as a PDF
library(ggpubr)
tbody.style = tbody_style(color = "black",
fill = "white", hjust=1, x=0.9)
grob_table <- ggtexttable(df, rows = NULL,
theme = ttheme(
colnames.style = colnames_style(color = "white", fill = "#173ACC"),
tbody.style = tbody.style
)
)
grid.arrange(table_image)
You are almost there. You just need to import your image (could be png, jpeg or svg) then pass it to grid::rasterGrob. Use the options in rasterGrob to adjust size etc. Then pass your grob table to gridExtra::grid.arrange
logo_imported <- png::readPNG(system.file("img", "Rlogo.png", package="png"), TRUE)
lg <- grid::rasterGrob(logo_imported)
gridExtra::grid.arrange(lg, grob_table)
You can then either render this to pdf by adding it to an rmarkdown report (probably best), or you can save directly to pdf via
gridExtra::grid.arrange(lg, grob_table)
pdf(file = "My Plot.pdf",
width = 4, # The width of the plot in inches
height = 4)
I would like to modify the format of my Hoverinfo in Plotly Dash:
Im working with plotly and not plotly express for some reasons.
My Code:
fig.add_trace(go.Bar(x = summe2[monat_week], y = summe2['Umsatz'], name='Umsatz', offsetgroup = 1, marker_color = 'blue'), secondary_y = False)
fig.update_traces(hovertemplate = "Umsatz:%{y:20,.2f}'+ 'X%{monat_week}: %{x}", secondary_y=False)
At the beginning when i hovered on on my chart it displayed 100K instead of 100,000
With hovertemplate = "Umsatz<:%{y:.2f} I fixed it and now im getting 100000.00 as intended but I have no clue how to set an group delimiter 3 that im getting the 100000.00 displayed
like 100,000.00. I found an older post from here but its not working from me
Link: Plotly: How to format numeric yaxis and hover text?
I Wrote the code for the formatting as in the previous link but nothing changed and maybe you could tell me if i can change the formatting to EU like 100.000,00 instead of 100,000.00
via Dash fig.update_traces(hovertemplate = 'Umsatz: %{y: 20,.2f}') should work to display it like 100,000.00 but there is no difference to
%{y: .2f} Display: 100000.00
Thank you in advance
Greetings LittleStudent
I'm trying to define a function that will create lists from multiple text files and print a random element from one of the weighted lists. I've managed to get the function to work with random.choice for a single list.
enter code here
def test_rollitems():
my_commons = open('common.txt')
all_common_lines = my_commons.readlines()
common = []
for i in all_common_lines:
common.append(i)
y = random.choice(common)
print(y)
When I tried adding a second list to the function it wouldn't work and my program just closes when the function is called.
enter code here
def Improved_rollitem():
#create the lists from the files#
my_commons = open('common.txt')
all_common_lines= my_commons.readlines()
common = []
for i in all_common_lines:
common.append(i)
my_uncommons = open('uncommon.txt')
all_uncommon_lines =my_uncommons.readlines()
uncommon =[]
for i in all_uncommon_lines:
uncommon.apend(i)
y = random.choices([common,uncommon], [80,20])
print(y)
Can anyone offer any insight into what I'm doing wrong or missing ?
Nevermind. I figured this out on my own! Was having issues with Geany so I installed Pycharm and was able to work through the issue. Correct code is:
enter code here
def Improved_rollitem():
#create the lists from the files#
my_commons = open('common.txt')
all_common_lines= my_commons.readlines()
common = []
for i in all_common_lines:
common.append(i)
my_uncommons = open('uncommon.txt')
all_uncommon_lines =my_uncommons.readlines()
uncommon =[]
for i in all_uncommon_lines:
uncommon.append(i)
y = random.choices([common,uncommon], [.8,.20])
if y == [common]:
for i in [common]:
print(random.choice(i))
if y == [uncommon]:
for i in [uncommon]:
print(random.choice(i))
If there's a better way to do something like this, it would certainly be cool to know though.
I have a shiny application that allows my user to explore a dataset. The idea is that the user explores the dataset, and any interesting things the user finds he will share with his client via email. I don't know in advance how many things the user will find interesting. So, next to each table or chart I have an "add this item to the report" button, which isolates the current view and adds it to a reactiveValues list.
Now, what I want to do is the following:
Loop through all the items in the reactiveValues list,
Generate some explanatory text describing the item (This text should preferably be formatted HTML/markdown, rather than code comments)
Display the item
Capture the output of this loop as HTML
Display this HTML in Shiny as a preview
write this HTML to a file
knitr seems to do exactly the reverse of what I want - where knitr allows me to add interactive shiny components in an otherwise static document, I want to generate HTML in shiny (maybe using knitr, I don't know) based on static values the user has created.
I've constructed a minimum not-working example below to try to indicate what I would like to do. It doesn't work, it's just for demonstration purposes.
ui = shinyUI(fluidPage(
title = "Report generator",
sidebarLayout(
sidebarPanel(textInput("numberinput","Add a number", value = 5),
actionButton("addthischart", "Add the current chart to the report")),
mainPanel(plotOutput("numberplot"),
htmlOutput("report"))
)
))
server = shinyServer(function(input, output, session){
#ensure I can plot
library(ggplot2)
#make a holder for my stored data
values = reactiveValues()
values$Report = list()
#generate the plot
myplot = reactive({
df = data.frame(x = 1:input$numberinput, y = (1:input$numberinput)^2)
p = ggplot(df, aes(x = x, y = y)) + geom_line()
return(p)
})
#display the plot
output$numberplot = renderPlot(myplot())
# when the user clicks a button, add the current plot to the report
observeEvent(input$addthischart,{
chart = isolate(myplot)
isolate(values$Report <- c(values$Report,list(chart)))
})
#make the report
myreport = eventReactive(input$addthischart,{
reporthtml = character()
if(length(values$Report)>0){
for(i in 1:length(values$Report)){
explanatorytext = tags$h3(paste(" Now please direct your attention to plot number",i,"\n"))
chart = values$Report[[i]]()
theplot = HTML(chart) # this does not work - this is the crux of my question - what should i do here?
reporthtml = c(reporthtml, explanatorytext, theplot)
# ideally, at this point, the output would be an HTML file that includes some header text, as well as a plot
# I made this example to show what I hoped would work. Clearly, it does not work. I'm asking for advice on an alternative approach.
}
}
return(reporthtml)
})
# display the report
output$report = renderUI({
myreport()
})
})
runApp(list(ui = ui, server = server))
You could capture the HTML of your page using html2canvas and then save the captured portion of the DOM as a image using this answer, this way your client can embed this in any HTML document without worrying about the origin of the page contents
I am a biologist trying to do computer science for research, so I may be a bit naïve. But I would like to a make a table containing information from a data frame, with a hyperlink in one of the columns. I imagine this needs to be an html document (?). I found this post this post describing how to put a hyperlink into a data frame and write it as an HTML file using googleVis. I would like to use this approach (it is the only one I know and seems to work well) except I would like to replace the actual URL with a description. The real motivation being that I would like to include many of these hyperlinks, and the links have long addresses which is difficult to read.
To be verbose, I essentially want to do what I did here where we read 'here' but 'here' points to
http:// stackoverflow.com/questions/8030208/exporting-table-in-r-to-html-with-hyperlinks
From your previous question, you can have another list which contains the titles of the URL's:
url=c('http://nytimes.com', 'http://cnn.com', 'http://www.weather.gov'))
urlTitles=c('NY Times', 'CNN', 'Weather'))
foo <- transform(foo, url = paste('<a href = ', shQuote(url), '>', urlTitles, '</a>'))
x = gvisTable(foo, options = list(allowHTML = TRUE))
plot(x)
Building on Jack's answer but consolidating from different threads:
library(googleVis)
library(R2HTML)
url <- c('http://nytimes.com', 'http://cnn.com', 'http://www.weather.gov')
urlTitles <- c('NY Times', 'CNN', 'Weather')
foo <- data.frame(a=c(1,2,3), b=c(4,5,6), url=url)
foo <- transform(foo, url = paste('<a href = ', shQuote(url), '>', urlTitles, '</a>'))
x <- gvisTable(foo, options = list(allowHTML = TRUE))
plot(x)