How to scroll sidebar google maps in order to fetch all results on Selenium Java? - google-maps

I have a google map search that I would like to extract with Selenium Java. The code above only gets 7 or 8 results.
public class A {
public static void main(String[] args) throws IOException, InterruptedException {
...
String urlGmaps = "https://www.google.com/maps/search/Air+Conditioning+Contractor+Air+Conditioning+Repair+Service+Contractor+OR+OR+OR+Repair+OR+Service+%22SAN+DIEGO%22/#32.7916817,-117.9672179,8z/data=!3m1!4b1";
driver.get(urlGmaps);
driver.manage().window().maximize();
Thread.sleep(8000);
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10));
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("/html/body/div[3]/div[9]/div[9]/div/div/div[1]/div[2]/div/div[1]/div/div/div[2]/div[1]")));
String scrolling_element_xpath="/html/body/div[3]/div[9]/div[9]/div/div/div[1]/div[2]/div/div[1]/div/div/div[2]/div[1]";
//identify elemet which will appear after scrolling
// identify scrolling element first i.e. the sidebar
WebElement scrolling_element = driver.findElement(By.xpath(scrolling_element_xpath));
// use height of element to determine if need to scroll
long last_height = (long) ((JavascriptExecutor) driver).executeScript("return arguments[0].scrollHeight", scrolling_element);
System.out.println(last_height);
// pause before next scroll, to let page load
int SCROLL_PAUSE_TIME = 2;
int t = 0; // number of times we have scrolled
// Loop the scrolling until cannot scroll anymore
while (true) {
System.out.println(t);
// Scroll down to bottom of whatever is currently loaded
((JavascriptExecutor) driver).executeScript("return arguments[0].scrollHeight", scrolling_element);
t = t+1;
// Wait to load page
try {
Thread.sleep((long) (SCROLL_PAUSE_TIME*1000));
} catch (InterruptedException e) {
e.printStackTrace();
}
// Check if more scrolling required
long new_height = (long) ((JavascriptExecutor) driver).executeScript("return arguments[0].scrollHeight", scrolling_element);
System.out.println(new_height);
if (new_height == last_height) {
break;
}
last_height = new_height;
}
Document doc = Jsoup.parse(driver.getPageSource());
Elements elements = doc.getAllElements();
for(Element e: elements) {System.out.println(e.text());}
}
}
I've increased the amount of time int SCROLL_PAUSE_TIME = 2; and didn't work. I'd like to fetch all results in the sidebar

Related

Added items from a places/photo api fetch to a recyclerview and the view is empty

I have been working on an app that requires me to get information such as a Place Name and a PLace Photo from google PLaces Api and set it into a RecyclerView. I am stuck because I managed to get the code to work with no errors but the RecyclerView is empty. What is wrong with my code?
I am stuck because I don't know where the problem is. When I run the code, all the fetchs work and the tags show up in the Log so I am completely lost. My first thought is that I am displaying the code wrong but then I have no recourse to step forward and change it to something else because I am not sure if it would be better or worse.
This is the Fragment for the RecyclerView Item:
public class VenueList extends Fragment{
ArrayList<VenueItem> vIL = new ArrayList<>();
private PlacesClient placesClient;
private Context contextForPlaces;
place ids for the places I am currently using
String[] clubs = {"ChIJO_uSYKNZwokRAC7RLeB0oZ8", "ChIJAQBEylJYwokRLbnrAchQImk",
"ChIJU_26rfpYwokRTNf2K1-7p8E", "ChIJ38hxfnhZwokRx1HSFLj790w", "ChIJBwnlGrdZwokRpf61pMm860c"
, "ChIJpSIzqrhZwokR1KnVMoVty_g", "ChIJMRV7375ZwokRAfltF6Y-wYw", "ChIJYabdHPhYwokRPmAV8GtM3gs",
"ChIJi2dSjQRZwokRuXUKcv4riVc", "ChIJKaKVI79ZwokRN8WicODOIAw", "ChIJwXI8Fb5ZwokRr4JjG4HxSP8",
"ChIJ6bU_E4ZZwokR2ZDbY_IhhrI"};
#Override
public void onAttach(Context context) {
super.onAttach(context);
contextForPlaces = context;
}
#Nullable
#Override
public View onCreateView(#NonNull LayoutInflater inflater, #Nullable ViewGroup container, #Nullable Bundle savedInstanceState) {
View view = inflater.inflate(R.layout.menu_venue, container, false);
RecyclerView vRV = view.findViewById(R.id.view_venue);
List<Place.Field> placeFields = Arrays.asList(Place.Field.NAME, Place.Field.PHOTO_METADATAS);
if (!Places.isInitialized()) {
Places.initialize(contextForPlaces, "AIzaSyCKGd3fqmtsDklRGMhnkuIy1GS-j6gRBh8");}
placesClient = Places.createClient(contextForPlaces);
vRV.setHasFixedSize(true);
RecyclerView.LayoutManager vLM = new LinearLayoutManager(this.getActivity());
RecyclerView.Adapter vAdapter = new VenueAdapter(vIL);
// run through each photo to make sure it has a place attached to it then insert each photo and place into the vIL
//createBitmap for fetchPhoto
for (String club : clubs) {
FetchPlaceRequest request = FetchPlaceRequest.newInstance(club, placeFields);
placesClient.fetchPlace(request).addOnSuccessListener((response) -> {
Place place = response.getPlace();
PhotoMetadata photoMetadata = place.getPhotoMetadatas().get(0);
String attributions = photoMetadata.getAttributions();
FetchPhotoRequest photoRequest = FetchPhotoRequest.builder(photoMetadata).setMaxHeight(200).build();
placesClient.fetchPhoto(photoRequest).addOnSuccessListener((fetchPhotoResponse) -> {
Bitmap bitmap = fetchPhotoResponse.getBitmap();
vIL.add(new VenueItem(/*Photo*/bitmap, place/*Name*/));
Log.i(TAG, "Photo Should Be Up: ");
}).addOnFailureListener((exception) -> {
if (exception instanceof ApiException) {
ApiException apiException = (ApiException) exception;
int statusCode = apiException.getStatusCode();
// Handle error with given status code.
Log.e(TAG, "Place not found: " + exception.getMessage());
}
});
Log.i(TAG, "Place found: " + place.getName());
}).addOnFailureListener((exception) -> {
if (exception instanceof ApiException) {
ApiException apiException = (ApiException) exception;
int statusCode = apiException.getStatusCode();
// Handle error with given status code.
Log.e(TAG, "Place not found: " + exception.getMessage());
}
});
}
vRV.setLayoutManager(vLM);
vRV.setAdapter(vAdapter);
return view;
}
This is the part of the RecyclerView Adapter I changed. I used to be a getResourse for the image because the image was from the drawable folder
public void onBindViewHolder(#NonNull VenueViewHolder venueViewHolder, int i) {
VenueItem currentItem = vIAL.get(i);
if(currentItem.getVenueImageResource() == null){
venueViewHolder.vIV.setImageResource(R.drawable.ic_android);
}else
venueViewHolder.vIV.setImageBitmap(currentItem.getVenueImageResource());
venueViewHolder.vTV.setText((CharSequence) currentItem.getVenueDescription());
}
The Item itself which I also had to change a bit from its original. I made the string a Place and the int a Bitmap. I thought that would work.
public class VenueItem {
private Bitmap venueImageResource;
private Place venueDescription;
public VenueItem(Bitmap vIR, Place description) {
venueImageResource = vIR;
venueDescription = description;
}
public Bitmap getVenueImageResource() {
return venueImageResource;
}
public Place getVenueDescription() {
return venueDescription;
}
}
I want to be able to request a place name and a photo of the place using the placesClient and precent it in the for of a RecyclerView. I know the place Ids are correct because the Log returns the names of all the places. But they do not show up on the RecyclerView
I figured out the answer myself.
vRV.setLayoutManager(vLM);
vRV.setAdapter(vAdapter);
I just had to put these two lines into the for loop under the list item so that each item could escape before meing erased

ItextPDF Adding Headers and Footers Complex Format

A couple of days ago I asked this question:
Itext PDF How To Add HTML Pre-formatted to PDF, but #bruno-lowagie told me to follow instructions on this existing thread: How To Add HTML Headers And Footers to a Page, I followed carefully the instructions, but found that that approach works for simple html headers and footers like:
<h1>Header Only Line</h1>
or
<h2>Footer Only Line</h2>
But my use case requires to add more complex data in header and footer like images, So I tried with a header that has an img element pointing to an image in the same server like this:
http://localhost:8080/DocGen/resources/images/main_header.jpg
I added some start and end "marks" to see if they got processed so my header was like this:
<p>----Header Start---</p>
<p><img alt="" src="http://localhost:8080/DocGen/resources/images/main_header.jpg" style="height:126px; width:683px" /></p>
<p>--Header End--</p>
But I'm getting an output pdf like this:
Edited: As you can see it doesn't show the image and didn't also show my end mark.
What should I do to successfully add headers and footers with images embedded?
Thanks a lot.
P.S: Sorry for any inconvenience as I am new here, and I hope my question is clear.
EDIT: Code, it's like in the other thread:
public class HtmlHeaderFooter {
private String DEST = null;//"results/events/html_header_footer.pdf";
private String HEADER = null;
private String FOOTER = null;
private float leftMargin;
private float rightMargin;
private float topMargin;
private float bottomMargin;
private Rectangle pageSize = null;
public class HeaderFooter extends PdfPageEventHelper {
protected ElementList header;
protected ElementList footer;
public HeaderFooter() throws IOException {
header = XMLWorkerHelper.parseToElementList(HEADER, null);
footer = XMLWorkerHelper.parseToElementList(FOOTER, null);
}
#Override
public void onEndPage(PdfWriter writer, Document document) {
try {
ColumnText ct = new ColumnText(writer.getDirectContent());
ct.setSimpleColumn(new Rectangle(36, 832, 559, 810));
for (Element e : header) {
System.out.println("Element on header: " + e.toString());
ct.addElement(e);
}
ct.go();
ct.setSimpleColumn(new Rectangle(36, 10, 559, 32));
for (Element e : footer) {
System.out.println("Element on footer: " + e.toString());
ct.addElement(e);
}
ct.go();
} catch (DocumentException de) {
throw new ExceptionConverter(de);
}
}
}
public void createPdfAlt(String outputFile, String inputFile){
Document document = new Document(pageSize, leftMargin, rightMargin, topMargin, bottomMargin);
FileOutputStream outputStream;
try {
outputStream = new FileOutputStream(DEST);
//System.out.println("Doc: " + document.);
PdfWriter writer = PdfWriter.getInstance(document, outputStream);
writer.setPageEvent(new HeaderFooter());
document.open();
PdfContentByte cb = writer.getDirectContent();
// Load existing PDF
PdfReader reader = new PdfReader(new FileInputStream(inputFile));
PdfImportedPage page = writer.getImportedPage(reader, 1);
// document.setPageSize(reader.getPageSize(1));
// Copy first page of existing PDF into output PDF
document.newPage();
cb.addTemplate(page, 0, 0);
document.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (DocumentException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
In my Managed Bean I set Header, Footer, outputfile and so on:
HtmlHeaderFooter htmlHeaderFooter = new HtmlHeaderFooter();
htmlHeaderFooter.setFOOTER(footerContent);
htmlHeaderFooter.setHEADER(headerContent);
//htmlHeaderFooter.setPageSize(xml2pdf.getPageSize());
htmlHeaderFooter.setPageSize(com.itextpdf.text.PageSize.A4);
htmlHeaderFooter.setLeftMargin(template2Export.getLeftMargin());
htmlHeaderFooter.setRightMargin(template2Export.getRightMargin());
htmlHeaderFooter.setTopMargin(template2Export.getSuperiorMargin());
htmlHeaderFooter.setBottomMargin(template2Export.getInferiorMargin());
htmlHeaderFooter.setDEST("salidaConHeaderAndFooter.pdf");
htmlHeaderFooter.createPdfAlt("PDFCompleto1.pdf", "test3.pdf");
EDIT 2: Header should look like this
If you are talking about the html code "as is" it's like this:
<p>----Header Start---</p>
<p><img alt="" src="http://localhost:8080/DocGen/resources/images/main_header.jpg" style="height:126px; width:683px" /></p>
<p>--Header End--</p>
You draw the header here:
ct.setSimpleColumn(new Rectangle(36, 832, 559, 810));
So you allow for about 22pt height (832 - 810) to draw all the header material.
On the other hand your header is expected to display this
<p>----Header Start---</p>
<p><img alt="" src="http://localhost:8080/DocGen/resources/images/main_header.jpg" style="height:126px; width:683px" /></p>
<p>--Header End--</p>
This header requires two paragraphs plus 126px (94.5pt). Thus, it does not fit. Consequently, only the first paragraph (which is the only header content that fits) is drawn.
You might want to start by allowing a lot of space, e.g.
ct.setSimpleColumn(new Rectangle(36, 832, 559, 0));
and then reduce it step by step according to your requirements.

WebDriverException: unknown error: Element is not clickable at point

Performing actions on Chrome using WebDriver
I have a webpage with .xqy extension. where I perform some actions and open the first frame. Then after doing some actions on the first frame I open a second frame and then a third frame.
Now, I need to perform something on the first frame so I close the third frame where selenium's focus is currently on and then the second frame by using the following code:
WebDriver dObjExit = driverObj.switchTo().frame(driverObj.findElement(By.xpath("html/body/div[4]/iframe"))).switchTo().frame(driverObj.findElement(By.xpath("//body[#class='dlg-page']/div[4]/iframe")));
dObjExit.findElement(By.xpath("//p[#class='modal-footer']/button")).click();
Now, I'm just left with the first frame and I use the following code to click upon an element on it:
WebDriver dObjExit1 = driverObj.switchTo().parentFrame();
ObjExit1.findElement(By.xpath("//button[#id='srch-save']")).click();
But Selenium throws the following error:
Exception in thread "main" org.openqa.selenium.WebDriverException:
unknown error: Element is not clickable at point (54, 88). Other
element would receive the click:
Any idea about the resolution? Also tried using Actions class but to no avail.
Use JavascriptExecutor to overcome from this issue:-
WebElement element= driver.findElement(By.xpath("YOUR XPATH"));
JavascriptExecutor executor = (JavascriptExecutor) driver;
executor.executeScript("arguments[0].click();", element);
You can write 2 methods before line with error, to wait for page to be loaded and all ajax are executed, eg:
public static void waitForPageLoaded() {
ExpectedCondition<Boolean> expectationLoad = new
ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver driver) {
return ((JavascriptExecutor) driver).executeScript("return document.readyState").toString().equals("complete");
}
};
try {
Thread.sleep(250);
WebDriverWait waitForLoad = new WebDriverWait(driver, 33);
waitForLoad.until(expectationLoad);
} catch (Throwable error) {
Assert.fail("Timeout waiting for Page Load Request to complete.");
}
}
public static void waitForAjaxFinished() {
ExpectedCondition<Boolean> expectationAjax = new
ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver driver) {
return ((Boolean)((JavascriptExecutor)driver).executeScript("return jQuery.active == 0"));
}
};
try {
Thread.sleep(250);
WebDriverWait waitForAjax = new WebDriverWait(driver, 33);
waitForAjax.until(expectationAjax);
} catch (Throwable error) {
Assert.fail("Timeout waiting for Ajax Finished to complete.");
}
}

Display images in observable collection one after the other as user scrolls in windows phone 8

I have a json array of image URLs added into an observable collection and I want to display the first image on the page such that when a user scrolls horizontally, next or previous images in the array shall display on the screen. Help me achieve this.
Here's how I download the image URLs via json and add them to the observable collection
public event PropertyChangedEventHandler PropertyChanged;
private ObservableCollection<readPageModel> readPages = new ObservableCollection<readPageModel>();
public ObservableCollection<readPageModel> Read_Pages
{
get
{
return readPages;
}
set
{
readPages = value;
if (PropertyChanged != null)
{
PropertyChanged(this, new PropertyChangedEventArgs("Read_Pages"));
}
}
}
public void DownloadData()
{
WebClient client = new WebClient();
client.DownloadStringCompleted += client_DownloadStringCompleted;
client.DownloadStringAsync(new Uri("http://########/mob/ranges/id/3/limit/10/offset/0/r_id/6", UriKind.Absolute));
}
private void client_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
{
try
{
if (!string.IsNullOrEmpty(e.Result))
{
string data = e.Result;
var items = JsonConvert.DeserializeObject<readModel[]>(data);
foreach (var x in items)
{
Read_Pages.Add(x);
}
}
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
You can take scrollviwer in the xaml after this within this scrollviewer take stack panel with horizontal orientation. and then from c# code add image control to this stack panel.
You can have one image control in content panel and implement below code :
public Page()
{
InitializeComponent();
GestureListener gestureListener = GestureService.GetGestureListener(ContentPanel);
gestureListener.DragCompleted += gestureListener_DragCompleted;
//set the initial image to Image control
}
void gestureListener_DragCompleted(object sender, DragCompletedGestureEventArgs e)
{
// User flicked towards left
if (e.HorizontalVelocity < 0)
{
// Load the next image if Downloaded
}
// User flicked towards right
if (e.HorizontalVelocity > 0)
{
// Load the previous image
}
}
you would also needed to have one variable for tracking the index of image to be loaded

How to convert .aspx to Image in asp.net with forms authentication

I have tried this code
public System.Drawing.Bitmap CaptureWebPage(string URL)
{
// create a hidden web browser, which will navigate to the page
System.Windows.Forms.WebBrowser web = new System.Windows.Forms.WebBrowser();
// we don't want scrollbars on our image
web.ScrollBarsEnabled = false;
// don't let any errors shine through
web.ScriptErrorsSuppressed = true;
// let's load up that page!
web.Navigate(URL);
// wait until the page is fully loaded
while (web.ReadyState != WebBrowserReadyState.Complete)
System.Windows.Forms.Application.DoEvents();
System.Threading.Thread.Sleep(1500); // allow time for page scripts to update
// the appearance of the page
// set the size of our web browser to be the same size as the page
int width = web.Document.Body.ScrollRectangle.Width;
int height = web.Document.Body.ScrollRectangle.Height;
web.Width = width;
web.Height = height;
// a bitmap that we will draw to
System.Drawing.Bitmap bmp = new System.Drawing.Bitmap(width, height);
// draw the web browser to the bitmap
web.DrawToBitmap(bmp, new System.Drawing.Rectangle(0, 0, width, height));
return bmp; // return the bitmap for processing
}
protected void btnConvert_Click(object sender, EventArgs e)
{
Bitmap bitmap = new Bitmap(CaptureWebPage(txtUrl.Text));
Response.ContentType = "image/jpeg";
bitmap.Save(Response.OutputStream, ImageFormat.Jpeg);
bitmap.Dispose();
bitmap.Dispose();
Response.End();
}
I am able to capture google.com,but not able to capture images of our web site since it contains form authentication so it redirects me to login page and login page is captured.Please help to pass through authentication.Can we use sessions,cookies or anything with the WebBrowser to valid it Or something.Please suggest.
Try on the intended page on your website :
System.Text.StringBuilder sb = new System.Text.StringBuilder();
System.IO.StringWriter sw = new System.IO.StringWriter(sb);
Page.RenderControl(new HtmlTextWriter(sw));
and then use the HTML in the string builder to render the page like the example below (Reference : Convert HTML string to image) :
using System;
using System.Drawing;
using System.Threading;
using System.Windows.Forms;
class Program
{
static void Main(string[] args)
{
var source = #"
<!DOCTYPE html>
<html>
<body>
<p>An image from W3Schools:</p>
<img
src=""http://www.w3schools.com/images/w3schools_green.jpg""
alt=""W3Schools.com""
width=""104""
height=""142"">
</body>
</html>";
StartBrowser(source);
Console.ReadLine();
}
private static void StartBrowser(string source)
{
var th = new Thread(() =>
{
var webBrowser = new WebBrowser();
webBrowser.ScrollBarsEnabled = false;
webBrowser.DocumentCompleted +=
webBrowser_DocumentCompleted;
webBrowser.DocumentText = source;
Application.Run();
});
th.SetApartmentState(ApartmentState.STA);
th.Start();
}
static void
webBrowser_DocumentCompleted(
object sender,
WebBrowserDocumentCompletedEventArgs e)
{
var webBrowser = (WebBrowser)sender;
using (Bitmap bitmap =
new Bitmap(
webBrowser.Width,
webBrowser.Height))
{
webBrowser
.DrawToBitmap(
bitmap,
new System.Drawing
.Rectangle(0, 0, bitmap.Width, bitmap.Height));
bitmap.Save(#"filename.jpg",
System.Drawing.Imaging.ImageFormat.Jpeg);
}
}
}