Golang Selenium package - connect to selenium server and headless chrome - google-chrome

I'm using the Go selenium package https://godoc.org/github.com/tebeka/selenium
And I'm running headless chrome + selenium-server inside a docker container on localhost:4444
The server seems to be fine since I can access the web console via http://localhost:4444/wd/hub/static/resource/hub.html
But I'm trying to get the "Hello world" example to work with the existing docker container.
This is the example from the GoDocs page for the selenium driver:
// Run some code on play.golang.org and display the result
package main
import (
"fmt"
"time"
"github.com/tebeka/selenium"
)
var code = `
package main
import "fmt"
func main() {
fmt.Println("Hello WebDriver!\n")
}
`
// Errors are ignored for brevity.
func main() {
// Connect to the selenium server
caps := selenium.Capabilities{"browserName": "firefox"}
wd, err := selenium.NewRemote(caps, "http://127.0.0.1:4444")
if err != nil {
fmt.Println(err)
}
defer wd.Quit()
// Get simple playground interface
wd.Get("http://play.golang.org/?simple=1")
// Enter code in textarea
elem, _ := wd.FindElement(selenium.ByCSSSelector, "#code")
elem.Clear()
elem.SendKeys(code)
// Click the run button
btn, _ := wd.FindElement(selenium.ByCSSSelector, "#run")
btn.Click()
// Get the result
div, _ := wd.FindElement(selenium.ByCSSSelector, "#output")
output := ""
// Wait for run to finish
for {
output, _ = div.Text()
if output != "Waiting for remote server..." {
break
}
time.Sleep(time.Millisecond * 100)
}
fmt.Printf("Got: %s\n", output)
}
I tried changing the "browserName" to "chrome" but I get this error:
panic: got content type "text/html", expected "application/json"
goroutine 1 [running]:
main.main()
/home/user01/Code/golang_src/golang_exercises/33_selenium/selenium.go:28 +0x457
exit status 2
I can't find anything in the GoDoc selenium documentation regarding the chrome browser and how to connect to it via the selenium-server.
I would appreciate any hints as to what might be going wrong here.
Update:
It seems that removing the URL address and leaving it empty has fixed the connection problems:
wd, err := selenium.NewRemote(caps, "")
That said, I'm still having issues with the example. Mainly it seems like it connects to the Go Playground website, gets the right elements, but when it comes to sending the input elem.SendKeys(code) it doesn't send it properly and the text box is empty. Resulting in bad output from the Playground:
Got: can't load package: package main:
tmp/sandbox573608783/main.go:1:1: expected 'package', found 'EOF'
Program exited.

try run it in headless mode:
caps := selenium.Capabilities{"browserName": "chrome"}
chromeCaps := chrome.Capabilities{
Path: "",
Args: []string{
"--headless", // <<<
"--no-sandbox",
"--user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0.2 Safari/604.4.7",
},
}
caps.AddChrome(chromeCaps)
wd, err := selenium.NewRemote(caps, fmt.Sprintf("http://127.0.0.1:%d", port))

I also use selenium in Docker, it can run by this:
wd, err := selenium.NewRemote(caps, "http://127.0.0.1:4444/wd/hub")

After some debugging I found out that the issue was due to the fact that my Docker container did not have the X server inside it.
When the selenium package tried to send input it would generate this error message:
unknown error: unknown error: an X display is required for keycode conversions, consider using Xvfb
(Session info: headless chrome=60.0.3095.5)
(Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.4.0-77-generic x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 40 milliseconds
Build info: version: '3.3.1', revision: '5234b32', time: '2017-03-10 09:04:52 -0800'
System info: host: 'e3bf5382c62d', ip: '171.14.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-77-generic', java.version: '1.8.0_121'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities [{applicationCacheEnabled=false, rotatable=false, mobileEmulationEnabled=false, networkConnectionEnabled=false, chrome={chromedriverVersion=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5), userDataDir=/tmp/.org.chromium.Chromium.mFhqlU}, takesHeapSnapshot=true, pageLoadStrategy=normal, databaseEnabled=false, handlesAlerts=true, hasTouchScreen=false, version=60.0.3095.5, platform=LINUX, browserConnectionEnabled=false, nativeEvents=true, acceptSslCerts=true, locationContextEnabled=true, webStorageEnabled=true, browserName=chrome, takesScreenshot=true, javascriptEnabled=true, cssSelectorsEnabled=true, unexpectedAlertBehaviour=}]
Session ID: 681f9de5f91baeaaa3100cf297767a2d
I have not yet installed the X server inside the Docker container but I'm confident that it will fix the error that occurs when sending input to the headless chrome instance via selenium.

Related

Can't connect to RDS mysql in Golang only when ECS

I try to lunch application in ECS.
There is no problem when lunching in my local docker environment.
But it can't accrss to api server in ECS because of rds connection problem.
I use golang in api server and mysql for db.
I call db.go in main.go
func main() {
db := db.NewDatabase(os.Getenv("MYSQL_USER"), os.Getenv("MYSQL_PASSWORD"), os.Getenv("MYSQL_HOST"))
Error occurs when connecting to rds database
func NewDatabase(user, password, host string) *Database {
db, err := sql.Open("mysql", user+":"+password+"#tcp("+host+":3306)/article")
if err != nil {
panic(err.Error())
}
err = db.Ping()
// error occurs here
if err != nil {
panic(err.Error())
}
I deploy it to elastic beanstalk.
I checked environment variables are correctly set.
Here is the full source code:
https://github.com/jpskgc/article
I expect there is no error in elastic beans.
But the actual is not.
I want to know solution for that.
Here is the error log in elastic beanstalk.
-------------------------------------
/var/log/containers/server-4c66c8d1848a-stdouterr.log
-------------------------------------
panic: dial tcp 172.31.26.91:3306: connect: connection timed out
goroutine 1 [running]:
article/api/db.NewDatabase(0xc00002401b, 0x4, 0xc00002a00f, 0xb, 0xc00002800b, 0x3c, 0xdb94f2)
/app/db/db.go:20 +0x3bc
main.main()
/app/main.go:18 +0xee
"connection timed out" means that there are firewall limitation, and you can also check your mysql whitelist, which should has ip of your ECS.

Go MySQL with SSL - unexpected EOF

I'm using the Go MySQL driver as part of a project. Recently, there was a need to enable SSL on the MySQL database I am connecting to. I have two servers that are configured with TLSv1.1. On one server, I am able connect successfully, but on the other I am getting unexpected EOF from the library.
I tried setting MaxIdleConnections to 0 as described in Golang MySQL error - packets.go:33: unexpected EOF, but no luck. Prior to enabling SSL, connections to the server were working. There are other apps not written in Go that are able to connect successfully using the same credentials.
When my app runs, the ping command fails immediately. The MySQL server logs are saying bad handshake. Is there something off with my connection code below, or is there a database setting that could be refusing to connect with skip-verify?
import (
"database/sql"
"io/ioutil"
"strings"
"text/template"
log "github.com/sirupsen/logrus"
// Import runtime for MySQL
mysql "github.com/go-sql-driver/mysql"
)
type DatabaseConnection struct {
Hostname string
Port int
Database string
Username string
Password string
}
func CreateSSLConnection(db *DatabaseConnection) (*sql.DB, error) {
templateString := `{{.Username}}:{{.Password}}#{{.Hostname}}:{{.Port}}/{{.Database}}?tls=skip-verify`
conn, err := connectToDatabase("mysql", templateString, db)
// error: "unexpected EOF"
err = conn.Ping()
return conn, err
}
func connectToDatabase(databaseType string, connectionTemplateString string, db *DatabaseConnection) (*sql.DB, error) {
connectionTemplate := template.Must(template.New("connectionString").Parse(connectionTemplateString))
builder := strings.Builder{}
err := connectionTemplate.Execute(&builder, db)
if err != nil {
log.WithFields(log.Fields{
"template": connectionTemplateString,
"error": err,
}).Error("Failed to create connection string")
return nil, err
}
return sql.Open(databaseType, builder.String())
}

Selenium chromedriver unknown error while running chrome 59 headless with browsermob proxy

I wrote a simple Selenium test (opening a page) of a secured site in a headless mode using Chrome 59 beta version.
I'm getting an the following exception while executing my code. The exception is thrown while initializing the driver
When I rerun my script after commenting the headless option (options.addArguments("headless")) it shows Chrome and running fine but my objective is to run it as headless. Could you please provide your thoughts on resolving my problem?
Exception:
Starting ChromeDriver 2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b) on port 4971
Only local connections are allowed.
Exception in thread "main" org.openqa.selenium.WebDriverException: unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.2 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 60.14 seconds
Build info: version: '3.4.0', revision: 'unknown', time: 'unknown'
Here are the steps:
I'm using Scala with SBT on Mac.
Chrome 59 beta version
ChromeDrive 2.29 release version.
Added the following dependencies
"org.seleniumhq.selenium" % "selenium-chrome-driver" % "3.4.0"
"org.seleniumhq.selenium" % "selenium-support" % "3.4.0"
"net.lightbody.bmp" % "browsermob-core" % "2.1.4"
Scala Code:
val username = "username"
val password = "password"
val domainname = "yoursecuredomain.com"
val browserMobProxyServer = new BrowserMobProxyServer()
browserMobProxyServer.start(0)
browserMobProxyServer.autoAuthorization(domainname, username, password, AuthType.BASIC)
val seleniumProxy = ClientUtil.createSeleniumProxy(browserMobProxyServer)
val options = new ChromeOptions()
options.addArguments("headless")
options.addArguments("--disable-gpu")
options.setBinary("""/Applications/Google Chrome.app/Contents/MacOS/Google Chrome""")
val desiredCapabilities = new DesiredCapabilities()
desiredCapabilities.setCapability(ChromeOptions.CAPABILITY, options)
desiredCapabilities.setCapability(CapabilityType.PROXY, seleniumProxy)
val driver: WebDriver = new ChromeDriver(desiredCapabilities)
val baseUrlString = s"""https://$domainname"""
driver.navigate().to(baseUrlString)
Thread.sleep(3000)
println("title: " + driver.getTitle)
driver.quit()
browserMobProxyServer.abort()
According to the 2.29 webdriver notes page, It says that
----------ChromeDriver v2.29 (2017-04-04)----------
Supports Chrome v56-58
so u have to downgrade your chrome version to be compatible with the latest chrome webdriver.
source:
https://chromedriver.storage.googleapis.com/2.29/notes.txt

Why connect to Google Cloud SQL failed inside Docker container but success outside Docker container?

I've written a piece of code in Golang to test Google Cloud SQL:
package main
import (
"database/sql"
"flag"
"fmt"
_ "github.com/go-sql-driver/mysql"
)
var addr = flag.String("db", "", "The database address")
func main() {
flag.Parse()
db, err := sql.Open("mysql", *addr)
if err != nil {
fmt.Println("mysql open failed: ", err)
return
}
defer db.Close()
err = db.Ping()
if err != nil {
fmt.Println("mysql ping failed: ", err)
return
}
fmt.Println("mysql ping success")
}
I've tested the above code, the output is mysql ping success
Then I want to test this function inside Docker container, the Dockerfile following:
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
ENTRYPOINT ["gcloud-sql-test"]
CMD ["-db=\"user:passwd#tcp(gcloud.sql.ip.address:3306)/database\""]
After building the Docker image, and run the container, I got the following output:
mysql ping failed: Error 1045: Access denied for user '"user'#'my.local.ip.address' (using password: YES)
I've already configured that my local IP can access Google Cloud SQL. I don't know why it doesn't work inside Docker container but works outside Docker container.
Updates, I've fixed the issue because of Dockerfile error
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
CMD ["gcloud-sql-test", "-db=user:passwd#tcp(gcloud.sql.ip.address:3306)/database"]
The main difference is to remove the quotation mark in the Dockerfile:CMD parameter, while you need the quotation mark when you execute the program from Terminal.
Try
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
CMD ["gcloud-sql-test","-db=\"user:passwd#tcp(gcloud.sql.ip.address:3306)/database\""]
CMD and ENTRYPOINT are different commands
I've fixed the issue because of Dockerfile error
FROM golang
ADD . $GOPATH/src/github.com/pdu/gcloud-sql-test
RUN go install github.com/pdu/gcloud-sql-test
CMD ["gcloud-sql-test", "-db=user:passwd#tcp(gcloud.sql.ip.address:3306)/database"]
The main difference is to remove the quotation mark in the Dockerfile:CMD parameter, while you need the quotation mark when you execute the program from Terminal.

TestNG Execution stops executing, when exception occurs, for NoSuchElement Found

Running a TestNG testcase in Eclipse, and getting the values from Ui, using attribute("value").
BUt while execution, if the webelement is not present/Found on UI, I need to take the UIvalue = 00:00.
try {
UIValue = driver.findElement(By.id("ctl00_MainContent_AllowanceGridView_SRow"+cellRosterPerioddd+cellRosterPeriodmm+"_"+PayBucket)).getAttribute("value");
// UIValue = WebelementforUIvalue;
}
catch (NoSuchElementException e) {
UIValue ="00.00";
}
But, When webelement is not Found, the execution stops, I do not want the execution to be stoppedm for the exceptions ?
#Stijn
The execution runs, when the WebElements are found on UI, but it thorws 'No such Element' and stops execution, if the web element is not present on UI, in such cases, need to take the value as '00:00'
# niharika_neo, Here is the Stack Trace below
{code}
Excel PayBucket:JRM1
Excel PayBucket Value:03:40
UI PayBucket:[[ChromeDriver: chrome on XP (0a57a30b08559c4aa0fe4124c5950760)] -> id: ctl00_MainContent_AllowanceGridView_SRow2812_JRM1]
03:52
Failed
Excel PayBucket:VOTPY
Excel PayBucket Value:02:01
UI PayBucket:[[ChromeDriver: chrome on XP (0a57a30b08559c4aa0fe4124c5950760)] -> id: ctl00_MainContent_AllowanceGridView_SRow2102_VOTPY]
03:52
Failed
Excel PayBucket:VOTFA
Excel PayBucket Value:02:00
no such element
(Session info: chrome=39.0.2171.71)
(Driver info: chromedriver=2.10.267521,platform=Windows NT 6.1 SP1 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 25 milliseconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.42.2', revision: '6a6995d', time: '2014-06-03 17:42:03'
System info: host: 'rainmaker2011-5', ip: '192.168.8.70', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.7.0'
Session ID: 0a57a30b08559c4aa0fe4124c5950760
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities [{platform=XP, acceptSslCerts=true, javascriptEnabled=true, browserName=chrome, chrome={userDataDir=C:\Users\LAKSHM~1.PAL\AppData\Local\Temp\scoped_dir14544_10752}, rotatable=false, locationContextEnabled=true, version=39.0.2171.71, takesHeapSnapshot=true, cssSelectorsEnabled=true, databaseEnabled=false, handlesAlerts=true, browserConnectionEnabled=false, nativeEvents=true, webStorageEnabled=true, applicationCacheEnabled=false, takesScreenshot=true}]
PASSED: IntegrationTests
{code}