Sikuli: I am not able to click on an Element present in a PopUp window - sikuli

I am new to the Sikuli API and I am using the Sikuli-java-jar file. I want to successfully click on a desktop element using the screen and pattern classes.
So I was trying to create an automation script to install software.
I am able to successfully launch the installer but not able to click on the Next Button present in the pop-up window.
I am not getting any error, it is just that clicking on the image fails.
appInstaller("E:\\Sikulimages\\tc.png");
appInstallers("E:\\Sikulimages\\next.png");
public static void appInstaller(String path) throws FindFailed{
s=new Screen();
img=new Pattern(path);
s.exists(path);
s.wait(img,2000);
s.doubleClick(img);
}
public static void appInstallers(String path) throws FindFailed, InterruptedException{
s=new Screen();
img=new Pattern(path);
s.click(img);
}

I think that the reason is that default Similarity (0.7) is insufficient for small buttons and text.
Sikuli has public class Settings ,
which hosts public static double MinSimilarity = 0.7;
That value is good enough for most image recognition, but it fails for small texts. To get small text clicked, you need to raise similarity for particular pattern, or, like I do, make Settings.MinSimilarity = 0.9;
Sometimes even 0.9 is not enough to recognize small text, well, then try 0.95 , it is usually what helps to pinpoint even smallest texts.

In addition to RPWheeler's answer, if there is something right next to your target image that will appear the same way every time, you can also take a bigger screen clip and then tell Sikuli which part of that bigger image to click on.
For example: if your "Next" button is part of a row of buttons, take the screen clip to include the whole row. Then, in the IDE, double click on the image, and go to the "Target Offset" tab. Here, you'll see an example of your image, and and you click on the part of the image you want Sikuli to click on. -- the default is always the center of the rectangle, but you can change it with the target offset.
You can accomplish the same thing in code like this
t = find(yourImage).targetOffset(dx,dy)
click(t)
where dx and dy are a positive or negative number of pixels away from the center point of the rectangle.

For installing application you need admin rights. So For running any task that require admin rights you need to run sikuli in admin mode. So right click on runSikulixcmd.bat and select Run as administrator for launching the Sikuli and then run the test.
If you are running the test from command prompt run it in admin command prompt.

Related

Libgdx, Setting the full screen resolution during runtime causes application to render at the wrong size

I am only using the desktop Application, no mobile.
I am experimenting with letting the user set the screen resolution during run time. I give him the Display Modes available and he applies one. This part actually works. The problem occurs when i save this mode and try to set this display mode the next time they launch the game.
I am using preferences to store the mode the user selected. I am unable to access preferences before the Create method in my Game class, or in the DesktopLauncher Object, where you normally set up the config file and pass it into the application. So my DesktopLauncher looks like this.
val config = Lwjgl3ApplicationConfiguration()
config.setFullscreenMode(Lwjgl3ApplicationConfiguration.getDisplayMode())
Lwjgl3Application(MainGame(), config)
I use the current screen resolution on the creation of the application. Then in my Create method in my MainGame class i get the mode they set from preferences and i set it like so...
override fun create() {
var modes = Gdx.graphics.displayModes.toList()
val mode = Gdx.graphics.displayMode
val preference: Preferences = Gdx.app.getPreferences("screenPreference")
val screenWidth = preference.getInteger("width", mode.width)
val screenHeight = preference.getInteger("height", mode.height)
val refreshRate = preference.getInteger("refreshRate", mode.refreshRate)
modes = modes.filter { it.width == screenWidth }
modes = modes.filter { it.height == screenHeight }
modes = modes.filter { it.refreshRate == refreshRate }
if (modes.isNotEmpty()) {
Gdx.graphics.setFullscreenMode(modes[0])
}
....
}
To summarize i get the list of modes, i pull from preferences what was set last, and i filter the list according to what was in preferences. This should leave one item left in the list and i apply it. If for some reason the list is empty, then i don't set it, or there is no preference set i just apply the current mode again.
This is where the weird stuff happens. I have checked all the numbers when creating my screens and cameras, and they are all correct. I do receive the correct resolution, but the application doesn't render correctly. Below are a couple examples of what happens.
In the first image you see the bounds of the application to the screen. My application only renders in the bottom corner, and the rest is black. What happened to achieve this effect is i started the application with a smaller resolution than my native resolution, so 1280x1024, then in my create method i set the application full screen mode to 1920x1080 before building the rest of my application. I have checked my cameras and my viewports, and they all have the resolution 1920x1080, but the image is not filling the entire screen.
And a second.
This one is what happens when i reverse the settings. So i start at native resolution 1920x1080, and in my create method i set it to 1280x1024, again before creating the rest of my application. This gives me black bars on both sides of the image like id expect, but the application is HUGE, and only a portion of it fits in the window, the rest goes out of bounds, as depicted by the dotted lines.
It will remain like this the entire time, unless i change the resolution while the application is running, it will then correct itself for the rest of the applications life.
I am confounded by this effect i am getting, and am looking for an answer as to why, or how to fix it.
I found the issue that was causing the image to render incorrectly. I was setting the display mode in the create() function in my main game class. This function is not run on the rendering thread, and you do not want to use Gdx.graphics on anything other than the rendering thread, as described in the libgdx wiki https://github.com/libgdx/libgdx/wiki/Threading
There is a function where you can pass in a lambda to be run on the rendering thread.
Gdx.app.postRunnable {
Gdx.graphics.setFullscreenMode(Gdx.graphics.getDisplayMode(modes[0]))
}
After passing that into postRunnable the game renders correctly on launch.

Can Sikuli be used for web testing?

I was working with Sikuli for desktop application like notepad,
but want to know like can I open new tab in browser using Sikuli?
Yes, using Sikuli you can also automate browser. Take images of required web elements (Open new tab button in your case) and stimulate click action using Sikuli APIs.
Yes, you can use Sikuli for web testing.
In this code example, you can use sikuli app to: open browser --> New tab --> Give a like on a webpage (UTAD in this case):
click("mozilla_icon.png")
click("nova_aba.png")
click("endereco.png")
paste("www.utad.pt")
type(Key.ENTER)
wait(8)
click("like_btt.png")
wait(5)
Images of sikuli code:
yes you can use sikuli for performing open tab first take the image of new tab control like in case of FF https://drive.google.com/file/d/0B09BIsDTY_AuZFpMWko1U3BFc0E/view?usp=sharing .Save this image on you local machine. But make sure that image which is used for reference is visible on screen.Call the function mentioned below by and give it Absolute path to above mentioned image.
//provide absolute path in img
public void click_Image(String img)
{
s = new DesktopScreenRegion();
target = new ImageTarget(new File(img));
r = s.find(target);
// Create a mouse object
mouse = new DesktopMouse();
// Use the mouse object to click on the center of the target region
mouse.click(r.getCenter());
}

Share Target and JumpListItemBackgroundConverter

I've added the Share Target declaration to my app for the data format WebLink and everything works as expected. However, when I add a JumpListItemBackgroundConverter or JumpListItemForegroundConverter anywhere in the app, the app hangs on the splash screen when you enter the app using the Share from IE. No exception, no crash, the debugger doesn't even stop. All I get is a cryptic error in the output window, "The program '...' has exited with code -1073741819 (0xc0000005) 'Access violation'." The documentation for those converters say they're fine with universal apps, just that they've been moved to a different namespace. Has anyone been able to get these two things to work in the same app? If so, where did I go wrong? Or is there something better than those two converters to get the LongListSelector look and feel?
Steps to reproduce:
Create a new universal app. I chose hub.
Add a share target of format WebLink to the appxmanifest declarations.
Add a new page to point the share contract to.
Add the OnShareTargetActivated code to app.xaml.cs to open the new page. See code below
Add a JumpListItemBackgroundConverter to the resources of the main page of the app. You don't need to apply it to anything, just declaring it is enough to break the sharing.
Go to IE and share a link. It should hang on the splash screen.
Code for app.xaml.cs:
protected override async void OnShareTargetActivated(ShareTargetActivatedEventArgs args)
{
// Replace SharePage with the name of the share target page
var rootFrame = new Frame();
rootFrame.Navigate(typeof(SharePage), args.ShareOperation);
Window.Current.Content = rootFrame;
Window.Current.Activate();
}
It turns out this is a bug in the emulator. It works if you test on a physical device.
MSDN Forum - JumpListItemBackgroundConverter and Share Target in Windows Phone 8.1

Flash CS3 (AS3) is giving my graphics (buttons and movieclips) outlines

So, I'm pretty much a beginner in Flash and Actionscript (using AS3, as I said in the title), and I'm trying to make a basic escape the room game. I haven't gotten far, and right now that's because every time I test my game (or publish preview it) the graphics get this annoying outline. Here it is when tested: http://i305.photobucket.com/albums/nn228/chokingondrama/flash.png
Every outline corresponds to some object present in the game, most of which have an alpha component of 0 since they're on different sides of the room. This didn't happen before, but once I added the code that allowed the player to change their view with the arrow (each viewpoint/wall is a different frame) these appeared.
It's a little different when published to HTML, basically it just gives each image a white background: http://i305.photobucket.com/albums/nn228/chokingondrama/html.png
Also, it would be nice if somebody could give me advice on how to make sure importing to flash won't result in lower quality.
Thanks in advance. If needed, I'll post any part of the code.
Some tips:
Don't set alpha to 0, instead use the visible property, setting movieclip.visible = false will make it a lot more efficient.
As for the importing and quality, after you import to stage or library, bring up the library (ctrl + l), and right click on the file you imported, go to properties. If it's an image, set compression to lossless, and allow smoothing.
For audio, go to file-> publish settings, and change audio stream and audio event (whichever you might use) to 128kbps.
As for your main question, I need more info, if you want you can post your source. It might be because of how you are placing your graphics on the stage.
For each of your MovieClips in question:
Try disabling button mode and see if the rectangles go away.
movieClipName.buttonMode = false;
If that doesn't help, or you really want button mode, try setting
movieClipName.tabEnabled = false;
There's a chance that since you added keyboard interaction each of your MovieClips are now expecting to be selected by the user when they press the tab key, much like any normal web form.
tabEnabled in the docs
You could also try
movieClipName.focusRect = false;
focusRect in the docs

What's the command to take a picture in Sikuli

I'm using Sikuli IDE. I'd like to know what the command to take a screenshot is, so I can capture the screen at the end of a test.
Something like this
try :
if bla bla bla:
print("blablabla")
else:
TAKESCREENSHOT() #------------------> What command do I put here?
print("TEST_FAILED")
The function is capture, as in
screen = Screen()
file = screen.capture(screen.getBounds())
print("Saved screen as "+file)
It takes a screen-shot, saves it in a file, and gives you the path to that file back.
See the Sikuli documentation on it for full details.
Cheap Sikuli trick for screencaps is to have a defined region, then capture the region.
So if you've got a Chrome browser you want to cap, just set it up something like this:
App.focus('Chrome.app')
ChromeWindow = App('Chrome.app').window()
That will both focus the computer to the target application, and define a region composed of the window parameters of the application. Then run this:
capture(ChromeWindow)
Then use shutil (import shutil) to move the file around to wherever you need it in your local directories. I usually put that code pile into a function I can call when needed TakePicture(Name) where Name is what I want to call the screencap when called in a particular test. Sikuli is both powerful and easy!
To make a screenshot of the window that has focus you simple can use:
focusWindow = App.focusedWindow()
regionImage = capture(focusWindow)
shutil.move(regionImage, os.path.join(r'C:\Screenshots', 'Dummy1.png'))