Classify a program from a screenshot - language-agnostic

I need to write and algorithm that can detect which state an application (used to fill out forms) is in based on screenshots.
It has 2 inputs:
A: Approximately 2-10 screenshots from an application with different tabs selected. These are made by the user, so i can instruct him to things like "select the upper area of the program" or "select the whole window", but i can not expect pixel-perfect precision.
B: a screenshot of one of these states. The forms are filled with different data.
The goal is to determine which screenshot from "A" is from the same state as "B".
An example screenshot:
An example based on this screenshot:
A input: 10 screenshots from this program with "Menu","Sale Order","Purchase order",... tabs selected
B input: the screenshot above.
The task is to determine which of the 10 screenshots matches this image.
I have tried to use an image descriptor algorithm, (SURF) but it has a really high error ratio, since its not made for such tasks.
Anyone has an idea how to make such classification? Should i use some filter (e.g median or blur) on the screenshots, and then run trought some classification algorithm? Or extract some other feature to classify (FFT,histogram,..)?

I guess you can use the tab width instead of the tab label, which is much easier to calculate. For example, {"Menu", "Sale Order", "Purchase Order"} all have different widths.
If you have to look inside the tab, you can attempt some template matching.

Detect the text of each tab, then look at the background color.
Alternatively, locate one of the menu icons for pixel-level registration, then do a pointwise sampling to determine which tab is selected.

Related

Using RStudio DevTools, can you prevent the Environment Pane from hiding column values with ellipses (...)

I'd like my data to at least extend horizontally to the edge of the environment pane, but it gets truncated even if there is plenty of room left in the pane. Columns of class "chr" appear to be restricted to showing only the first 4 entries (yellow arrows in the first picture), and columns of class "num" and "int" tend to show either 5 or 10 values (green arrow in the first picture).
Right-clicking on one of the rows and selecting "Inspect element" brings up RStudio DevTools; using that, I can at least enable a scroll bar by changing "overflow-x" from 'hidden' to 'scroll'. However, the scroll bar only scrolls to the original 4, 5, or 10 values that were shown, and then you still see the ellipsis. Changing "text-overflow" from 'ellipsis' to any of the other options seems to have no effect, nor does changing any of the other properties (padding-left/right, border, white-space).
I'd just like to see more data in that pane or enable a scroll bar at the bottom of the pane instead of for each individual row, but I can't find any settings to change to enable this. The theme I'm using is Tomorrow Night Bright if that matters.
You definitely can't do it using RStudio DevTools. RStudio implements their display using Chrome, so RStudio DevTools is just Chrome DevTools: the truncation of values has already happened before Chrome sees it.
There's probably a variable somewhere in the RStudio sources that controls how the truncation takes place, but I don't know where. You can look through the sources here if you want: https://github.com/rstudio/rstudio . I kind of doubt that it will be something you can control from R, but you never know.
So I can't offer a general solution, but I can offer a small hack. Normally I use this when I want to look at an expression: many debuggers let you watch expressions, but RStudio just shows you existing variables. However, if you want to see x+y in the environment pane, you can just calculate it in the console:
`x+y` <- x+y
and it will be displayed. (You've also created a new variable, and possibly stomped on an existing one.) This doesn't change like a live expression, so it's not that great, but it's better than nothing.
For your purpose, the thing to calculate would be something to display the missing bits of your display. If you want to see more manufacturer values, then do something like
mfr <- paste(mpg$manufacturer, collapse = " ")
This creates one long string, and will show as much of it as will fit.

this formula uses scope which is not presently supported for evaluation - is this relate to my O365 license type?

I want to group my Gallery then I follow training on Youtube, here was my formula :: GroupBy(MCUT,"Machine_Eng","GBY") then i move to item on Gallery and put formula on Title.Text = ThisItem.GBY then I found this messgae
"this formula uses scope which is not presently supported for evaluation" and also have small X in red circle.
Here is my screen ---
Here are my question
is this about limitation for my O365 license?
how to check my license type?
thank you in advance for all helps ^__^
This error is shown only in the "watch expression", meaning that it won't be able to show you its value under the formula bar. That is because in a gallery there will be many rows, and the value of 'ThisItem' will vary among each row and thus the editor can't know which value to choose. This is true regardless of which license you have.
The error that you have is something different. If you hover the mouse on the term 'GBY' over the squiggly red line, you will see more information about the error. Based on the image, it looks like you are trying to use this expression in a label - and since GBY is a table (the group of MCUT rows that have a common 'Machine_Eng' value), this cannot be used in a label.
I had the same issue and it turns out that the column I was using allowed multiple entries. When I changed the column, it worked perfectly.

How to find same logos, UI elements using sikulix

I am new to sikuli.
I am using it for functional test automation of my java spring standalone application. I am having the same set of images(logos) at multiple locations on the screen. and they are generated dynamically as per the application data. In that case how can I find and click on certain buttons/elements in the UI.
Example: like, share, comment text/image will be there several times in facebook page. in that how to we click on particular buttons out of all.(facebook is just an example) i want the similar solution for my application.
I think what you're asking is- given a case where the same image or UI element is present multiple times on the same page, how can I specify one of these elements in one location versus the same element in another?
The quick and easy way to do this would be to capture a larger picture, like as much of the window as you can, and click the image in parentheses after the click action; this opens a window reading file, matching preview, and target offset. Click target offset, use the mouse wheel to zoom in, and click on the exact point of the picture that you'd like to click.
It's possible to click in multiple places on a captured image, provided you use different click() commands, and you can set the target offset for each click() command without needing a new picture.
With just this one image, you can potentially click every square on the board provided you take the time to set the offset for each of the 64 click() commands it'd take to do so. http://files.chesscomfiles.com/images_users/tiny_mce/cigoL/Chessboard.png
The more difficult solution would be setting up regions, and searching those regions for the matching pictures, but for what you're looking for this should- for more about regions and setting it up check links below. Cheers!
http://sikulix-2014.readthedocs.io/en/latest/index.html
Based on your FB example you could use setROI() to focus on the top post on the page (imagine a rectangle that covers a complete post on the news feed) from there I would loop:
if exists(profile pic):
click(like button)
elif exists(profile pic 2): # If you are testing more than one user
click(like button)
else : type(Key.DOWN) #repeat as needed
Getting the next post into the ROI may be tricky but using WHEEL_DOWN may help if you can adjust your lines traveled with each wheel click on your system.

SSRS Line Chart: Adding an Average/Target line to Line Chart

I'm sure I'm missing something obvious but it has so far eluded me.
My current report:
I have a dataset of employees' absences at my company.
It is essentially broken down by department, date and time lost.
I have charted this as below:
The manager is shown their own departments dynamically using a filter on the series group.
By using a report level filter I can cache the dataset overnight for all departments.
My preferred report:
In order to give some context to the data I would really like to add another line showing the same data for the whole organisation.
This data is already captured and I can add that "department" for all users so they get a graph as below (new line in blue with square markers):
The trouble is that I want the extra line to appear completely differently to the existing lines
I don't want the extra line to have markers necessarily
I don't want the extra line to appear in the same legend
I don't want the extra line to have an action (i.e. be clickable)
In my perfect world, there wouldn't even be a line, just a change in background colour above and below where the line would be. I haven't been able to find a pretty example of what I am talking about so instead I have produced this hideous travesty in ms paint:
Any help gratefully received!
Can change all of that information/treatment by modifying the series.
Firstly change the Chart Type of the series to Area
Which gives you something like....
All other settings you mention can be done in the series properties
eg
Right Click Series, Select Properties, Legend, Check the box marked
[Do not show this series in the legend]
Set the Action (in series properties) to None
Set the markers as you want
I have achieved partial success using the StripLines property for the Y Axis:
Screenshot of setting:
I added a Member then changed the BackgroundColor and StripWidth.
The StripWidth determines the height of the shaded area and can be an expression.

How to Format the Labels in a Pie Chart in SSRS 2008 R2

In my SSRS 2008 R2 Pie Chart I want to display 3 values which are calculated fields in my dataset.
The dataset collects a bunch of stats for call center agents. Among them are ProductiveTime, ExcepctionTime, NonProductiveTime, for, of course, a given time period.
If you think you need a bit more details read this paragraph:
The systems keeps track of each change of status in an agent. For example, on break, lunch, in a meeting, training, etc. Then in my stored procedure I create an additional column that has the category of that status, so, for example, lunch, on break will be NonProductiveTime, and so on. Between the selected time period there will be hundreds of entries logging each time there is a change in status, and how long the status lasts.
So back to the Pie, I am adding the SUM of those 3 values to the Chart Data, each one being a series.
The problem I am having, is that when I select the Series Label Properties and configure the format for the Number, only the first series picks it up. It doesn't make a difference if I do it using the UI or the properties window.
What am I doing wrong?
i had this same issue, if you just use the arrows (little blue arrows at the top of the chart data box) to move the value to the top of the list you can edit the label details/code/expression etc for that series. then just repeat this for all the values you've added.
i'm sure there's a better way but this worked so that's what i did.
It seems that when you are editing a label it is only ever editing the item/value at the top of the list, after you've done your labels then just shuffle them back into the order you wanted them listed
I have found you can get this with formatting the 'Label' property of the collection 'Chart Series Labels'. I have also ran into issues with problems of getting enough info of how to format. I think you can get what you want by some built in 'system syntaxes' used.
Here is an example I used to format the YValue and give me a percent on it's value as well:
#VALY{###,##0} (#PERCENT{p0})
Please see a more inclusive list on this msdn page:
http://msdn.microsoft.com/en-us/library/bb677551.aspx
You can format your series labels by going to property of each labels, First Right click on all series properties and tick "show data labels" and then select first property and press F4a property window will open ->under LABELS--> find fonts you can set here.
You can set other properties also here.
I couldn't figure this one out so I went ahead and created a new stored procedure that gives me the data as rows with two columns: one column for the property and one column for the value.
That way I can use just one series for the chart, and to display as percentage, I found this one to be very useful: http://bipassion.wordpress.com/tag/ssrs-pie-chart/
Looking at the answers above and my latest VS 2015
I used these steps
1- Clicked on the Value in "Chart Data" box
2- Clicked on "Show Data Labels"
3- Press F4 to open the "Properties" window on the side
4- Expand "Label" section in the "Properties" window
5- Change "Label" and "Format" property under Label section in the "Properties" window
6- I got a message box says if I want to change the label property then I have to set "UseValueASLabel" to False