I noticed in reviewing https://pagure.io/fedora-qa/os-autoinst-distri-fedora/pull-request/275 that the "help", "preferences" and maybe "about" needles are kinda redundant with several other app tests. e.g. the gnome/apps/gnome-text-editor/gte_preferences_submenu_light-* needles are the same as the system-monitor sysmon_menu_preferences needle. There are several needles which are all just the "Help" button from a standard GNOME app burger menu. And several app tests also have needles for the "About" item from the burger menu; sometimes this needle includes the application name, but unless we're really worried they might omit or misspell it, we could just have a generic needle that matches only the "About" text and which should work for all apps.
gnome/apps/gnome-text-editor/gte_preferences_submenu_light-*
sysmon_menu_preferences
Can you please go through all the apps and consolidate these needles into generic ones where possible, stored in needles/gnome? Thanks.
needles/gnome
Sure, I will do it. I guess it won't be any fun, though. :D
I went through the needles in graphical applications and I realized that many of the needles were part of the original designed that somehow counted on a fact that each application test will have its set of needles which inevitably led to duplicates. Now, I tend to believe that we should be more generic about the UI.
So, in step 1, I moved the button needles into a dedicated directory gnome/buttons and gnome/menus for the menu needles, i.e. all application buttons and menu items will be gathered in those directories which will allow for reusing needles much more, as well as the menu items. I would continue keeping the rest of the application needles in their own directories without further mixing.
gnome/buttons
gnome/menus
As step 2, I went through the scripts and changed specific needles into generic needles where I thought it was needed, and deleted the rest of unused needles. However, it turned out that many applications started to fail and I had to reneedle them. I looks to me like if there were subtle differences in menu items and buttons throughout applications, so the number of needles did not dicrease much.
As step 3, I tried to experiment with lowering the match level to 75 or even 70% and I was able to get rid of quite a number of those needles while the tests are still passing. Although we have never experimented with this, it might be a way to reduce the duplicates. On a small area, the needles often fail with 0 - 40% match, so keeping it around 75% will not harm the recognition much and will tolerate small glitches.
I now have step2 version and step3 version. Which one seems more plausible to you, @adamwill ?
Huh, interesting. Can you tell what the minor differences are? Did you do test runs on the public stg instance or just locally?
Match level below 90 does sorta instinctively worry me a bit, but...it may be okay...ideal would be to manage it without dropping the match level so low if possible, but if it's not possible to do that by being tight about match areas and so on, maybe it'd be alright. More detail / sample runs to look at would be great.
thanks for working on it!
Well, the minor differences are just small changes in font kerning or similarities. When I save two such needles and move between them quickly, I can see the fonts "shiver". I have only tried it locally, however, so I can flip the branches on stg and see how it goes there.
stg
ah, fun, so it's probably subpixel positioning or something :( I've seen that in the Zezere test too - I think because the MAC address changes and it's not using a monospace font, so the button positioning can change slightly depending on how wide the characters in the MAC address are. But in that case we only needed to drop the match level to 90.
next_button-startsetup-20200818 is needed for F35 and F36 live image build/install tests, see e.g. https://openqa.stg.fedoraproject.org/tests/2402471 . please put it back.
next_button-startsetup-20200818
do you know about the 'last use' view at https://openqa.stg.fedoraproject.org/admin/needles ? it's useful for checking whether a needle is actually being used...
Metadata Update from @lruzicka: - Assignee reset
Log in to comment on this ticket.