Vista Aero thumbnail and FindWindow affinity issue
Our software tries to find already existing dialog using FindWindow and activate it. If not found, we start a new instance.
This failed when Vista Aero is enabled. Here is the situation:
Open the application; hover the mouse on the tray and let the thumbnail come up for that application. Close the dialog application. Now we tried to open another instance. But to our surprise dialog app is not getting created. When we debugged we saw this behavior: when thumbnail is popped up, a new window is created with same title text as your original app. This thumbnail window persists even after you close you dialog app. When we tried launching our dialog, it used FindWindow(NULL, "<text>") and got a valid handle. But this handle is of thumbnail window. So, the application assumed one is already existing and tried to activate that and not launch a new instance.
Workaround for this is to search with specific class atom. Thumbnail window registers as "ThumbnailClass". By default, all our dialogs are created with class atom 32770 (0x8002). So, we made use of the class atom to narrow our search: FindWindow(MAKEINTATOM(0x8002),"<text>")
other observations: Thumbnail window persists until another thumbnail is shown.
tools used to debug: WinDbg; SPY++;