In RM Version 126.96.36.199 and the previous version for Macintosh, I have experienced situations in which RootsMagic becomes unresponsive -- no click anywhere in the window has any effect. I'm not sure if there is anything I may be doing to help cause this. This may be happening on wake from sleep or when RootsMagic has been idle for a while.
I use two monitors on my mid-2011 iMac under Mac OS X El Capitan 10.11.4. I also run RootsMagic as a full-screen app in its own space under Mission Control, if that matters.
I have ended up using Force Quit a couple of times. Command-option-esc brings up the force quit window where RootsMagic shows up as "wineloader". It doesn't say in that window that the app is unresponsive.
There may be a workaround. A couple of times I have regained responsiveness in RootsMagic by playing around with the window controls. Most recently, I double clicked the Mac title bar, which minimized the window. When I restored the window from the dock, it was working again.
Roots Magic allows multiple windows within itself. I have been opening the same file in two windows to have two views of the same database. Is this okay, or is it a "bad thing" that I shouldn't expect to work? It's a great feature if it doesn't cause problems!