So, I just had a somewhat better experience, although there are still severe problems changing resolution.
First, I stayed with the Direct3D renderer. Before mucking with the resolution, I set the video to 32-bit.
I found that you have to change resolution in steps. After a resolution change, the game only renders in a portion of the window. If you changed by too much, then you can't see the dialog to confirm the change. After it times out, it will restore you to the previous setting. By using a couple of steps, quitting and relaunching the game after each, I was able to get to full resolution.
At each step, after you change the resolution, you may not be able to control the cursor and thus quit the game, which is necessary for the change to stick. I worked around this on the Mac by using Command-W, which is a keyboard shortcut for closing the window. Unfortunately, Alt-F4 didn't seem to work. I don't know what you might use on Linux, although I suspect most window managers have a keyboard shortcut to force-close a window.
Once in the game, I didnt' see a problem with the mouse (other than the above). In the game's preferences, you might try DirectInput for the mouse.