Sunday, April 15, 2012

5-Button Mouse in Linux (VMWare Guest)

I just wrote a lengthy post about making 5-button IntelliMouse work under Fedora Linux v16, running as a guest OS in VMWare.
The post is available here. The text will be copied here but the original will remain on Google+. I find posting to Plus a bit easier but texts on the blog are better indexed by search engines.

5-Button Mouse in Linux

NOTE: If you are looking for a solution, just scroll down to the bottom of the post. If you're interested in the messy ways of getting things done then read on.

One very annoying thing I found in Linux, running in a VM, was that there was no support for a 5-mouse button. That's how it seemed, at least. So I searched for advice on how to enable this. Surprisingly, there were very few posts describing these sort of problems with mice and most of them were pretty old. This very much seemed like, if there were any issues, they were all sorted out long ago. XWindow configuration would probably properly detect hardware.
Anyway, in my case it didn't so I decided to make it work. The first time I read an article or two and poked around configuration files. Installed imwheel package and added some shell scripts. I expected that everything would work after that and that I could go back to what I was actually doing. So I did not bookmark URLs, nor wrote down the steps or anything useful. When it did not work, I was a bit concerned about those stale lines in configuration and init scripts.
Fortunately, reading the articles below went through all those scripts and I disabled them. Below is the log of my second try.

Instructions on how to enable 5-button mouse (IntelliMouse) in Fedora Core #linux .
However, I'm using imwheel addition to the script and things still don't work properly.

Then I tried 2nd approach:

As this did not work either, I suspected VMWare had something to do with it. It turns out that VMWare only emulates 3-button mouse. No wonder Linux doesn't see more buttons, then.
This did not work initially. However, the setting I added got overwritten so I'm not sure if it works. But I found another answer with a different setting:

In the vmx file, these are the options to add:

usb.generic.allowHID = TRUE
mouse.vusb.enable = "TRUE"

I added lines from both articles and finally got events for 'back' and 'forward' buttons appearing in xev (X Event Viewer). Yay! Now it was only a matter of configuration of X in order to see them. If at all. All the buttons and the mouse wheel fired events in xev, which was a good sign.

Now, opening Chrome browser to test. The back and forward buttons seem to work OK but the mouse wheel acts weird. Scrolling down and up acts exactly the same as back and forward buttons. Time to check the configuration.
Editing /etc/X11/xorg.conf is where all the settings are. No need to install any additional packages or init files and scripts. The way I was going now, reducing all the options I added in the meantime, it seemed I would end up going back to the original X configuration and just changing the number of buttons in the settings from 5 to 7.

In order to quickly restart X to test the settings, use Ctrl+Alt+Backspace keyboard shortcut. X will re-read the xorg.conf file and load the new settings.

So, at this point I had the back and forward buttons working fine. The problem I had now was to properly set up the mouse wheel so it doesn't go back and forth but actually scroll the page up and down. Xev showed that the events were OK. Scrolling events were different to clicking events so trying another client program could be useful. 
Running Dolphin, file browser, showed that scrolling up and down moved the cursor up and down the files but also back to previous directories. 
So I found this:
This points to imwheel config. However, in imwheelrc file it is stated that sending button events is equivalent to having ZAxisMapping set to "4 5" in xorg.conf. This was the default config there, too.

So, again, the suspect is VMWare -
Now adding 
Option "Protocol" "IMPS/2"
to xorg.conf. Some people reported changing the protocol to make mouse wheel work again.

Then I tried adding 
Option "SendCoreEvents" "true"
to xorg.conf, according to

Trying the old suggestions with imwheel again, from
My .xinitrc in home directory still contained the call to imwheel. Disabling that did not change anything. Killing imwheel process causes scrolling to completely stop reacting to mouse wheel events.
Then I finally found all the references to imwheel and xmodmap in my ~/.xinitrc and commented them out. This brought back the scrolling after restart!
And the back and forward worked properly in Chrome!

So, to summarize what was necessary to make this work. First the VMWare settings. Add the following lines to .vmx file:

usb.generic.allowHID = TRUE
mouse.vusb.enable = "TRUE"

Not sure if both are required but everything works with this. Needs confirming whether only one of those is enough as they were advised in separate articles.
Then, set up the mouse config in xorg.conf:

Section "InputDevice"
  Identifier "AlensMouse"
  Driver "mouse"
  Option "Device" "/dev/input/mice"
  Option "Protocol" "IMPS/2"
  Option "Buttons" "7"
  Option "ButtonMapping" "1 2 3 6 7"
  Option "SendCoreEvents" "true"

These settings might also be simpler. For example, putting "auto" as a protocol or removing "SendCoreEvents". But, as it is already very late, I will call it a day here and go to sleep happy after enabling the back and forward mouse buttons in browser under Linux (guest with VMWare).

No comments: