The Serial Mouse uses a RS-232 style interface with -5 to -12 VDC as logical 1 and +5 to +12 VDC as logical 0. The PS/2 mouse interface is a TTL-style interface, which uses 0 - +2 VDC as logical 0 and +3 - +5 VDC as logical 1.
In addition - and to make things worser - the RS-232 is an asynchronous interface, the PS/2 interface is a synchronous, where the data is sent along with a clock signal. It uses a simplified 4-wires serial interface with +5VDC (for the transceiver), GND, keyboard / mouse clock and keyboard / mouse data. The data and clock line can be used from the keyboard / mouse controller and the attached device as well following a particular handshake, which defines which is the active sender and which is the receiver. So: it is not *that* easy conversing serial mouse to PS/2 and vice versa.
The dual mode mice have an automatic logic detection and sort of adaptive interface electronic, which detects whether the mouse is attached to a serial port or a PS/2 port and set the output drivers accordingly.