8. New Keyboard Code

This file describes the new keyboard code which was written in late '95 for scottb's dosemu-0.61, and adapted to the mainstream 0.63 in mid-'96.

It was last updated by R.Zimmermann <zimmerm@mathematik.uni-marburg.de> on 18 Sep 96 and updated by Hans <lermen@fgan.de> on 17 Jan 97. ( correction notes marked *HH -- Hans )

8.1 Whats New

What's new in the new keyboard code? A lot.

To the user:

• Most of the keyboard-related bugs should have gone away. Hope I didn't introduce too many new ones (-: Keyboard emulation should be more accurate now; some keys are supported that weren't before, e.g. Pause.
• The X { keycode } option is now obsolete. This was basically a bad hack to make things work, and was incompatible to X servers other than XFree86.

To the dosemu hacker:

• While the old code already claimed to be "client-server" (and was, to some extent), the new code introduces a clean, well-defined interface between the `server', which is the interface to DOS (int9, bios etc.), and the `clients', which are the interfaces to the user frontends supported by dosemu. Currently, clients are `raw', `slang' (i.e. terminal), and `X'.

  Clients send keystrokes to the server through the interface mentioned above (which is defined in "keyboard.h"), the most important functions being `putkey()' and `putrawkey()'.

• The keyboard server was rewritten from scratch, the clients were heavily modified.
• There is now general and efficient support for pasting large text objects. Simply call paste_text().
• The keyboard-related code is now largely confined to base/keyboard, rather than scattered around in various files.

There is a compile-time option NEW_KBD_CODE (on by default) to activate the new keyboard code. The old stuff is still in there, but I haven't recently checked whether it still works, or even compiles. Once the new code is reasonably well tested I'll remove it. ( *HH: the old code is made workeable and remains ON per default, it will stay maintained for a while, so we can easily check where the bugs come from )

Just like the old keyboard code, we still have the rawkeyboard=on/off and keybint=on/off modes.

8.2 Status

Almost everything seems to work well now.

The keyboard server should now quite accurately emulate all key combinations described the `MAKE CODES' & `SCAN CODES' tables of HelpPC 2.1, which I used as a reference.

See below for a list of known bugs.

What I need now is YOUR beta-testing... please go ahead and try if all your application's wierd key combinations work, and let me know if they don't.

8.3 Keyboard server interface

This is all you should need to know if you just want to send keystrokes to DOS.

Use the functions

• putrawkey(t_rawkeycode code);
• putkey(Boolean make, t_keysym key)
• putkey_shift(Boolean make, t_keysym key, t_shiftstate shiftstate)

You may also read (but not write!) the variable 'shiftstate' if necessary.

ehm... see the DANG comments in base/newkbd-server.c for more information...

NOTE: the server's queue is limited to a relatively small number of keyboard events (currently 15). IMO, it is not a good idea to let the queue be arbitrarily long, as this would render the behaviour more incontrollable if the user typed a lot of mad things while a dos program wasn't polling the keyboard.

For pasting, there is special support in base/keyboard/keyb_clients.c which runs on top of the server.

8.4 Keyboard server structure

[NOTE: you won't need to read this unless you actually want to modify the keyboard server code. In that case, however, you MUST read it!]

[Note: I'll have to update this. The queue backend works somewhat different now.]

The central data structure of the keyboard server is the dosemu keyboard queue (to be distinguished from the bios keyboard buffer, which is run by int09 and int16).

The keyboard server code can be largely divided into the `queue frontend' (serv_xlat.c, serv_maps.c), which does keycode translation, and the `queue backend' (serv_backend.c, serv_8042.c), which does the interfacing to DOS. The two sides communicate only through the queue.

Each queue entry holds a data structure corresponding to (mostly) one keypress or release event. [The exception are the braindead 0xe02a / 0xe0aa shift key emulation codes the keyboard processor `decorates' some kinds of keyboard events with, which for convenience are treated as seperate events.]

Each queue entry holds a up to 4 bytes of raw keycodes for the port 60h emulation, along with a 2-byte translated int16h keycode and the shift state after this event was processed. Note that the bios_key field can be empty (=0), e.g. for shift keys, while the raw field should always contain something.

queue handling functions

• static inline Boolean queue_empty(void);
• static inline void clear_queue(void);
• static inline void write_queue(Bit16u bios_key,t_shiftstate shift,Bit32u raw);
• static void read_queue(Bit16u *bios_key, t_shiftstate *shift, t_rawkeycode *raw);

Accordingly, the keyboard code is largely divided into two parts,

• the 'front end' of the queue, responsible for translating keyboard events into the 'queue entry' format.
• the 'back end' of the queue, which reads the queue and sends keycodes to DOS

The Front End

 putrawkey() -------->----+
      \   \               |
       \   v              |
        \  translate()    |
         \     |          |
          \    v          \    (t_rawkeycode[4])      /---QUEUE----/->---\---|-----------*------------------------> [ raw        ]
    /       \  \  (t_keysym+char)                     [            ]
 putkey() ->-\--*--------------> make_bios_code() --> [ bios_key   ]
    \         \                                       [            ]
     \         v                           /--------> [ shiftstate ]
      \---> do_shift_keys()               /           \------------/
                |                        /
                v        (t_shiftstate) /
            [shiftstate]---------------/


--------->  data flow (&calls, sometimes)
,,,,,,,,,>  calls

Functions in serv_xlat.c

• static Boolean do_shift_keys(Boolean make, t_keysym key);
• static Bit16u make_bios_code(Boolean make, t_keysym key, uchar ascii);
• static uchar translate(t_keysym key);
• static Boolean handle_dosemu_keys(t_keysym key);
• void putrawkey(t_rawkeycode code);
• void putkey(Boolean make, t_keysym key, uchar ascii);
• void putkey_shift(Boolean make, t_keysym key, uchar ascii, t_shiftstate s);

Any keyboard client or other part of dosemu wishing to send keyboard events to DOS will do so by calling one of the functions putrawkey, putkey, and putkey_shift.

putrawkey

is called with a single raw scancode byte. Scancodes from subsequent calls are assembled into complete keyboard events, translated and placed into the queue.

putkey & others

,,,to be documented.

The Back End

Queue Back End in keybint=on mode

                   EMULATOR SIDE        |    x86 SIDE
                                        |
                      ....[through PIC].|....................
                      :                 |           :        v
QUEUE      .....> out_b_8042() --> [ port 60h ] ----:---> other_int9_handler
|         :                             |        \  `.......    (:) (|)
|         :                             |         \         v   (v) (|)
+->int_chk_q()-> bios_buffer----> [ get_bios_key ]-----> default_int9_handler
      ^  \                           :  |                   |       (|)
      :   \----> shiftstate_buffer   :  |                   v       (v)
      :               |         .....:  |               bios keyb buffer
      :               v        v        |
      :          copy_shift_state() ----+-------------> bios shiftstate
      :                                 |
      :                                 |
      :                                 |
    backend_run()                       |

Abbreviations:
int_chk_q() = int_check_queue()
out_b_8042() = output_byte_8042()

Queue Back End in keybint=off mode

                  EMULATOR SIDE         |    x86 SIDE
                                        |
             kbd_process()              |
                  :     :               |
                  :     v               |
 QUEUE -----------:--> put_keybuf() ----+-------------> bios keyb buffer
     \            v                     |
      \--------> copy_shift_state() ----+-------------> bios shiftstate
                                        |
                                        |
                                        |

Functions in newkbd-server.c

• void do_irq1();
• void clear_keybuf();
• static inline Boolean keybuf_full(void);
• static inline void put_keybuf(Bit16u scancode);
• void copy_shift_state(t_shiftstate shift);
• static void kbd_process(void);

Transfer of the keyboard events from the dosemu queue to DOS is done in two completely different ways, depending on the keybint setting in dosemu.conf:

keybint=off

kbd_process() simply reads the queue until it finds a bios keycode (as we're not interested in raw scancodes without int9 emulation), which it stores in the bios keyboard buffer, while also copying the shift state to the appropriate bios variables.

keybint=on

As soon as a key is stored into the empty queue, kbd_process() triggers IRQ1 through the PIC emulation, which some time later will call do_irq1().

do_irq1() will prepare for the interrupt execution by reading from the queue and storing the values in the variables raw_buffer, shiftstate_buffer, and bios_buffer, and then call run_irq() to run the actual DOS interrupt handler.

again, there are two cases:

• the default int09 handler in the dosemu bios (base/bios_emu.S) will call the helper function get_bios_key(), which returns the translated bios keycode from bios_buffer and copies the shiftstate from shiftstate_buffer. The raw keycodes are not used. get_bios_key() may also return 0 if no translated keycode is ready.

  The int9 handler will also call the `keyboard hook' int15h, ax=????.

• if a dos application or TSR has redirected the keyboard interrupt, its handler might read from port 60h to get raw scancodes. Port 60h is of course virtualized, and the read returns the value from raw_buffer.

  Note that a mix between the two cases is also possible, e.g. a TSR's int9 handler first reads port 60h to check if a particular key was pressed, then gives over to the default int9 handler. Even these cases should be (and are, I think) handled properly.

  Note also that in any case, int9 is called once for each raw scancode byte. Eg.,suppose the user pressed the PgDn key, whose raw scancode is E0 51:

  • first call to int9:

                 read port 60h        = 0xe0
                 read port 60h        = 0xe0   (**)
                 call get_bios_key()  = 0
                 iret
    

    do_irq1() reschedules IRQ1 because further scancodes are in the queue
  • second call to int9

                 read port 60h        = 0x51
                 call get_bios_key()  = 0x5100    (bios scancode of PgDn)
                 iret
    

  (** multiple port 60h reads during the same interrupt yield the same result.)

This is not a complete documentation. If you actually want to hack the keyboard server, you can't avoid reading the code, I'm afraid ;-)

8.5 Known bugs & incompatibilites

• behaviour wrt. cli/sti is inaccurate, because the PIC code currently doesn't allow un-requesting if IRQ's.
• emulation of special 8042 and keyboard commands is incomplete and probably still somewhat faulty.
• the 'internal' keyboard flags in seg 0x40, like E0 prefix received etc. are never set. This shouldn't hurt, for all but the most braindead TSRs.
• CAPS LOCK uppercase translation may be incorrect for some (non-german) national characters.
• typematic codes in X and non-raw modes are Make+Break, not just Make. This shouldn't hurt, though.
• in X mode, shift+Gray cursor keys deliver numbers if NumLock is off. This is an X problem, and AFIK nothing can be done about it.
• in X, something may be wrong with F11+F12 handling (and again, possibly necessarily wrong).
• the Pause key works in terms of raw scancodes, however it's function is not implemented (i.e. it doesn't actually halt DOS execution.)
• in terminal (i.e. slang) mode, several things might be wrong or at least improveable.
• there is no difference between the int16h functions 0,1 and the extended functions 0x10,0x11 - i.e. 0,1 don't filter out extended keycodes.
• keyb.exe still doesn't work (hangs) - most probably due to the above.

8.6 Changes from 0.61.10

• adapted to 0.63.55
• adapted to 0.63.33
• renamed various files
• various minor cleanups
• removed putkey_shift, added set_shiftstate
• in RAW mode, read current shiftstate at startup
• created base/keyboard/keyb_client.c for general client initialisation and paste support.

8.7 TODO

• find what's wrong with TC++ 1.0
• implement pause key
• adapt x2dos (implement interface to send keystrokes from outside dosemu)
• once everything is proved to work, remove the old keyboard code