VESA mode 1440x900x32 is listed in the VESA BIOS but produces garbled screen. Any suggestions on how to troubleshoot this? 1440x900x16 works essentially as normal, but the colors seem slightly off (greens appear blueish, yellows appear pale).

in general, you cannot debug vesa. there might be a bug in the emulation tho. you find these by looking at gigabytes of machine code traces and see if theres anything odd. it might be that these vesa modes just do not work due to not being used/tested. use the -t flag to realemu and redirect std error to a file, then run a mode switch (be patient, it might take hours with tracing enabled).

This is the output of changing from VESA mode 1440x900x16 to 1440x900x32 with realemu -t:

http://plan9.stanleylieber.com/hardware/thinkpad/x301/2776-p4u/vesa.txt

This is the output of changing from VESA mode 1440x900x16 to 1440x900x32 with realemu -p:

http://plan9.stanleylieber.com/hardware/thinkpad/x301/2776-p4u/realemu.p.txt

ok, looking thru the machine code trace. i see it is programming the PIPEB source image size to 768x480 instead of 1440x900. looking thru the code traces, it seems it swaps mode 0x16b (1440x900x32) with mode 0x162 (768x480x32) for some reason.

there are two tables involved in this. a 3 byte per entry (mode alias?) table at c000:7921 where the format is like (xx yy zz) where xx is the low byte of the mode it wants to alias, or ff or end of table. yy seems to be the mode to map into (for paged framebuffer?) and zz is the mode the thing gets replaced.

in our case zz is 0x62, thats why it programms the thing wrong. that mode is then looked up in another table that contains the actual resolution. that table starts at c000:0268 with 5 bytes per entry. format sems to be xx ?? ab cd ?? where xx is the mode (0x62) and ab cd is a pointer to some other stucture. that pointer + 8 is decoded like this:

017a38 20004062 00001628 00000004 00e00008 000012d4 00071423 00007bdc 00007b6c 0000 c000 0000 c000 cszoPdI 45b1 e8 CALL [3dcb] 017a39 20004062 00001628 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 0000 c000 0000 c000 cszoPdI 3dcb 8a MOV AH, ES:[DI+4] 017a3a 20003062 00001628 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 0000 c000 0000 c000 cszoPdI 3dcf c0 SHR AH, $04 017a3b 20000362 00001628 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 0000 c000 0000 c000 cszoPdI 3dd2 8a MOV AL, ES:[DI+2] 017a3c 20000300 00001628 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 0000 c000 0000 c000 cszoPdI 3dd6 8a MOV BH, ES:[DI+7] 017a3d 20000300 00001028 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 0000 c000 0000 c000 cszoPdI 3dda c0 SHR BH, $04 017a3e 20000300 00000128 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 0000 c000 0000 c000 cszopdI 3ddd 8a MOV BL, ES:[DI+5] 017a3f 20000300 000001e0 00000004 00e00008 000012d4 00071423 00007bdc 00007b6a 0000 c000 0000 c000 cszopdI 3de1 c3 RET 0

reslting in x and y dimensions in AX and BX that will later be programmed into the graphics card.

i need your realmodemem file. with that, i can decode these two tables and possibly fix up the entry.

ok, theres the mode alias table at c000:7921. i put commas between the entries and you can see, that mode 6b is the only one that maps to 0x62 in the 3rd byte (see the *).

0000000 01 ff ff,01 30 30,03 32 32,05 34 34,07 38 38,3a 0000010 3a 3a,3c 3c 3c,11 41 41,14 43 43,17 45 45,1a 49 0000020 49,4b 4b 4b,4d 4d 4d,12 50 50,15 52 52,18 54 54, 0000030 1b 58 58,5a 5a 5a,5c 5c 5c,60 60 60,61 61 61,62 0000040 62 62,63 63 63,64 64 64,65 65 65,66 66 66,67 67 0000050 67,68 68 68,69 69 69,6a 6a 6a,6b 6b*62,6c 6c 6c, 0000060 6d 6d 6d,6e 6e 6e,6f 6f 6f,70 70 70,71 71 71,ff

so, we can try to patch that single byte from 62 to 6b like in the other modes. and maybe notify intel about this bios bug.

to patch the byte:

dd -if /dev/realmodemem -bs 1 -iseek 817532 -count 1 | dd -of /dev/realmodemem -bs 1 -oseek 817533 -count 1 -trunc 0

confirmed. this works. i wont say one cannot debug vesa bios anymore :)

added work arround in aux/vga for this in r2cd7add619a5, needs testing.

confirmed, works. the wrong colors are a display problem as it can be reproduced with other operating systems. output on external monitor is flawless. so closing this bug now.