** Update Oct 1, 2018
I spent some time with my new character editor (available on GitHub as part of the upcoming SDK, look for an announcement soon.)
The image on the right is RC2 of what I've decided to call the NuPET ASCII character set. This character set is 100% compatible with 7-bit ASCII and largely compatible with PETSCII, with the shifting of certain character blocks.
This will be the default character set of the Foenix and the Nu256 virtual machine.
Notable changes are:
1. Shifted graphic characters have been removed from the lower 7 bits. Add 96 to PETSCII characters in the 96-127 range to get the NuPET version of the symbol.
2. The vertical bar (|), tilde (~), underscore (_), braces ({ and }), and backslash (\) replace the Britsh Pound (£) and CBM-Only glpyhs in the lower half of the character set. This makes NuPET ASCII 100% compatible with standard 7-bit ASCII.
3. PETSCII does not include half of the symbols needed for bar graph drawing. Those are added in the $90 row.
4. PETSCII does not have double-bar box graphics. Those are added in the $E0 row with single-double intersections in $8A-$8F.
5. Other assorted symbols missing from the default PETSCII set are added.
6. I'm still toying with making letters 5 rows to give lower case letters proper descenders. This changes the look of the character set by quite a bit, though, and some people might find it less pleasing than the standard CBM character set. I'd like to get your input on this. (The letters in the above image have been converted. The numbers have not.)
The characters included are done with an eye toward building text-based GUI screens. All standard GUI elements should be included with this iteration of the character set. If you find a necessary element is missing, please let me know.
It's worth noting that some graphic characters overlap the ranges used by PETSCII control characters.
To print NuPET graphic symbols, print $0F (15, or Shift In). This will display the NuPET symbol for everything except CR, LF, and ESC.
To use CBM control characters, print $0E (14, Shift Out). All CBM control characters (cursor controls, colors, etc) will be interpreted the same way as a Commodore computer
In all modes, Sending the ESC character will prefix an ANSI control sequence. For example, {ESC}[2J will clear the screen.
Original Post
So I've been digging through some prior work I did, especially related to character sets.
As all you Commodore aficionados know, the character set introduced in the Commodore PET computer is not actually ASCII; it is mostly ASCII, but with some choices that make interacting with ASCII systems sometimes challenging. We will call this character set PETSCII, to make things simple.
The ASCII grave (`), tilde (~), backslash (\), pipe (|), and underline (_) symbols are not present on PETSCII.
Many glyphs are present in the upper-case/graphics set that are not present in ASCII (all the shifted letters).
In upper/lower case mode, the case of the text is reversed: upper-case letters in ASCII are lower case letters in PETSCII and vice-versa.
I'm going to propose that we reconcile the PETSCII and ASCII character sets with the a new way of handling text. In addition to the existing upper-case/graphics and lower-case text modes, I'm proposing adding an additional 128 glyphs that cover the 1982 IBM PC character set (also known as Code Page 437), several glyphs that fix shortcomings in the PETSCII set, and a character set mapping feature that allows several predefined maps, as well as custom maps, in the VICKY graphics processor.
It would work like this: the built in character ROM would contain room for 1024 character slots. When VICKY renders the screen, she can select a character from any of the character slots, based on the current selected character map. This is similar to the way the VIC chips handle character mapping, but with 4 banks instead of two. In addition, VICKY should be able to select a character map from VRAM, as well, like the VIC chips.
The predefined maps would include:
PETSCII upper case/graphics (A, ♠, £)
PETSCII upper case/lower case (a,A, £)
IBM ASCII: a, A, \, |, ~, `, _, and all glyphs in ANSI Code Page 437.
PET/ASCII: codes 32-127, standard ASCII characters, including `,~,\,|,_. 160-255 contains all PET graphic glyphs and 32 new glyphs to round out the block drawing set, plus some bullets and other assorted symbols.
Switching between character sets is currently done by printing CHR$(14) (text mode), CHR$(142) (upper/graphics mode). I propose adding CHR$(15) and CHR$(143) as IBM ASCII and PET/ASCII, respectively, with PET/ASCII being the default character mode on bootup. I suggest extending the Shift/C= keyboard command so that it cycles through all 4 character sets.
I've attached the PET/ASCII layout, to show you the proposed PET/ASCII character set. The left image is the original PETSCII character set, with redundant characters removed. The right image is my proposed character set. Note the extra glyphs starting at $E0.
Finally... a justification for these changes:
An ASCII compatible character set is necessary for data interchange with other computers. PETSCII's lack of compatibility with the PC has long been a thorn in users' sides. This creates a standard that is easier for developers to use, without the need to resort to uploading custom character sets. Without a standard, every person who writes a text editor or terminal program will end up rolling his own, which will cause undesired fragmentation in the software community.
PETSCII is still desirable because it's unique and a bit of a calling card for Commodore users. Its inclusion is necessary if we are to support CBM BASIC.
The PET/ASCII mode lets a user include all of the original Commodore glyphs, including some not originally included (but which should have been.) This is, I believe, the ultimate evolution of the 8-bit Commodore character set.
Stefany, I've been playing with my terminal/simulator, and I think a 3-page system is going to be best way to go, at this point. (Yes, I've got some redundant info in this post. It's just easier to summarize, at this point.)
Page 1: Character code. This is the ASCII value of the character. (Codes 1-32 will be blank or filled with a representation of the command code's function, such as a down-and-left arrow for Return).
Page 2: Color Map. Each byte will contain the foreground and background color for the character cell. The upper 4 bits is background, and the lower 4 bits is foreground.
Page 3: Attribute Map. The upper 4 bits selects a character map. The lower 4 bits selects the rendering mode for the character - Bold, Italic, Underline, Reverse.
I know you also want to do 8-bit color modes, so if you support 8-bit text color, this would mean adding one more page, for 4 pages total.
In order to support multiple code pages or character sets, we should have 16 registers that set the address of the font data. Fonts will always use 4K or 8KB of memory, and so we can assume fonts will always be loaded on a 4K boundary and will reside in VRAM during system operation.
Fonts are always stored with 1 byte per raster line. Each bit represents one pixel in the row. So the letter A, for example, would look something like this:
bits Pixels Value in decimal 00011000 | ** | 24 00100100 | * * | 36 01000010 | * * | 66 01111110 | ****** | 126 01000010 | * * | 66 01000010 | * * | 66 01000010 | * * | 66 00000000 | | 0(I know you know how this works; I'm spelling this out for folks who may never had designed a custom font.)
An 8x16 font would be similar, just with more rows and with the character image extended further vertically. I figure on using 10 rows for upper case characters , 5 or so for lower case characters, and 4-5 rows for descenders (the bottom part of j, g, etc.) that will leave 2-3 rows for line spacing.
If you use any exotic rendering modes, like 7x14, the rasterizer should crop an 8x16 font. So the first line of a font will always fall on an 8-byte boundary and will always be 8 bits wide.
I will include a total of 8 fonts in the system, an 8x8 and 8x16 version of the following fonts:
PETSCII Upper Case - this is the default font on a Commodore 64, 128, and VIC-20
PETSCII Lower Case - this is what you get when you press C= + Shift on a Commodore
PET ASCII - ASCII-ordered font with PET symbols in the upper 128 positions.
IBM ASCII - ASCII-ordered font with IBM graphic symbols in the upper 128 positions.
This will use a total of 48K of ROM: 16K for the 8x8 fonts and 32K for the 8x16 fonts.