Pitfall II ROM recognitiion

[trebor68]

Gopher2600 only recognizes and supports the "bad" Pitfall II NTSC ROM dump and not the verified "good" dump:

Good Dump (CRC32 B960C29E).

Bad Dump (CRC32 097CE7AD).

Pitfall_II_Dumps.zip

[JetSetIlly]

Thanks. The first step of cartridge fingerprinting is based on size. I remember thinking when I was implementing DPC that 10495 was a strange size. I didn't stop long enough to think why.

It's an easy enough fix. It'll be in the next binary release of the emulator. If you really need to load the good dump in the meantime, you can force the use of DPC from the command line with

gopher2600 --mapping=DPC <rom file>

But here's the thing and you might be the man to ask, as far as I can tell a DPC rom dump should only be 10240 bytes. Two regular banks of 4096 bytes and 2048 bytes of the 'static' data only accessible by the DPC chip.

What is the extra data? Is it just random noise from the dumping process?

[SpiceWare]

My understanding is the extra 255 bytes are the RNG sequence generated by DPC.

Found some info here:
https://forums.bannister.org/ubbthreads.php?ubb=showflat&Number=123431

[JetSetIlly]

Thanks. That's interesting.

I'm just reviewing my RNG generator for DPC. I'm using the method described in the patent but I didn't get it quite right the first time. I've changed it so it better matches the patent text, but I don't think it ultimately matters because the numbers will be random in any case.

Is it worth the time to use the random data from the dumps to verify that the RNG implementation is correct?

If the dumps are not good for this, how else could we verify that it's correct? Moreover, how can we verify that the cartridge actually implements the RNG described in the patent?

[SpiceWare]

I don't know who made the dump so don't know if it's the actual RNG sequence created by DPC, or an arbitrary RNG sequence they appended in order to get Pitfall II working in emulation. To get the sequence I'd have the cartridge dumper read address $1000 255 times.

From the comments in Stella:

  // Clock the random number generator.  This should be done for every
  // cartridge access, however, we're only doing it for the DPC and
  // hot-spot accesses to save time.

extracting the RNG sequence after getting the ROM dump would most likely result in the sequence starting at a different point than if it were read before getting the dump.

[JetSetIlly]

Yes. It would be good to be able to replicate the random number stream exactly but I don't think it's necessary.

For games purposes, the sequence effectively starts when the player begins a new game. So even if you only pump the RNG on RNG access (as Stella does according to the comment you've identified) it will be sufficiently random.

That comment also prompted me to check how much CPU time the RNG pump consumes, so I did some profiling.

So 1% of the emulation time is spent just advancing the RNG. Is it worth limiting the pump in the same way Stella does? I'm not sure. I prefer implementing it as described in supporting documentation as closely as I can. But my constraints are different to the Stella teams I suppose. I don't have to worry about it running on embedded hardware for example.

Still it's something to consider especially as we can't know whether the RNG sequence is exactly the same as on hardware.

[ale-79]

Where does this new dump come from?
All the "Pitfall II" roms I have (2 NTSC and one PAL) are 10495 bytes.
I believe they were originally obtained using the 7800 "DevOs" dumper by Eckhard Stolberg.

This is from the source of the PC utility:

void dump_P2BS(void)
{
unsigned int z;
   for(z=0; z<2; z++)
   {
      write_byte(0xfff8+z, z);
      dump_noBS(15, 0xf0);
      read_buffer(0xff00, 0xf8);
      bufcount=bufcount+2;
      read_buffer(0xfffa, 6);
      printf("%3u\r",bufcount>>8);
   }
   write_byte(0xf050, 0xff);	/*counter for extra ROM low byte value*/
   write_byte(0xf058, 0x07);	/*counter for extra ROM high byte value*/
   for(z=0; z<8; z++)
   {
      read_onespot(0xf008, 256);
      printf("%3u\r",bufcount>>8);
   }
   write_byte(0xf070, 0x00);	/* reset random number generator */
   read_onespot(0xf000, 255);
   printf("\nadding 255 bytes of random number generator data");
}

devkit2.zip
So the sequence comes from the cartridge.

The 255 RNG sequence in the known dump is this one:

00 01 03 07 0f 1e 3d 7a  f4 e8 d0 a1 43 87 0e 1c
39 72 e5 cb 97 2f 5f bf  7f fe fd fb f7 ee dc b8
71 e3 c7 8e 1d 3b 76 ed  da b4 68 d1 a3 47 8f 1f
3f 7e fc f9 f3 e6 cd 9b  36 6d db b6 6c d9 b2 64
c8 91 23 46 8d 1b 37 6f  df be 7d fa f5 ea d4 a9
52 a4 49 92 25 4a 94 29  53 a6 4d 9a 34 69 d3 a7
4f 9e 3c 78 f0 e0 c1 82  04 09 12 24 48 90 21 42
85 0a 14 28 51 a2 45 8b  17 2e 5d bb 77 ef de bc
79 f2 e4 c9 93 27 4e 9c  38 70 e1 c3 86 0c 18 31
63 c6 8c 19 33 67 ce 9d  3a 74 e9 d2 a5 4b 96 2d
5b b7 6e dd ba 75 eb d6  ad 5a b5 6a d5 ab 56 ac
58 b1 62 c4 88 11 22 44  89 13 26 4c 98 30 61 c2
84 08 10 20 40 81 02 05  0b 16 2c 59 b3 66 cc 99
32 65 ca 95 2b 57 ae 5c  b9 73 e7 cf 9f 3e 7c f8
f1 e2 c5 8a 15 2a 55 aa  54 a8 50 a0 41 83 06 0d
1a 35 6b d7 af 5e bd 7b  f6 ec d8 b0 60 c0 80   

It contains every number from "00" to "fe" without repetitions.
In the new "good" dump the byte "fd" is appended at the end, so it's now repeated twice. Shouldn't the sequence repeat itself after a value is returned the second time? What about the values that precede the first occurrance of "fd"? You only gets those once and then never again?

[trebor68]

It may be useful, I gathered what information I found and posted here, earlier last year to the MAME/MESS forum: https://forums.bannister.org/ubbthreads.php?ubb=showflat&Number=123431

Posting below as well for convenience:
"The "missing" 255 bytes tacked on the end of the known dump is some RNG data generated by the DPC chip. The RNG aka "poly" bytes do not exist in the ROM so they are not included in the dump."
SOURCE: sanni/cartreader#801

"DPC uses a 10K ROM, which is comprised of two 4K banks plus an additional 2K that's used to store graphics data. Selection between the 4K banks is done via standard F8 bank switching, while the 2K is not directly accessible to the 6502."
SOURCE: https://forums.atariage.com/topic/159670-dpc-sprite-demo/

DPC chip patent
"Title:
Video memory system
United States Patent 4644495

Abstract:
An improved video memory system includes a program memory, a display data memory, and a plurality of data fetchers. The data fetchers are used to indirectly address the display data in the display data memory. The data fetchers are programmed during vertical blanking so that selected display data is fetched at selected vertical display positions. During each scan line each data fetcher is "read" by: (1) decrementing a counter in the data fetcher; (2) comparing the counter value against preselected top and bottom values; and (3) using the counter value to indirectly address display data that is to be displayed on the current scan line if the counter value is between the top and bottom values. This relieves the host computer of having to keep track of the current vertical display position, thereby freeing it to use the saved computer cycles to produce more interesting viedo games with more complex display graphics. In addition there are provided data fetchers that can be programmed to periodically issue signals usable either for drawing a line with a predefined slope or modulating the amplitude of a sound generator."
SOURCE: https://www.freepatentsonline.com/4644495.html

"You can search for the patent on the chip (#4644495) which I think gives a high-level description of the various functions. I didn't find the patent quite as useful as emulator source code when trying to figure out how the DPC worked so I could simulate it in hardware (the Harmony cart is the only cart that can simulate the DPC chip, or at least the only one that has.)

The DPC has some features that are not used by Pitfall II. I'm not certain what all of those are. My own breakdown of the various functions is as follows, at least as used in Pitfall II:

Read registers:
$1000-$1004: Random number generator. Reads return an 8-bit random number.
$1005-$1007: Music fetcher. Reads return the current amplitude of the multiple-voice music waveform that will be written into the 2600's sound volume register. Typically this will be read at least once every scanline.
$1008-$100F: Display data reads. Reads from one of 8 data streams containing graphics, and increments the graphics pointer.
$1010-$1017: Display data reads ANDed with flag. Reads from one of 8 data streams containing graphics, ANDs the data with a flag, and increments the graphics pointer. Flag is $FF if the low byte of the graphics pointer is between "top" and "bottom" register (see below) or $00 otherwise (this isn't the whole truth - if the low byte of the pointer is stored to a value between "top" and "bottom" then the flag will be $00.)
$1018-$101F: Same as $1010-$1017, but nybbles are swapped. Not used by Pitfall II.
$1020-$101F: Same as $1010-$1017, but bits in the byte are reversed. Not used by Pitfall II.
$1018-$101F: Same as $1010-$1017, but byte is shifted one bit right. Not used by Pitfall II.
$1018-$101F: Same as $1010-$1017, but byte is shifted one bit left. Not used by Pitfall II.
$1038-$103F: Return the current value of the flag. Apparently not used by Pitfall II.

Write registers:
$1040-$1044: Write to "top" register.
$1045-$1047: Write music frequency of channel 1-3. Frequency is 20 kHz/value
$1048-$104F: Write to "bottom" register.
$1050-$1057: Write to low byte of graphics pointer (pointer is an address in a 2k graphics ROM)
$1058-$105C: Write to high byte of graphics pointer (appears to only store the lower 3 bits)
$105D-$105F: Control byte for music channel (on/off)
$1060-$106F: Apparently not used by Pitfall II, but may have another function.
$1070-$1077: Random number reset. Apparently not used by Pitfall II.
$1078-$107F: Apparently not used by Pitfall II, but may have another function.

There is also a "draw line" function that is not used by Pitfall II. It apparently calculates a value to store in HMOVE to help with drawing lines. I think it may have been originally designed for a vine but that never made it to the game."
SOURCE: https://forums.atariage.com/topic/1...work/?do=findComment&comment=1750556

"Pitfall II for the Atari has a DPC chip which generates RNG tables (that I don't believe exist until it generates them), which it uses to make the eel enemies flicker (I think that's all). Current dumps out there, and what's on the DOM (1 yellow trusted, 1 third party) have the 255 bytes that is the generated RNG data appended to the end of the dump. This is what emulators that know ROMs will identify as Pitfall II as well. It also does something to the music on every scanline, the chip has a few functions, but only RNG tables are added to the dump.

The OSCR (which I use to dump) is intentionally not having the RNG table data read as it isn't part of the game ROM, and probably doesn't exist until the chip creates it and should be emulated by emulators like other chips (Super FX etc,) along with the RNG table generation code it runs, rather than packing the game with the resulting tables, though emulators don't typically do this from what I've seen. I understand the Harmony Cart can emulate it, but I'm not sure if it needs RNG data from the ROM to do so.

Also without the RNG tables it is 10,240 (a proper 10kb), but the current one is 255 bytes bigger than that."
SOURCE: https://forum.no-intro.org/viewtopic.php?t=7137&sid=6d83c76edcb76e6f7b2c4902a8dce4d0

[JetSetIlly]

I've added a test for the DPC RNG and use the data in the ROM dumps to compare against.

https://github.com/JetSetIlly/Gopher2600/blob/796fc05cd16ef826fb754925d8a3d61d2c5def72/hardware/memory/cartridge/mapper_dpc_test.go

In answer to my own question above: Yes, Pitfall2 implements the RNG as it is described in the patent.

@ale-79
In the new "good" dump the byte "fd" is appended at the end, so it's now repeated twice. Shouldn't the sequence repeat itself after a value is returned the second time? What about the values that precede the first occurrance of "fd"? You only gets those once and then never again?

Yes, the value 0x80 should be followed by 0x00.

Each value in the sequence should only appear once and 0xff is not a part of the sequence.

0xff is an impossible value and can never be mutated into another value using the method described in the patent.

So I don't think the "bad" dump is actually bad. If anything the "good" dump is more of a mystery because of the additional 0xfd.

We can also say that the terminating byte of 0x0a is unnecessary too.