Page 1 of 2

Punter File Transfer - Image BBS 3.0 / Vice 3.6.1 / tcpser

Posted: Tue May 03, 2022 2:10 pm
by Cube Inc
I set up an Image 3.0 BBS last night for the first time and WOW - am I impressed. I mean, just look at this:

Image - The ability to customize which masking characters are used to cover up the users' password as they type when logging on? That is attention to detail!

I wanted to try some file transfers, which seemed to be working when using CCGMS 2021 in one Vice session to another CCGMS 2021 running in a different Vice session, both using their own instances of tcpser on separate ports. Transferring in either direction in this configuration worked without a single bad block, and at 38400 no less. (Though it did seem to pause a lot between blocks... I'm not certain it was transferring at the full 38.4 kbps, but regardless - the files went through perfectly.) When I tried uploading from CCGMS to Image BBS 3.0 however, I could sometimes get one "good" block, and the rest were bad. Watching the data scroll across each screen I could see that things were different on each pass of what should have been the same block, so clearly some data was being dropped / corrupted along the way, and not in the same place or fashion on each pass.

I decided to look at what Vice was sending and receiving at each end, and here is what I found:

Image - Left: Vice running CCGMS. Right: Vice running Image BBS 3.0

On the left, the CCGMS side, it tries to send four 0xff bytes to the BBS, but the BBS receives only two 0x7f bytes. 0xff is a magic byte, both for the IP232 protocol and for the Telnet NVT protocol, so back into the source of tcpser I go to try and figure out what instruction is converting these literal 0xff bytes into misinterpreted, and then mis-translated 0x7f bytes.

My guess is that tcpser is operating in ASCII mode and therefore stripping off the MSB of the even numbered 0xff bytes after discarding the odds as (erroneous) NVT_IAC bytes.

... Actually, it looks like both tcpser instances are negotiating for binary transfer mode on connect, which is good, but they keep bouncing back and forth another NVT command incessantly, like one side is reiterating the command instead of acknowledging it... This is intriguing; Still digging...

Image

Re: Punter File Transfer - Image BBS 3.0 / Vice 3.6.1 / tcpser

Posted: Tue May 03, 2022 5:25 pm
by Cube Inc
After fixing a bit of a strange Telnet NVT issue where (ironically) both instances of tcpser would fire back and forth NVT echo negotiation commands as fast as they possibly could, I was able to confirm that each byte the CCGMS "Vice" was sending out was being received by the Image BBS "Vice". Still, I could see bytes missing or changing on each subsequent pass of the block transfers on the Image screen. Basically, it looked like Image kept "dropping" one or more byte in the middle of the block, so the remaining bytes would then get shifted.

Here's the interesting part: In the middle of the transfer, after having 45 or so bad blocks - I put the Image BBS Vice into warp mode and the rest of the file transferred successfully, all the way to completion of the 129 block file.

This suggests to me that whatever is preventing the bytes from being interpreted properly within Image BBS (I was using the UX area) does so because something is taking too long to process or the data is arriving too quickly for all of it to be read before the buffer overflows. Evidently it is able to keep up to the data flow when in warp mode, which makes me think that something is just not fast enough when running it normally. Maybe that has to do with the fact Vice is emulating a CMD hard drive and/or RAMLink, I do not yet know. I am pretty sure I remember seeing in the ACIA code in Vice that it only grabs one byte at a time. I'm going to take a look there and see if anything else leaps out that might help.

Image

Re: Punter File Transfer - Image BBS 3.0 / Vice 3.6.1 / tcpser

Posted: Tue May 03, 2022 6:35 pm
by Cube Inc
I modified Vice to log any ACIA overruns, and then tried uploading a file to Image under Vice. Guess what?

Code: Select all

RS232NET: FD[0]: Input 0x36 '6'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x4c 'L'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x57 'W'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x62 'b'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0xe2 '.'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
ACIA Overrun! Discarding received byte [e2] 
RS232NET: FD[0]: Input 0xf5 '.'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x50 'P'. 1
RS232NET: FD[0]: Input 0x10 '.'. 1
RS232NET: FD[0]: Input 0x5f '_'. 1
RS232NET: FD[0]: Input 0x10 '.'. 1
RS232NET: FD[0]: Input 0x7a 'z'. 1
RS232NET: FD[0]: Input 0x10 '.'. 1
RS232NET: FD[0]: Input 0x9f '.'. 1
RS232NET: FD[0]: Input 0x10 '.'. 1
RS232NET: FD[0]: Input 0xeb '.'. 1
RS232NET: FD[0]: Input 0x11 '.'. 1
RS232NET: FD[0]: Input 0xf2 '.'. 1
RS232NET: FD[0]: Input 0x11 '.'. 1
RS232NET: FD[0]: Input 0xf9 '.'. 1
RS232NET: FD[0]: Input 0x11 '.'. 1
RS232NET: FD[0]: Input 0x02 '.'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x0b '.'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x27 '''. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
ACIA Overrun! Discarding received byte [27]
RS232NET: FD[0]: Input 0x3b ';'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x5d ']'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x6e 'n'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0xcc '.'. 1
RS232NET: FD[0]: Input 0x1b '.'. 1
RS232NET: FD[0]: Input 0x7e '~'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x87 '.'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x92 '.'. 1
RS232NET: FD[0] Output 0x53 'S'.
RS232NET: FD[0] Output 0x2f '/'.
RS232NET: FD[0] Output 0x42 'B'.
RS232NET: FD[0]: Input 0x41 'A'. 1
RS232NET: FD[0]: Input 0x43 'C'. 1
RS232NET: FD[0]: Input 0x4b 'K'. 1
RS232NET: FD[0] Output 0x53 'S'.
RS232NET: FD[0] Output 0x2f '/'.
RS232NET: FD[0] Output 0x42 'B'.
RS232NET: FD[0]: Input 0x8f '.'. 1
RS232NET: FD[0]: Input 0x48 'H'. 1
RS232NET: FD[0]: Input 0xb8 '.'. 1
RS232NET: FD[0]: Input 0xf7 '.'. 1
RS232NET: FD[0]: Input 0xff '.'. 1
RS232NET: FD[0]: Input 0xff '.'. 1
RS232NET: FD[0]: Input 0x01 '.'. 1
RS232NET: FD[0]: Input 0x00 '.'. 1
RS232NET: FD[0]: Input 0x01 '.'. 1
RS232NET: FD[0]: Input 0x08 '.'. 1
RS232NET: FD[0]: Input 0x1c '.'. 1
RS232NET: FD[0]: Input 0x08 '.'. 1
RS232NET: FD[0]: Input 0xc5 '.'. 1
RS232NET: FD[0]: Input 0x07 '.'. 1
RS232NET: FD[0]: Input 0x9e '.'. 1
RS232NET: FD[0]: Input 0x32 '2'. 1
RS232NET: FD[0]: Input 0x30 '0'. 1
RS232NET: FD[0]: Input 0x37 '7'. 1
RS232NET: FD[0]: Input 0x38 '8'. 1
RS232NET: FD[0]: Input 0x14 '.'. 1
RS232NET: FD[0]: Input 0x14 '.'. 1
RS232NET: FD[0]: Input 0x14 '.'. 1
RS232NET: FD[0]: Input 0x14 '.'. 1
RS232NET: FD[0]: Input 0x14 '.'. 1
RS232NET: FD[0]: Input 0x14 '.'. 1
RS232NET: FD[0]: Input 0x14 '.'. 1
RS232NET: FD[0]: Input 0x4c 'L'. 1
RS232NET: FD[0]: Input 0x59 'Y'. 1
RS232NET: FD[0]: Input 0x4e 'N'. 1
RS232NET: FD[0]: Input 0x58 'X'. 1
RS232NET: FD[0]: Input 0x20 '.'. 1
RS232NET: FD[0]: Input 0x58 'X'. 1
RS232NET: FD[0]: Input 0x56 'V'. 1
RS232NET: FD[0]: Input 0x49 'I'. 1
RS232NET: FD[0]: Input 0x49 'I'. 1
RS232NET: FD[0]: Input 0x20 '.'. 1
RS232NET: FD[0]: Input 0x00 '.'. 1
RS232NET: FD[0]: Input 0x00 '.'. 1
RS232NET: FD[0]: Input 0x00 '.'. 1
RS232NET: FD[0]: Input 0x6c 'l'. 1
RS232NET: FD[0]: Input 0x97 '.'. 1
RS232NET: FD[0]: Input 0x1d '.'. 1
RS232NET: FD[0]: Input 0x4c 'L'. 1
RS232NET: FD[0]: Input 0xad '.'. 1
RS232NET: FD[0]: Input 0x1a '.'. 1
RS232NET: FD[0]: Input 0x0c '.'. 1
RS232NET: FD[0]: Input 0xbc '.'. 1
RS232NET: FD[0]: Input 0xb4 '.'. 1
RS232NET: FD[0]: Input 0xbf '.'. 1
RS232NET: FD[0]: Input 0x58 'X'. 1
RS232NET: FD[0]: Input 0xbc '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0xe3 '.'. 1
RS232NET: FD[0]: Input 0x64 'd'. 1
RS232NET: FD[0]: Input 0xe2 '.'. 1
RS232NET: FD[0]: Input 0xed '.'. 1
RS232NET: FD[0]: Input 0xbf '.'. 1
RS232NET: FD[0]: Input 0xea '.'. 1
RS232NET: FD[0]: Input 0xb9 '.'. 1
RS232NET: FD[0]: Input 0x97 '.'. 1
RS232NET: FD[0]: Input 0xe0 '.'. 1
RS232NET: FD[0]: Input 0x39 '9'. 1
RS232NET: FD[0]: Input 0xbc '.'. 1
RS232NET: FD[0]: Input 0x6b 'k'. 1
RS232NET: FD[0]: Input 0xe2 '.'. 1
RS232NET: FD[0]: Input 0x8c '.'. 1
RS232NET: FD[0]: Input 0x1c '.'. 1
RS232NET: FD[0]: Input 0xb4 '.'. 1
RS232NET: FD[0]: Input 0xe2 '.'. 1
RS232NET: FD[0]: Input 0x10 '.'. 1
RS232NET: FD[0]: Input 0x03 '.'. 1
RS232NET: FD[0]: Input 0x3b ';'. 1
RS232NET: FD[0]: Input 0x1d '.'. 1
RS232NET: FD[0]: Input 0xcc '.'. 1
RS232NET: FD[0]: Input 0xbc '.'. 1
RS232NET: FD[0]: Input 0xb5 '.'. 1
RS232NET: FD[0]: Input 0xab '.'. 1
RS232NET: FD[0]: Input 0x7b '{'. 1
RS232NET: FD[0]: Input 0x1d '.'. 1
RS232NET: FD[0]: Input 0xc5 '.'. 1
RS232NET: FD[0]: Input 0x1b '.'. 1
RS232NET: FD[0]: Input 0x7c '|'. 1
RS232NET: FD[0]: Input 0x09 '.'. 1
RS232NET: FD[0]: Input 0x8f '.'. 1
RS232NET: FD[0]: Input 0x09 '.'. 1
RS232NET: FD[0]: Input 0xa2 '.'. 1
RS232NET: FD[0]: Input 0x09 '.'. 1
RS232NET: FD[0]: Input 0xb5 '.'. 1
RS232NET: FD[0]: Input 0x09 '.'. 1
RS232NET: FD[0]: Input 0x1a '.'. 1
RS232NET: FD[0]: Input 0x1c '.'. 1
RS232NET: FD[0]: Input 0xbe '.'. 1
RS232NET: FD[0]: Input 0x09 '.'. 1
RS232NET: FD[0]: Input 0xca '.'. 1
RS232NET: FD[0]: Input 0x09 '.'. 1
RS232NET: FD[0]: Input 0xd3 '.'. 1
RS232NET: FD[0]: Input 0x09 '.'. 1
RS232NET: FD[0]: Input 0xdc '.'. 1
RS232NET: FD[0]: Input 0x09 '.'. 1
RS232NET: FD[0]: Input 0xed '.'. 1
RS232NET: FD[0]: Input 0x09 '.'. 1
RS232NET: FD[0]: Input 0xfe '.'. 1
RS232NET: FD[0]: Input 0x09 '.'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x15 '.'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x1c '.'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x24 '$'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x2b '+'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x33 '3'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x3a ':'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x44 'D'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x4b 'K'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x54 'T'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x5a 'Z'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x60 '`'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
ACIA Overrun! Discarding received byte [60] 
RS232NET: FD[0]: Input 0x67 'g'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x73 's'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x7f '.'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0xb0 '.'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0xdb '.'. 1
RS232NET: FD[0]: Input 0x0a '.'. 1
RS232NET: FD[0]: Input 0x02 '.'. 1
RS232NET: FD[0]: Input 0x0b '.'. 1
RS232NET: FD[0]: Input 0x0b '.'. 1
RS232NET: FD[0]: Input 0x0b '.'. 1
RS232NET: FD[0]: Input 0x14 '.'. 1
RS232NET: FD[0]: Input 0x0b '.'. 1
RS232NET: FD[0]: Input 0xd2 '.'. 1
RS232NET: FD[0]: Input 0x1b '.'. 1
RS232NET: FD[0]: Input 0x1a '.'. 1
RS232NET: FD[0]: Input 0x0b '.'. 1
RS232NET: FD[0]: Input 0x4a 'J'. 1
RS232NET: FD[0]: Input 0x0b '.'. 1
RS232NET: FD[0]: Input 0x76 'v'. 1
RS232NET: FD[0]: Input 0x0b '.'. 1
ACIA Overrun! Discarding received byte [76] 
RS232NET: FD[0]: Input 0x95 '.'. 1
RS232NET: FD[0]: Input 0x0b '.'. 1
RS232NET: FD[0]: Input 0xa4 '.'. 1
RS232NET: FD[0]: Input 0x0b '.'. 1
RS232NET: FD[0]: Input 0x04 '.'. 1
RS232NET: FD[0]: Input 0x0c '.'. 1
RS232NET: FD[0]: Input 0x61 'a'. 1
RS232NET: FD[0]: Input 0x0c '.'. 1
RS232NET: FD[0]: Input 0x8d '.'. 1
RS232NET: FD[0]: Input 0x0c '.'. 1
RS232NET: FD[0]: Input 0xbe '.'. 1
RS232NET: FD[0]: Input 0x0c '.'. 1
RS232NET: FD[0]: Input 0xe1 '.'. 1
RS232NET: FD[0]: Input 0x0c '.'. 1
RS232NET: FD[0]: Input 0x19 '.'. 1
RS232NET: FD[0]: Input 0x0d '.'. 1
RS232NET: FD[0]: Input 0x37 '7'. 1
RS232NET: FD[0]: Input 0x0d '.'. 1
RS232NET: FD[0]: Input 0x42 'B'. 1
RS232NET: FD[0]: Input 0x0d '.'. 1
RS232NET: FD[0]: Input 0x76 'v'. 1
RS232NET: FD[0]: Input 0x0d '.'. 1
RS232NET: FD[0]: Input 0x97 '.'. 1
RS232NET: FD[0]: Input 0x0d '.'. 1
ACIA Overrun! Discarding received byte [97]
RS232NET: FD[0]: Input 0x11 '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0x1f '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0x78 'x'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0x85 '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0x92 '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0x9f '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0xad '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0xba '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0xc7 '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0xd4 '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0xe1 '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
RS232NET: FD[0]: Input 0xef '.'. 1
RS232NET: FD[0]: Input 0x0e '.'. 1
ACIA Overrun! Discarding received byte [ef]
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x16 '.'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x2f '/'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x26 '&'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x1f '.'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x36 '6'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x4c 'L'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x57 'W'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x62 'b'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0xe2 '.'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0xf5 '.'. 1
RS232NET: FD[0]: Input 0x0f '.'. 1
RS232NET: FD[0]: Input 0x50 'P'. 1
RS232NET: FD[0]: Input 0x10 '.'. 1
ACIA Overrun! Discarding received byte [50]
RS232NET: FD[0]: Input 0x5f '_'. 1
RS232NET: FD[0]: Input 0x10 '.'. 1
RS232NET: FD[0]: Input 0x7a 'z'. 1
RS232NET: FD[0]: Input 0x10 '.'. 1
RS232NET: FD[0]: Input 0x9f '.'. 1
RS232NET: FD[0]: Input 0x10 '.'. 1
RS232NET: FD[0]: Input 0xeb '.'. 1
RS232NET: FD[0]: Input 0x11 '.'. 1
RS232NET: FD[0]: Input 0xf2 '.'. 1
RS232NET: FD[0]: Input 0x11 '.'. 1
RS232NET: FD[0]: Input 0xf9 '.'. 1
RS232NET: FD[0]: Input 0x11 '.'. 1
RS232NET: FD[0]: Input 0x02 '.'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x0b '.'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x27 '''. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x3b ';'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0x5d ']'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
ACIA Overrun! Discarding received byte [5d]
RS232NET: FD[0]: Input 0x6e 'n'. 1
RS232NET: FD[0]: Input 0x12 '.'. 1
RS232NET: FD[0]: Input 0xcc '.'. 1
RS232NET: FD[0]: Input 0x1b '.'. 1
RS232NET: FD[0]: Input 0x7e '~'. 1
I wonder if there's a way to have Vice hold the incoming byte until the next rx cycle.

Image

Re: Punter File Transfer - Image BBS 3.0 / Vice 3.6.1 / tcpser

Posted: Wed May 04, 2022 12:09 am
by Cube Inc
Well what do you know, I tweaked Vice's ACIA mechanism so that it only requests a new byte from the IP socket once the ACIA data register has been read (by Image or some other program interacting with the modem), and managed to do a multi-file upload of several hundred blocks before Image seems to have gotten stuck waiting for the last file to start, where CCGMS seems to have thought there was no other files to send. Not sure what happened there, I went away to check on something else and came back to a screen of GOO's in CCGMS and Image waiting for a file. Cancelled out with CTRL-X and uploaded more files, but this time the transfer seemed to stop after 34 blocks or so, and the BBS side is just receiving 'ACK' messages repeated about once / second. I am not sure what happened there either, but strangely enough the transfer seemed to complete and both sides moved onto the next file like nothing unusual had just happened. That seemed very odd, but it's been a long time since I've watched a file transfer very closely...

Basically this change gives the modem driver software on the C64 time to pick up each byte from the ACIA's data register. If the current byte in the register has not yet been pulled, Vice does not request more data from the socket connected to tcpser. Theoretically if tcpser tried to "push" data (IE: RINGs) but there was no software to retrieve that data, I think the worst that might happen would be that tcpser would eventually block :oops: :( :| :mrgreen: :geek: :ugeek: :twisted: :evil: :roll: 8-) :? :shock: :o :lol: :x :P :!: :?: :idea: :arrow: ;) :) :D :cry: <-- Those were added by my daughter, which sounds very cute but she's seventeen. haha. I love her... :oops: Ok, now that we've both attempted to make every single one of those faces ourselves, where was I? Oh yeah, file transfers.

So it would seem that there is a bug in either CCGMS or Image, and I'd lean towards CCGMS 2021 because it seems to end multi file transfers one file short. I just did a multi-upload of 14 files consecutively, but 15 files had been selected for transfer. Image was still waiting for the last file to start, and CCGMS was back at the terminal menu. Maybe I'll have to try a different term program with Image, but so far these preliminary results seem promising... At least so far it seems to have fixed the uploads to Image in Vice 3.6.1... I haven't tried downloading from Image yet, but maybe I will leave that for another day.

Image

Re: Punter File Transfer - Image BBS 3.0 / Vice 3.6.1 / tcpser

Posted: Thu May 05, 2022 12:03 pm
by Cube Inc
Good morning,

Uploading to Image 3.0 under Vice 3.6.1 seems to be working nicely, I've tried it now with CCGMS 2021 and CCGMS 2020 using an ACIA-emulated modem at 38400, and from CCGMS RLE+Music with an emulated Userport modem at 2400 baud. My user port modifications in Vice still need a bit more tweaking as still takes two characters being sent to the modem from the '64 before Vice actually connects, whereas with the ACIA code Vice attempts to reconnect to tcpser from the moment the connection is closed until it re-establishes the connection so that even the first "a" of an "at" command typed into whatever terminal is running in the '64 is echoed back by the tcpser modem. It works very nicely.

Downloading from Image TO anything is still inoperable in any of the terminal programs and modem types I've tried, and I believe it has to do with the IP232 protocol implementation in Vice and possibly in tcpser. I need to get out my notebook and diagram the entire process, especially [what takes place where] as there are portions of the process still a bit fuzzy in my mind. What is clear though from watching the bytes leaving and entering each Vice instance is that when Image goes to send a file to a terminal, the receiving end is getting too many 0xff bytes, resulting in a BAD block report, which is technically legitimate.

One thing I believe I have fixed in tcpser at this point has to do with it's implementation of Telnet's NVT protocol. In brief, Telnet has some of the same challenges as we do, trying to send modem status lines (DCD, DSR, DTR, CTS, RTS, RING, etc.) through the same medium as the actual data. In fact, the NVT protocol uses the very same approach as the IP232 protocol in that when one Telnet client needs to tell the other Telnet client about a feature it supports or to request that the other terminal echoes back typed characters, this is accomplished by first sending an 0xff byte, and then an action command and an option byte. The other Telnet client watches the incoming stream of data for these sequences and pulls them out of the stream before passing the remaining data onto the screen or printer.

Telnet, by default, uses 7-bit character encoding which is fine for text and most graphics - even when passing Commodore Graphics (CG) characters, but file transfers require 8-bit character encoding in order to work properly. This is why I was seeing those 0xff bytes (Binary: 0b11111111) being converted to 0x7f bytes earlier (Binary 0b01111111) - the highest bit gets lopped off and you end up with a "corrupt" or bad block. To make sure this conversion doesn't happen, on connection tcpser sends the "Enable Binary Mode" NVT command to the remote client and requests that it do the same, so that all data exchanged is treated as 8-bit binary data. What I was observing in practice however, is that only some of the time tcpser ran would the Binary mode get engaged. When it did, uploads worked fine. When it didn't, I couldn't even get the first block to transfer. It turned out that tcpser did not initialize some of the data structures that stored information about the state of the line connection, including the NVT flags. Uninitialized, those registers would contain random "garbage" data from the operating system that could change each time tcpser was run; sometimes that random data contained bytes that made it favorable to enable the binary mode, other times it contained data that caused the binary mode to not get enabled. I have modified tcpser now to initialize all the line registers properly, and it now goes into binary mode every time it connects to another client.

There have been multiple hindrances to successful communication, spread across many files in both tcpser and Vice, but I think I am narrowing in on 100% reliable communication...

Image

Re: Punter File Transfer - Image BBS 3.0 / Vice 3.6.1 / tcpser

Posted: Tue May 10, 2022 8:53 am
by hendrb01
If you can compile your TCPser version for Windows, I would not mind giving it a test on The Rat's Den for a few days.

While BUCKO has had luck with v1.14 for Windows, I was never able to, my network transfers with RTG would be hit or miss, and I consider RAD's participation in the NISA net to be of higher priority than file transfers. Though it would be very nice to get both working!

Re: Punter File Transfer - Image BBS 3.0 / Vice 3.6.1 / tcpser

Posted: Tue May 10, 2022 11:17 am
by Cube Inc
I understand your dilemma, and I'd tend to agree with you... Reliability is important. I do hope to be able to achieve both reliability AND some nice-to-have functionality.

Over the past couple of days I've spent a significant amount of time tracing all the IP232 handling of 0xff bytes through Vice and tcpser, and then of tcpser's NVT handling of 0xff bytes, and then verifying and double-checking that byte-for-byte, every single character that Image BBS was sending out of it's emulated modem was being correctly passed through and interpreted by the virtual modem in a second instance of Vice and that those bytes were being correctly passed into the emulated terminal (CCGMS 2021). Despite all this, I was still unable to get multi-Punter downloads to work from Image 3.0 to CCGMS 2021. CCGMS would get the filename and then sit there doing nothing -- all the while IMAGE is sending GOO over and over again indefinitely. Oddly enough, if I hit C= in CCGMS to "Abort" the transfer, it would then switch to the receiving screen, appear to start actually exchanging bytes - but all blocks turned out bad. Also it was worth noting that

I confirmed again that byte-for-byte, the data being exchanged was 100% accurate.

On a whim I decided to try a single download instead of multi, and low and behold it worked just fine! In fact, while Image is trying to send multiple files, if in CCGMS I treat each one as a manual download, it works just fine also. (Albeit more tedious).

The prudent thing for me to do at this point would be to learn myself up on the Punter protocol itself, but my gut feeling at the moment is that my problem may now be something to do with either the way Image is initiating the multi-download transfer OR the way CCGMS is responding to the multi-download transfer, and at this stage I'm leaning towards CCGMS since it just feels buggy - having to hit "abort" in order to start the transfer, etc. I am going to try another couple of terminals, probably Novaterm and maybe Handyterm on an emulated userport modem, and see how they fare with these same tests. I will report back with the findings, but we are SOOOO CLOSE!

Incidentally, while I thought the hurdle would be trying to compile for Windows on Linux, it turned out that the real challenge was not that part - that was relatively straight forward, but where I've really had a go of it has been getting the compiled executables to actually run on Windows! Technically I suppose they do in fact run, only they don't seem to actually do anything - whether starting from a command prompt or double-clicking on the icon I get the spinny thing (hourglass cursor) for a moment or two and then nothing. No errors, no messages, but no friendly blue 38,911 Basic Bytes Free either.

I wondered if my build environment was too old (it was quite old to be honest) so I spun up a Fedora 36 LXC container, installed all the requisite cross-compilation [stuff] and tried building it there. I haven't yet had a change to test those new .exe's on Windows 10 but I will do that later on today. I am highly optimistic though. I'll try to post a complete walk-through of the process too if anyone is interested.

... I just had CCGMS crash outright (went back to blue screen, "READY." prompt) but then another thought occurred to me - I am running both instances of Vice with a RAMLink, and the other day I discovered that I couldn't even start Handyterm with the RAMLink "plugged in". I should probably try - at least the terminal tests - with just the stock '64 to see if that makes any difference.

... It didn't. At least not with CCGMS 2021. What is still odd is that it appears the characters which are sent from Image look, in each emulation window, different when trying a Multi-Download. When I do a single Punter download, the two start blocks look identical, and that's how I know right away it is going to work.

This is what I see when trying to start a Punter multi-file transfer from Image 3.0 to CCGMS 2021:
Image

This is what I see when trying to start a Punter single file transfer from Image 3.0 to CCGMS 2021:
Image

If those two start blocks align, the transfer is going to succeed:
Image

Again - I have confirmed that byte-for-byte, the data being exchanged between the two virtual modems is identical in both directions.

Let's test a different terminal - Let's see what Novaterm does when I try to do a multi-file download from Image:

Image

Well gosh darn diddly ding dong CRAP! Novaterm happily downloaded all the files in multi-Punter mode without so much as a hiccup. It was CCGMS 2021 that was doing something funny!

I've now tested and confirmed: Multi-Punter uploads and downloads work perfectly with Novaterm 9.6c to and from Image 3.0 now, so maybe I'll try the same thing with Image Term itself to make sure it's happy with the "connection" also, since that's probably as close to doing a network transfer as I can simulate without actually joining the NISSA network, but it looks like my modifications to Vice and tcpser have fixed the file transfer issue in both directions.

Image

Re: Punter File Transfer - Image BBS 3.0 / Vice 3.6.1 / tcpser

Posted: Tue May 10, 2022 1:36 pm
by Cube Inc
Yes, Image Term to Image BBS 3.0 appears to work perfectly as well.

Image
Image BBS on the left - Image Term on the right, in the middle of a Punter Multi-File Download.

Image

Re: Punter File Transfer - Image BBS 3.0 / Vice 3.6.1 / tcpser

Posted: Tue May 10, 2022 2:28 pm
by Cube Inc
I tried a Punter multi-file download from Image 3.0 with CCGMS 2020 and ended up with a CPU JAM as soon as the file transfer started:

Image

This is the same spot in CCGMS 2021 where I would have had to press C= (Abort) to get it to proceed to the next screen where it would seem to then attempt to start the transfer, but never successfully. Interesting that both terminal programs glitch at this spot.

Still testing some other terminal programs between calls.

Image

Re: Punter File Transfer - Image BBS 3.0 / Vice 3.6.1 / tcpser

Posted: Tue May 10, 2022 7:33 pm
by Cube Inc
I had a steep learning curve today after I abandoned my attempts to compile these programs for Windows on Linux, and decided to try to compile them both on Windows. TCPser went through with no problem, ran the executable, got the help menu... Looks good so far, but I hit road block after speed bump after toll booth trying to make headway on getting it compiled on Windows. It's processing [yet again] now, and - at least on my setup - compiling under Windows is several orders of magnitude slower than compiling on the same hardware, running Linux - but we'll leave those discussions for another time and place. ;- P Funny enough, my google searches lead me to this post: https://sourceforge.net/p/vice-emu/mail ... gmail.com/ Where someone else had pre-traced my steps to a "T"! And in the end, one message in that thread held the key to moving forward, along with the accompanying face palm. I guess I was trying to do too many things at the same time today and completely missed one key step.

Getting closer...

Image