Carryall Data (Source Included)
BlitzMax Forums/BlitzMax Programming/Carryall Data (Source Included)
| ||
| I don't know if this has been done yet. This program will store and recall binary files directly from the text source, useful for BlitzMAX forum posting. In GFA, the editor was so permissive it let me store raw characters from 32-255 in data lines so I managed to make a nice 7-bit per character. BlitzMAX's IDE is not so forgiving, so for simplicity sake and to prevent future problems, I chose hexadecimal, 4-bits per character, 2-characters to make one byte of data. If there is a need for deeper compression of in-source data, I'm open to options. Click inside and press CTRL-A and CTRL-C to copy it to the clipboard. Then in BlitzMAX, press CTRL-N for a new program and CTRL-V to paste this program.  |
| ||
| This is not bad. It takes me back to the old days of Compute! Magazine, Incider et al. One possible (possibly paranoid) concern here is that this sort of embedded binary in source might be usable as an attack vector by someone. If you wanted to compress certain types of data you might go for run-length encoding. I also like your vector art encoding method. |
| ||
| How does this differ from base64? |
| ||
| I'm glad you like this, Endive. Here -- I've made some user friendly changes to it. I suppose anything can be made into a weapon. I know back in GFA I used code similar to this to pack in PKUNZIP and I used the SHELL to compress and decompress data thissaways. I have a compressor I built I'm using for S3 in BlitzMAX, it's not very small or smart though, so no RLE. This will be fine just for simple audio and images. As for the vectors, they seem to be a bit more trouble than they are worth, certainly for simple games and stuff. Happy you appreciate it. No, I'm working on a basic engine for new game programmers.  It uses: Local img_alen:TImage=img8load("15db2f92","800")
Local img_tank:TImage=img8load("0888aefc","468")
Local img_boom:TImage=img8load("a185581a","880")Where an 8x8 pixeled color image is produced from each img8load() and it's 8-character string.kfprimm: What is Base64 ? |
| ||
| https://en.wikipedia.org/wiki/Base64 It's a standard for encoding binary to text. I was just curious if your approach had any advantages. Brucey (of course!) has a module. https://github.com/BlitzMaxModules/bah.mod/blob/master/base64.mod/base64.bmx |
| ||
| Hmh, I'm reading the WIKI, Kfprimm. Is compression of English words that is wanted ? Some of the most common English words (I can think of) are to, for, the, of, at, that, a, in. Certainly those could be converted to +128 ASCII for good sentence compression. No, I am just compressing images here, and not very well, minju. A REAL compressor would use characters space through tilde with exceptions of quote and tilde as they would be reserved in a DEFDATA statement. That would still yield a good close 6-bits per character instead of 4-bits where it is now. To get the extra characters you could scan every byte and convert to 6-bit array, then sort by least on top. Determine which 'character' sorted to the top, let's say "|," record that as the first character, and also use that as your command stroke for the pertaining data. "||" = "|" "|1" = """ (quote) "|2" = "~" (tilde) I chose 4-bits cause it's easy to link 2 to make a single byte without any conversion needed for tilde or quote as they are not needed in the range. :) |
| ||
| As for the vectors, they seem to be a bit more trouble than they are worth, certainly for simple games and stuff. They're really not :) But if you're doing an engine for new programmers, I think it's very questionable to essentially encrypt the data for creating the images, unless you "ship" with a small pixel art program that generates character strings that they can cut and paste into their code. And what's the difference between that and pasting a filesystem path to a spritesheet? If you were publishing in a magazine I could definitely see why you would want to do that. Observation: many of your techniques and ideas seem to come out of a mindset from 1982. That's not necessarily bad, it's just funny. Personally I'm moving past all this retro graphics ASAP. I do it too but realtime raytracing is coming, why are we not getting ready for that and instead working on engines to mimic graphics from the Colecovision? Are you familiar with an operating system called LOSETHOS? Reinventing the wheel can be fun and educational but after a certain point it gets just ridiculous. https://www.youtube.com/watch?v=WAr-xYtBFbY |
| ||
| Hi Endive: It's curious you should mention magazines. I have written many a good code specifically for magazines in the past, as well as articles on advanced graphic programming both in BASIC and Machine-Language. I have yet to see raytrace code that is really easy to understand. It's all boggled down in one massive library calling another. Consider this, and this is just off the top of my head for a simple 3D polygon plotter: Local cube:TPolyObj[6] ' create a polygon, 6-sided, so it is a cube, it can have more sides but 6 are chosen for now.
Local img:Timage=LoadImage("smooth_stone.png"),x,y,z,h,v
For i=0 To 5
SetPolyObjSide cube,i,img ' use smooth_stone image drawn on each face of the cube
Next
DrawPolyObj cube,50,50,h,v,x,y,z ' where 50,50 is location, h&v are height & width, and x&y&z are coordinates of rotation (0-359),
Flip() ' real numbers so can have very smooth movements in decimal places
WaitKey()
EndThis is the kind of simplicity I am looking for, Endive.Failing that, I'm good with my scrolling parallax topview sprite & tile generation system. |
| ||
| It's curious you should mention magazines. I have written many a good code specifically for magazines in the past, as well as articles on advanced graphic programming both in BASIC and Machine-Language. Which ones? I have yet to see raytrace code that is really easy to understand. It's all boggled down in one massive library calling another. Consider this, and this is just off the top of my head for a simple 3D polygon plotter: Raytracing is inherently difficult to understand although it may be easier to understand than rasterizing. The reason for that is the 3d transform and projection matrices. I think probably there's going to be an OpenGL feature soon that will just do it for you but it will probably be on the order of 4 years from now. Your method above is kind of like GLUT. |
| ||
| @dw817 base64 is a binary to text encoding scheme similar to the one you are using. Difference is that instead of breaking the data into 4 bit chunks, it will break the data into 6 bit chunks. This means the data can fit within a 7 bit printable ASCII value. This is good for using binary data on systems that only accept ASCII, such as old Telnet systems and many text editors. I believe it is still the preferred way to send attachments over Email (not totally sure on that as email attachments are done automatically and can easily switch schemes without the user knowing). The main benefit of base64 over your version is that the text strings will be about 30% smaller. Also, since it is an established standard, implementations are as easy as copy/paste code. You might also want to check on the Incbin command. It will embed binary data into the compiled .exe file and can be accessed similar to a hard drive. |
| ||
| Morning, Endive: * Call A.P.P.L.E. Magazine, I think I did some work with Personal Computer too. One article was on game theory at the time. I was pointing out the importance of replay value - a video game that you would want to play again later. Another article was on getting 560 pixels across in the Apple ][, which before my article was impossible to achieve. I wrote a series of 1-liners. That is, full games and utilities written in a single line of code. I marketed many game and utility packages through Call A.P.P.L.E. at the age of 10. I developed the first truly usable for Apple ][ RPG Maker when I was 12. When I was 14, I completed Orbs Of Ankhar. A lot of these things have faded into obscurity today, however. As for 3D. I shouldn't have to wait 4-years. B4GL does some great 3D graphics in very small code today, but it's not so good at handling pixels, nor can you use INCBIN to attach files directly to the EXE. It also has no TrueType display ability - something I'm rather enjoying in my code today. TomToad: 7-bit data would be character #32 to character #160. You can't paste these into INCBIN for several reasons. The Tilde ~ is a special character for strings and PRINT. The quote " character is used to denote data between DefData. Another problem, using characters higher than 127 yields wildly different results, IE: Scroll up on OUTPUT to see that some 7-bit ASCII are translated as 16-bit numbers. For instance, character 128 translated to read back as ASCII 8364. Now, 6-bit data. I could write a 6-bit converter, likely smaller than Base64, can give it some thought. :) |
| ||
| Another article was on getting 560 pixels across in the Apple ][, which before my article was impossible to achieve. I remember Super Hi-Res. There was a Beagle Bros. 2-liner for that. As for 3D. I shouldn't have to wait 4-years. If 3D is all you want, you don't have to wait 4 years. If it's realtime raytracing you want and you want it at reasonable framerates you are going to need to wait 4 years because it's a hardware issue. Should and is are two different things. |
| ||
Looking for the Code to Carryall ? Skip all this text from a message I wrote and scroll down.  Ah, but I did the HIRES first, Endive. And yes, it wasn't available on the Apple. What I did was write to both HIRES pages simultaneously and flip them back and forth visually using a rough 60hz generated from machine-language loop. It flickered but it worked. During the loop the only thing you could really do was wait for a keystroke. Hit a key and the extended pixel effect is spoiled until the loop returned. As for the easy 2-line BASIC code, you are referring to the Apple computer I never had. Dad didn't go for the ][c, which did have 560 pixels across natively, no we went straight to Commodore Amiga. As for 3D, nope, speed is not that important, just some 3D effects. What would really be great would be something that can take a flat rectangular image and make it into a 3D trapezoid. I'd work that right in my code. Oh, took a bit of thinking, found I couldn't use greater than, less than, tilde, quote, really all kinds of characters for my 6-bit converter. I tried to paste in here and then copy out and it was all garbled every time. I didn't think about greater than and less than giving problems inside a {CODEBOX} statement, but I guess they are. But then I remembered playing a game on Nintendo that you had to use digits upper and lower-case letters to enter a lengthy password to save your game. 64 of them, the perfect number ! So, I'm using that password table as a kind of reference to the data for safe characters. it's a little rough on the edges when you convert, sometimes saves 1-2 zeroes extra, won't affect embedded pictures or sound, definitely affect text, but works otherwise pretty good. Anyways, here is Carryall Data a la "Boxcar" Edition which will let you append binary files to your text source, 118-lines not counting DefData.  |
| ||
| The last example you did is very similar to base64. When I said base64 was 7 bit, I meant the output data was 7 bit ASCII, not that the input was 7 bit. The input is actually 6 bit like your example, 64 different values. The values are then converted to one of 64 ASCII characters. A-Z, a-z, 0-9 and + and /. No tildes, nor brackets, nor any other special characters that might mess things up. |
| ||
| Oh ... Why would an output of 7-bit data be needed for anything, Tom ? I would LIKE to have just used characters 40-104 but apparently you can't use greater than or less than in a codebox despite it being code. So, I thought about that game "Willow" for the NES I played years ago and the password method they used to retrieve your saved game.  As you can see, there are 64-characters, and they are very normal looking. An O (oh) and zero (0) look very much alike in most fonts (except Willow's), so I changed that in my code to a + and the ! and ? to parentheses. All in all, I think it's pretty neat what it does, and likely can be used for other programming languages that don't internally bind media files like BlitzMAX. |
| ||
| You use 7 bit because that is what ASCII uses. ASCII has 95 printable characters, and that can't be represented in 6 bits. It is exactly what your example is doing. Note that just because the output is 7 bit ASCII doesn't mean it is being stored 7 bits per character. It is being stored with as many bits as the character scheme being used. On the code box, it is actually using 8 bit UTF-8 and when copied/pasted into the MaxIDE editor, it will paste as 16 bit UCS. This will not create any problems with encoding/decoding as characters 32-126 are exactly the same in most common character encoding schemes. So you can encode binary data into 8 bit ANSI, post it to a web forum using UTF-8, Copy/Paste into a Unicode text editor, save as ISO-8859-1, and it will still decode properly. |
| ||
Well now years ago I did write a 7-bit to 8-bit converter and back again. Back then I was blessed to have a BIT TEST function part of GFA. Here is the main part of the code:PRINT #99,CHR$(ASC(FNC$(D$,K + 1)) - 35 - 128 * BTST(Bit8,K));I'm - not sure where you can post 7-bit characters in a forum Online today - nor will BlitzMAX accept it in DefData ? |
| ||
| I'm - not sure where you can post 7-bit characters in a forum Online today - nor will BlitzMAX accept it in DefData ? You will not be posting 7 bit characters. You will be using however many bits the editor uses for its encoding. Base64 is limiting its character choices to what exists in 7bit ASCII as those characters are the same for all character encodings. It will expand the bits to fit into whatever encoding is being used. So for a forum, an extra 0 bit will be added to make it UTF-8. In MaxIDE, 9 extra 0 bits will be added to make it 16 bit UCS. Since both encodings are exactly the same for character 32-126, nothing else will need to be done. For example, if you decided to use as a character, it will not work. In the MaxIDE, it will be encoded with character 163, but when you copy/paste the character into a forum, it will be translated to UTF-8 which uses character 49827. That is the reason for using 7 bit ASCII. All the common encodings are built on top of ASCII. The letter A in ASCII is character 65. When it is copied into MaxIDE, it will now be 16 bits, but it will still be character 65. When you c/p to a forum, it will be 8 bits, but will still be character 65. This way, data encoding/decoding will be completely independent to the character encoding and size. BTW, the reason ASCII is 7 bit is because the 8th bit was used for parity. Necessary in the early days of telecommunications. |
| ||
| That's a history lesson I enjoyed reading ! Thanks, Tom. I always thought IBM was stingy with that 8th bit. As you say, apparently it was used for accurate communications. Now had =I= a chance of custom system characters, you might have something like this I was working on when BBSs first showed 8-bit characters.  I knew ANSI was there, but I wanted something better, and likely would've done this given the time. I did manage to write the DOOMSDAY BBS in QBasic 1.0 years ago. Had many callers, lots of friends, and was going to implement all of this when the Internet showed up and changed all that. |
   |