1 Cool Things in Small SpacesThe Art and Science of Intros and Bytetros
2 Who am I? I am “Polaris”, the founder of the Northern Dragons Demo Group. I am also an active coder for the Northern Dragons and have written code for several 4kb productions. The most recent* being “4kb Evolutio” released at Assembly 2004. Yes, you can contact me – just me! Note: Northern Dragons is a North American based demo group, so we are proud to attend Pilgrimage. Recent * - not including Pilgrimage production info in this seminar, I’m not sure they have been released yet! Aside: We are also the group that wrote the 2004 Pilgrimage Invitation.
3 What are we supposed to talk about?Showcase some example intro’s and bytetros. Highlight what’s special about intros compared to demos and their role in the demo scene. Tell some Tales from the field: Design advice Code techniques - specific to Windows 32 Your requests, and questions are important! There is a 15 minute period for questions at the end, but do ask questions during the presentation too! Presentation will be made available afterwards, so don’t worry too much about taking notes. Just raise your hand to snag my attention to ask questions.
4 What am I not talking about?Intro’s larger than 16kb, in terms of techniques. (64kb’s are quite different) How to use any particular intro creation tools [such as .werkkzeug1] Things outside of the win32 platform. (Outside my realm of expertise) API “wars”. (For example: OpenGL vrs Direct 3D) Note: I am discussing bytetros, but mostly intros.
5 What do you want me to talk about?The seminar is “in a Power Point can”, but I’m flexible! Room Survey: (show hands) How many people are here are active in the demo scene? How many people are actively writing productions? What roles? (Coder?) (Musician?) (Artist?) What platform? (Win32? Linux? Other?) Does anyone have specific things in mind they want answered during the presentation?
6 What are Intros? The first intros were “cracktros”. These programs are graphical “brag tags”, that appeared at the start of a cracked games. These gave greetings to other crackers, and displayed the credits of the crack authors using pseudonyms. Eventually, people started making little graphical programs for fun, instead of just in cracks. This gave birth the demo scene in general. Demos are real-time computer productions of a larger size (typically MB now), where as intro’s are limited size.
7 Why make them? Intros explore the technical edge of what’s possible in a small space, as well as exploring real-time art. Each one is a personal brag tag saying “look what we can do”! Every intro is really an experiment showing “apparently this is possible in xxxx bytes” [whatever the size is]. Still, some intros have more “spirit” and design than others.
8 King of the Kings – Demo PartiesDemos and intros are often shown at parties, Compos (competitions)… where they compete for the audience favor [and prizes]. Pilgrimage is one of many events where compos are held. Competitions help “spur” intro creators on, to out do each other. There will probably be some intros released at Pilgrimage ;-)
9 What does a small intro look like?Let’s watch some samples. The following samples just a few I have chosen, so be sure to check out demo scene sites for more: These productions have been released at demo parties, so please rank your favorites as you watch. We’ll compare your ranking to the party ranking after.
10 4KB Intro Viewing Order 1. San Angeles Observation 2. Smells Like Fish 2. Smells Like Fish 3. Evolutio 4. 99 Beers 5. The Etherium
11 256 Bytetro Viewing Order Dropz Tube Tube
12 Lets Watch!
13 San Angeles Observation by Armada & TraumaRelease date : August 2004 Party : Assembly 2004 Ranked : 1st
14 Smells Like Fish by Axes DeniedRelease date : August 2004 Party : Assembly 2004 Ranked : 8th
15 Evolutio by the Northern DragonsRelease date : August 2004 Party : Assembly 2004 Ranked : 4th
16 99 beers by Siffo Release date : August 2003 Party : Assembly 2003Ranked : 9th
17 The Etherium by The Northern DragonsRelease date : August 2003 Party : Assembly 2003 Ranked : 14th
18 Dropz by Baze / 3SC Release date : 1998 Party : Demo Bit 1998Ranked : 1st
19 Tubes by 3SC Release date : August 2001 Party : Syndeecate 2001Ranked : 1st Bonus info: Pilgrimage invite contains a tribute to tubes…. A “tunnel effect” is mimicked after tubes
20 Surprised by the Rankings?Intros have evolved from just pushing technical limits, to having a sense of “design”, and appealing to the audience. The importance of “Design” is discussed well in “Tips for Demomakers”: Also check out the 4KB intro compo studio from assembly for some discussion about design. ftp://ftp.scene.org/pub/parties/2004/assembly04/assemblytv/4k_intro_compostudio.avi&id=250850
21 So you want to create an intro? (Technical Notes)Intros typically feature real-time music and graphical effects. Because of the small space, data isn’t typically stored. Instead algorithms that generate the data have to be used. The most important thing about writing an intro isn’t the platform, or coder tricks. It’s about the algorithms in the first place.
22 A simple example textureThe texture on the right is 256x256, with in colour [grey scale]. Even as a .jpg [with compression] it takes a total of 6KB to store. This is larger than many intro’s! Aside: This is the same texture used in one of the to be released ND Pilgrimage 2004 productions. It’s called an environment map
23 Using algorithms (Example)To generate our texture, we instead need to use an algorithm. This un-optimized algorithm generates this texture: for(y=0;y < 256;y++) { for(x=0;x < 256; x++) // the algo height_value=(1-(sqrt(((x-128)*(x-128))+((y-128)*(y-128)))/128))*255; } Note – even without code size optimizations, it takes no more than 200 bytes in a basic C compiler – a savings of over 15 Times! This is only a small example – you need to think algorithmically about, textures, geometry, music, and code! Notice this is essentially a distance formula.
24 Programming Tools Once you start thinking algorithmically, it becomes time to code. Your intro or bytetro will eventually become an executable program, running on a pc. This usually involves an assembler or compiler of some type, to generate the binary code executed. For many this is assembly…. but….
25 Language Examples Java. Yes it’s possible! [Jar file with J3D]Material Demo - 2,078 bytes Turbo Pascal Gargaj/Ümlaüt Design – Hugi 22 Delphi Migeel – Hugi 28 I should mention that I don’t have a resource for the java link, because I haven’t published my j3d test yet ;-). However a simple java app with j3d in a jar can be less than 4kb. (To do: Show Material Demo..2,078 bytes)
26 Language Examples C/C++ Assembly Language: OthersKwak by Calodox, 7th place Assembly 2002 Evolutio by Northern Dragons, 4th Place Assembly 2002 Assembly Language: The Etherium (NASM) Trees (MASM) Others Of course…. Use your imagination! I mention some specifically, as I know how they were built.
27 Pushing the technical limitsThe first stage is to push the limit of the algorithm. The second stage is to push the limit of the Language selection. (A compiler or assembler) While many avoid Assembly language, it gives you the most options and flexibility to “push limits”. This may involve using features in ways they weren’t first designed. It isn’t the only option however! The third stage is to push the limit of the operating system itself. This deals with OS limitations, or what a “executable” file is on that Operating system.
28 Second Stage: Specific to C/C++The largest changes possible relate to setting your compiler options, and how you code in your C/C++ compiler. Take the following example: #include
29 Tweaking VC++ The same program, altered just a little bit…. Becomes 1KB in size! #pragma comment(linker,"/ENTRY:main") #pragma comment(linker,"/MERGE:.rdata=.data") #pragma comment(linker,"/MERGE:.text=.data") #pragma comment(linker,"/IGNORE:4078") #pragma comment(linker,"/OPT:NOWIN98") #include
30 What happened? It’s the same program, but we’ve changed the compiler options, cutting out the “fluff” that the compiler includes by default. These include: Microsoft Visual C++ Runtime Library 4kb section alignment [not required] Merged sections in the exe: .text (default code section), .rdata (default read-only data section) .data (initial data section)
31 Pushing the Operating SystemThe biggest challenge is the exe alignment, and headers. Windows 32 programs have a lot of extra “fluff” in them, that you don’t need. The MS-DOS had a light program. This way it supported two types of programs: .COM files - limited to 64kb, but no headers! .EXE files - for large exe For this reason most bytetros are actually MS-DOS .COM file programs. Fluff = I’m talking about exe headers alignment and such.
32 Phase #1 Using an executable packerBecause of the extra stuff in the exe header, coders started using executable packers. For details on the win32 PE format: Executable packers store the original exe using compression, and attach a decompression routine that writes the it exe to memory and then executes. A good standard exe packer is UPX:
33 Phase #2 – Using a file dropperThe problem is, the resulting compressed exe – is still a windows program. It has a certain amount of “fluff” that you still can’t avoid. To get around this, a technique known as file dropping was developed. DOS MZ header DOS stub PE header Section table Section 1 Section 2 Section ... Section n Notice – you still get all the stuff on the right with every exe
34 File Dropping File dropping takes a Windows executable program and attaches it to a MS-DOS .com file. This .com file writes out the attached windows executable to disk, runs it, and then deletes it once finished. Because this accesses the local disk for the temporary file, this technique “drops” a file. Be careful! Various parties have various rules. Some outlaw writing anything to disk! Include personal story here if there is time: Trees 4kb at Assembly 2002
35 File Dropping The write to disk routine takes a few bytes on top of the regular windows executable. The improvement comes from using a .com file packer. This is like a regular exe packer, but for the MS-DOS platform. A very good and popular one is apack. This technique is characterized by a MS-DOS window that shows up before the production starts. With this technique it’s possible for a 16kb to shrink down to 4kb. For Example – the Etherium bytes -> 4008 bytes Show off a file dropper intro here.
36 File Dropper Tool LinksA few Tools to create file droppers: PE2EXE [aka PE2COM] – From Frankie / Smash Designs GEM Dropper 1.0 Warning: Many tools drop a file into C:\. This may mean your production won’t work on Windows XP [post SP1], without administrator privileges. Read your tool documentation. (Or make your own!)
37 Phase #3 – Cab Dropping No matter how you slice it… a file dropper must have the executable code to decompress the original windows exe to disk. In Hugi 28, Gem Introduced a technique called “CAB dropping”. Since then this has become the “defacto” standard for 4kb’s, most of the entries in Assembly kb Intro Compo are cab droppers of some type.
38 Cab Dropping A cab dropper uses the operating systems extract program to decompress the exe. Cab files are similar to zip files. Win95/98/ME: c:\windows\command\extract.exe WinNT/2K/XP: c:\windows\system32\extrac32.exe To do the decompression you write a batch file It calls extract.exe, and or extrac32 runs the win32 program, then deletes it. Cab files use LZX compression
39 Cab Dropping OS DependenciesBecause of platform variations, often a cab dropper will call both extract and extract32. One works. The other results in something like: 'extract' is not recognized as an internal or external command,operable program or batch file. For this reason some feel that cab dropping is “messy”.
40 Cab Dropping – A single fileIt’s also possible to trick the cab file format into having the batch file commands in it. This results in a single .bat or .cmd, that contains your production. It’s possible to avoid problems with root folder writing by writing to the operating system %temp% folder. The joined cab / batch file has more info that shows up in the header. Specifically: 'MSCF' is not recognized as an internal or external command, operable program or batch file.
41 Space savings / Tool LinksIn summary (The Etherium) 16474 bytes – pure windows exe 4008 bytes – as a file dropper 3856 byte – as a cab .bat file Cab Dropper tool links: Gem Cab Dropper 20to4 Cab Dropper
42 The Future As operating systems add features, new options become available for creative programming. In the future we can look forward to a variety of more cool things in small spaces! (on every type of operating system)
43 The Future The key is to be creative in the way you use your computer!Algorithm Code Operating System And DESIGN!
44 Question Period / DiscussionsAny questions? Northern Dragons 4KB Intro Kit distribution