Game Engines 2021W Lecture 20

Notes

Compression in Game Engines
---------------------------

How much do you know about compression?
 - lossy vs lossless?

* huffman encoding
* perceptual coding
* lempel-ziv

Why?
 - games care about performance, space and time
 - compression trades off time for space
   - but sometimes you effectively get both

lossless compression
 - compressed version can be used to reproduce original bit-for-bit
 - zip files

lossy compression
 - some information is thrown away during compression

Information theory
 - information: number of bits needed to represent something
    - but, LEAST number of bits
 - so part of information theory is always compression

Maximum information is from random processes
 - random bits can only be reproduced by copying them, no algorithm
   can do so without being equal or bigger than the bits

Run length encoding
 - represent repeated runs explicitly
 - i.e., say 20 0's instead of 00000000000000000000
   - only works if you expect to have many runs
   - if you have few, you can actually make the output bigger than the input

All compression algorithms try to transform data into a representation equal to the amount of information in it
 - i.e., the theoretical amount of information = size of data

An approximation of the amout of information in something...how big is it when you compress it!

With lossy compression, we throw away information
 - output doesn't have all the information of the input

lossy compression seems like a bad idea
 - don't do it with code!

idea: throw away information that won't be missed
 - patterns that are imperceptible
 - so, you have to have a model of what is perceptible!

For audio, lossy compression is common because our ears can't detect many
patterns in sound

MP3 was developed by someone at the Fraunhoffer institute, listening to a song by Suzanne Vega
 - did it until he couldn't hear any artifacts after compression, i.e.,
   the frequencies that were thrown away were imperceptible

More recent codecs, i.e., AAC, use more sophisticated models of human perception

Same thing for images and video
 - what patterns do people not notice?

More sophisticated perceptual models allow for higher compression ratios but at the cost of more computation, especially during compression
 - typically compression is much more expensive than decompression
 - but the tradeoff varies


Latest trend in compression is AI-assisted compression
 - e.g., feed data into a neural net, let it reproduce it
 - neural net identifies patterns that can later be reproduced

most compression is based on some form of dictionary
 - identify the "words" that make up the document
 - then compress by storing the dictionary and then which words are used, with
   the words identified as indices in the dictionary

Can lead to great compression if you use a pattern recognizer, but artifacts get very weird
 - instead of weird blocks, you get weird faces
 - you see this in image enhancement, which is really just reverse compression


As a game developer, why do you care about all this?
 - need to choose the right compression for game assets
   - PNG vs JPEG (lossless vs lossy)
   - wav vs ogg/mpg/aac

You're making a space vs time tradeoff
 - bigger files for faster processing
 - but the cost of bigger files can be more of a cost than the processing

PS5, XBox Series X
 - part of performance gain is hardware decompression
   - so you actually get better performance with compressed assets
   - particularly important for textures

Why do games get so big?
 - a large part is using uncompressed assets, duplicated assets for performance
   - duplication is for slow seek times of spinning hard drives, optical media

But new consoles have SSDs and hardware decompression
 - so can get smaller games that perform better

compression has big future as perception really is compression