Notes
Compression in Game Engines
---------------------------
How much do you know about compression?
- lossy vs lossless?
* huffman encoding
* perceptual coding
* lempel-ziv
Why?
- games care about performance, space and time
- compression trades off time for space
- but sometimes you effectively get both
lossless compression
- compressed version can be used to reproduce original bit-for-bit
- zip files
lossy compression
- some information is thrown away during compression
Information theory
- information: number of bits needed to represent something
- but, LEAST number of bits
- so part of information theory is always compression
Maximum information is from random processes
- random bits can only be reproduced by copying them, no algorithm
can do so without being equal or bigger than the bits
Run length encoding
- represent repeated runs explicitly
- i.e., say 20 0's instead of 00000000000000000000
- only works if you expect to have many runs
- if you have few, you can actually make the output bigger than the input
All compression algorithms try to transform data into a representation equal to the amount of information in it
- i.e., the theoretical amount of information = size of data
An approximation of the amout of information in something...how big is it when you compress it!
With lossy compression, we throw away information
- output doesn't have all the information of the input
lossy compression seems like a bad idea
- don't do it with code!
idea: throw away information that won't be missed
- patterns that are imperceptible
- so, you have to have a model of what is perceptible!
For audio, lossy compression is common because our ears can't detect many
patterns in sound
MP3 was developed by someone at the Fraunhoffer institute, listening to a song by Suzanne Vega
- did it until he couldn't hear any artifacts after compression, i.e.,
the frequencies that were thrown away were imperceptible
More recent codecs, i.e., AAC, use more sophisticated models of human perception
Same thing for images and video
- what patterns do people not notice?
More sophisticated perceptual models allow for higher compression ratios but at the cost of more computation, especially during compression
- typically compression is much more expensive than decompression
- but the tradeoff varies
Latest trend in compression is AI-assisted compression
- e.g., feed data into a neural net, let it reproduce it
- neural net identifies patterns that can later be reproduced
most compression is based on some form of dictionary
- identify the "words" that make up the document
- then compress by storing the dictionary and then which words are used, with
the words identified as indices in the dictionary
Can lead to great compression if you use a pattern recognizer, but artifacts get very weird
- instead of weird blocks, you get weird faces
- you see this in image enhancement, which is really just reverse compression
As a game developer, why do you care about all this?
- need to choose the right compression for game assets
- PNG vs JPEG (lossless vs lossy)
- wav vs ogg/mpg/aac
You're making a space vs time tradeoff
- bigger files for faster processing
- but the cost of bigger files can be more of a cost than the processing
PS5, XBox Series X
- part of performance gain is hardware decompression
- so you actually get better performance with compressed assets
- particularly important for textures
Why do games get so big?
- a large part is using uncompressed assets, duplicated assets for performance
- duplication is for slow seek times of spinning hard drives, optical media
But new consoles have SSDs and hardware decompression
- so can get smaller games that perform better
compression has big future as perception really is compression