Hey everybody I’ve just published a new article in the networked physics series: Snapshot Compression.
Some folks out there think they can do better than the compression described in the article. Now I don’t doubt for one second that better compression approaches exist, but exactly how much better is your encoding than the one described in the article? Don’t just tell me your encoding is better. PROVE IT.
It’s time for a good old fashioned walk-off.
Here are the terms of the competition. Old school rules.
If you can beat my best encoding over this dataset by 10% I’ll link to your blog post describing your compression technique from within the article. Your blog post must fully describe your encoding and include source code in the public domain that proves the result. Compression must be lossless.
The data set to be compressed is position and orientation data (smallest 3) in integer format post-quantize with delta encoding baseline. Your job, if you choose to accept it, is to encode each frame snapshot into packets using the least number of bits.
When reporting your results please include the following: number of packets encoded, average packet size in bytes, average kbps across the entire data set. To calculate this add up the number of bytes for all of your packets (if you have encoded bits not bytes, please round up to the next byte per-packet as you can’t send fractional bytes over UDP). Divide the total bytes by the number of packets to get the average packet size in bytes. To calculate bytes per-second multiply average packet size by 60 (60 snapshots per-second). To convert bytes/sec to kbps multiply bytes per-second by 8 and divide by 1000 (*not* 1024!). My best result so far is below 256kbps.
IMPORTANT: You must encode and decode each frame individually into packets (eg. 901 cubes at a time). You may not encode the entire data set at once. You cannot assume that all packets will be received by the sender. You cannot assume that all packets will be received in-order. You may not rely on information inferred from previously encoded/decoded packets when decoding a packet unless you embed that information in the packet or can infer that information even in the presence of packet loss and out of order packets. You may not write any decompression scheme that relies on putting packets in a queue and processing them in-order as this delays realtime data delivery of snapshots and defeats the purpose. Encryption and decryption must be plausibly realtime. Static dictionaries trained on the data set are acceptable. You may not claim a win based on a dictionary trained on the same data set that is being compressed. Final results will be judged against an unreleased data set. If necessary, separate datasets for training and encoding will be provided for the final judging.
Data format is fixed records. 901 cubes per-frame. Frames 0..5 are duplicates of initial state. Start your encoding at frame 6 and encode frame n relative to frame n-6.
Each cube struct is as follows:
struct DeltaData { int orientation_largest; int orientation_a; int orientation_b; int orientation_c; int position_x; int position_y; int position_z; int interacting; };
Best of luck!