There's a FMV player for the 8-bit Sega Master System ( https://github.com/bravesoftdz/tiler ,) which compresses to a fixed number of dynamically updated tiles. The codec in TFA and Tiler seem very similar, with Tiler also taking advantage of the SMS's ability to flip tiles horizontally or vertically.
A higher spec version of Tiler is available as TileMotion, which has a JavaScript demo decoding 1080p video in software here: https://gligli.github.io/tilemotion/demo/
Choosing 8×8 blocks is very likely highly suboptimal when it comes to the quality of the codec.
The point of the 4×2 tiles is exactly to avoid the blockiness artifacts of vector quantization (visible in the article).
Larger tiles increase the average error and entropy because of the way VQ works - it tries to find blocks that look like each other and then encode all similar blocks by referencing a single tile (usually choosing the "average" or "mode" the blocks).
As tiles grow larger, the chance to find highly similar blocks obviously decreases (and deviation from the representative tile increases).
Reducing one of the dimensions is probably better positioned on the quality-speed frontier.
Once you see the tile map of this in action, you realize how genius and simple this idea is. You can only update 32 tiles a frame but it can preload tiles in advance of them being displayed provided there is low enough motion and spare space to allocate.
Would love to see something like this ported to similar systems such as Amiga, Mega Drive and Neo Geo. All 68K, all tile based.
Also seeing that gameplay at the end, absolutely wonderful. Would not have figured it was even remotely possible on a system like that, but there it is.
I'm sure the effort is good, but the article is not. It reads like breathless corporate crap. It also doesn't have any video of this FMV, though what I see in the HTML source is an attempt to use a third-party site to embed youtube videos, which does nothing on Firefox.
Doesn't work:
<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FPhOa3KCE_nM%3Ffeature%3Doembed&display_name=YouTube&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DPhOa3KCE_nM&image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FPhOa3KCE_nM%2Fhqdefault.jpg&type=text%2Fhtml&schema=youtube" allowfullscreen="" frameborder="0" height="480" width="640" title="Command & Conquer Intro on Atari ST" class="dr g ii al cs" scrolling="no"></iframe>
I wanted to see how big the example clips encoded down to. Or is this just a technical test to see what it's capable of playing back, not an attempt to optimize the video size?
> Each 8×8 block is turned into a short DCT (Discrete Cosine Transform) feature vector; distance is measured there instead of raw pixels.
Was this your idea or the agent's? Because it feels peculiar to essentially describe JPEG, modulo the entropy coding, without mentioning its name. It was released in 1992 -- before C+C. Which makes me wonder why Westwood didn't just use JPEG too...
It's not JPEG like at all, unlike JPEG it is using the DCT only for the loss function, so it can have frequency weights, having the same approximate effect as JPEG's quantization matrices (at least, according to the slopped-out description). Whether that actually improves things for this type of content is unclear.
There's a FMV player for the 8-bit Sega Master System ( https://github.com/bravesoftdz/tiler ,) which compresses to a fixed number of dynamically updated tiles. The codec in TFA and Tiler seem very similar, with Tiler also taking advantage of the SMS's ability to flip tiles horizontally or vertically.
A higher spec version of Tiler is available as TileMotion, which has a JavaScript demo decoding 1080p video in software here: https://gligli.github.io/tilemotion/demo/
Choosing 8×8 blocks is very likely highly suboptimal when it comes to the quality of the codec.
The point of the 4×2 tiles is exactly to avoid the blockiness artifacts of vector quantization (visible in the article).
Larger tiles increase the average error and entropy because of the way VQ works - it tries to find blocks that look like each other and then encode all similar blocks by referencing a single tile (usually choosing the "average" or "mode" the blocks).
As tiles grow larger, the chance to find highly similar blocks obviously decreases (and deviation from the representative tile increases).
Reducing one of the dimensions is probably better positioned on the quality-speed frontier.
Once you see the tile map of this in action, you realize how genius and simple this idea is. You can only update 32 tiles a frame but it can preload tiles in advance of them being displayed provided there is low enough motion and spare space to allocate.
Would love to see something like this ported to similar systems such as Amiga, Mega Drive and Neo Geo. All 68K, all tile based.
Also seeing that gameplay at the end, absolutely wonderful. Would not have figured it was even remotely possible on a system like that, but there it is.
I'm sure the effort is good, but the article is not. It reads like breathless corporate crap. It also doesn't have any video of this FMV, though what I see in the HTML source is an attempt to use a third-party site to embed youtube videos, which does nothing on Firefox.
Doesn't work:
Let's go to the Youtube video it's meant to show: https://www.youtube.com/watch?v=PhOa3KCE_nM
Now let's put Youtube's direct embed code in there:
Now it works just fine.
>I'm sure the effort is good, but the article is not. It reads like breathless corporate crap.
That is because it is AI-written.
In case anyone wondering what the "famous" MOVEP instruction does, this:
Writes to the 8 bytes pointed at by A0 in this format:
Where xx are bytes left as they are.
Something like this:
Gives you this at the address pointed to by A0:
I wanted to see how big the example clips encoded down to. Or is this just a technical test to see what it's capable of playing back, not an attempt to optimize the video size?
> Each 8×8 block is turned into a short DCT (Discrete Cosine Transform) feature vector; distance is measured there instead of raw pixels.
Was this your idea or the agent's? Because it feels peculiar to essentially describe JPEG, modulo the entropy coding, without mentioning its name. It was released in 1992 -- before C+C. Which makes me wonder why Westwood didn't just use JPEG too...
Fun fact, DCT's inventor: https://en.wikipedia.org/wiki/Nasir_Ahmed_(engineer)
Almost every lossy image and video codec uses DCT as well, not just JPEG.
Which ones came before JPEG?
H.261 is from 1988.
JPEG is from 1992.
It's not JPEG like at all, unlike JPEG it is using the DCT only for the loss function, so it can have frequency weights, having the same approximate effect as JPEG's quantization matrices (at least, according to the slopped-out description). Whether that actually improves things for this type of content is unclear.