# HG changeset patch # User Laurens Holst # Date 1609697268 -3600 # Sun Jan 03 19:07:48 2021 +0100 # Node ID 6a8cf1bcdadd00908846c2aeeae499520d89a3a1 # Parent caea1eec985cacbc9190630a9a5d49a5e1d25554 yjk: New “The YJK screen modes” article diff --git a/articles/index.php b/articles/index.php --- a/articles/index.php +++ b/articles/index.php @@ -40,6 +40,7 @@
  • A guide to scrolling game engines on MSX MAP
  • V9938 VRAM timings · by the openMSX team
  • V9938 VRAM timings, part II · by the openMSX team
  • +
  • The YJK screen modes MAP
  • Other topics diff --git a/articles/yjk/index.php b/articles/yjk/index.php new file mode 100644 --- /dev/null +++ b/articles/yjk/index.php @@ -0,0 +1,601 @@ + + + + + The YJK screen modes + + + + + + + + + +

    The YJK screen modes

    + +
    +
    Interactive YJK colour space visualisation
    +

    + + +

    +

    + + + + + + + + + + +

    +

    + + +

    +
    + +

    An exploration of the YJK screen modes of the MSX2+.

    + + + + +

    Introduction

    + +

    + The Yamaha V9958 VDP used in the + MSX2+ and MSX turbo R adds two high colour display modes which use a different + colour encoding than traditional RGB, called YJK and YJK+YAE. These modes + expand the colour count to 19268 and 12499 + 16 respectively, at the cost of + colour attribute clash between groups of 4×1 pixels. +

    + +

    + The YJK mode is MSX-BASIC’s SCREEN 12. It can show up to 19268 colours, and is + particularly suited for photographic images because these are generally fairly + unaffected by colour attribute clash. See + art gallery. +

    + +

    + The YJK+YAE mode is MSX-BASIC’s SCREEN 10 and 11. It can show up to 12499 YJK + colours, and 16 palette colours (out of 512). Since the palette colours are set + per-pixel they can be used to hide colour bleeding, while the YJK colours can + be used to make the picture more colourful. Due to this YJK+YAE is more versatile + in situations when you also want to display a user interface, like in a game. See + art gallery. +

    + +

    + An additional benefit of these new screen modes have compared to SCREEN 8 is + that their sprites make use of the palette, rather than a limited set of + predefined colours. +

    + + + +

    Selecting the YJK modes

    + +

    + The MSX2+ BIOS CHGMOD + routine can be called with the MSX-BASIC SCREEN number in register A. +

    + +

    + On the VDP the YJK mode is selected by following the procedure to select + SCREEN 8 (G7), and then setting the YJK bit of register 25. To select the + YJK+YAE mode, additionally set the YAE bit. +

    + +
    + + + + +
    VDP register 25
    76543210
    r#250CMDVDSYAEYJKWTEMSKSP2
    +
    + +
    + + + + + + + +
    Combination of YJK and YAE data
    YJKYAEVRAM data
    00Via the conventional colour palette
    1
    10A=0: Via the RGB → YJK conversion table
    1A=1: Via the colour palette
    +
    + +

    + Since a number of MSX2 computers exist that do have a V9958 but no BIOS + support, as well modded MSX2 computers with their VDP replaced, consider using + CHGMOD to + switch to SCREEN 8 and then setting the YJK and YAE bits in r#25 + manually. The other bits can be left at 0. +

    + +

    + In MSX-BASIC, SCREEN 12 and 11 have no special support for drawing in YJK + other than using 8-bit colour operations. However SCREEN 10 is special, + it gives a palette-colour view of YJK+YAE and allows you to draw on it like + in SCREEN 5. Switching between SCREEN 10 and 11 will not clear the screen. +

    + + +

    The encoding of YJK

    + +

    + YJK breaks down into three components. The Y component specifies brightness (luma) + and a little bit of blue. The J and K components contain most of the colour + information (chroma), expressed as a difference between Y and red and green, + respectively. The concept behind this encoding is that the human eye is more + sensitive to brightness than colour, and the colour resolution can therefore be + reduced with relatively limited impact on the perceived result. +

    + +

    + Use the visualisation widget above to get a feel of the YJK colour space. + You can interact with this widget to view the various colour ramps available. + The horizontal axis is the J value, the vertical axis is K. The column on the + right controls the Y amount. All colours in the Y column can be used in the + same four-pixel group. As you increase Y the colour becomes brighter, but + notice that it becomes slightly more blue as well. This can be seen well in + the grayish hue when both J and K are zero. +

    + +

    + In YJK mode the VDP stores the J and K components in 6 bits as a two’s complement + signed value (-32 to 31). These are shared between 4 adjacent pixels. The Y component + is a 5-bit unsigned value (0 to 31). This one can be specified per-pixel. +

    + +
    + + + + + + + +
    YJK (SCREEN 12)
    C7C6C5C4C3C2C1C0
    1 dotY1Klow
    1 dotY2Khigh
    1 dotY3Jlow
    1 dotY4Jhigh
    +
    + +

    + In YJK+YAE mode the VDP still stores the J and K components in 6 bits as a + two’s complement signed value (-32 to 31). However the Y component can now + only specify even values (0, 2, 4 .. 30), as its least significant bit is + used for the attribute bit A. If the A bit is set, the top four bits of + the Y component specify a palette colour to use instead. +

    + +
    + + + + + + + +
    YJK+YAE (SCREEN 10-11)
    C7C6C5C4C3C2C1C0
    1 dotY1AKlow
    1 dotY2AKhigh
    1 dotY3AJlow
    1 dotY4AJhigh
    +
    + +

    + The palette is still specified as 9 bits, with 3 bits per component. However + the V9958’s DAC is 15-bit, so there must be some kind of mapping in place. + Indeed, the 3-bit RGB palette component values map to 5 bits as follows: +

    + +
    + + + + + + + + + + + +
    3-bit to 5-bit conversion
    V9938V9958
    00
    14
    29
    313
    418
    522
    627
    731
    +
    + +
    +
    3-bit to 5-bit conversion
    +

    + + + cout + = + + 4.5 + + cin + + + +

    +

    + + + cout + = + cin + << + 2 + OR + cin + >> + 1 + + +

    +
    + +

    This applies to all screen modes, by the way.

    + + +

    Conversion between YJK and RGB

    + +

    + The Y, J and K values specify 17 bits of information in total. The V9958 + converts these to 15-bit RGB with 5 bits per colour component. The + V9958 manual describes the + following conversion formulas for YJK. The formulas for YJK+YAE are the same, + but the least significant bit of Y is always zero so all its values are even. +

    + +
    +
    YJK to RGB
    +

    + + r = y + j + +

    +

    + + g = y + k + +

    +

    + + b = + + (5y - 2j - k) + / 4 + + + +

    +
    + +
    +
    RGB to YJK
    +

    + + y = + + (4b + 2r + g) + / 8 + + + +

    +

    + + j = r - y + +

    +

    + + k = g - y + +

    +
    + +

    + Indeed those are fine formulas and perfect mathematical inverses of each other. + However they do not take into account that the VDP both rounds and clips the + result of the YJK to RGB conversion. +

    + + +

    Rounding

    + +

    + Let’s consider the rounding first. The y, j, k and r, g, b values are all + integers, so when the blue value is calculated, it is rounded down to the + nearest integer value (“floor”). To compensate for this in the RGB to YJK + formula, you need to round up (“ceil”) the value resulting from the y. +

    + +
    +
    YJK to RGB (rounded)
    +

    + + r = y + j + +

    +

    + + g = y + k + +

    +

    + + b = + + (5y - 2j - k) + / 4 + + + +

    +
    + +
    +
    RGB to YJK (rounded)
    +

    + + y = + + (4b + 2r + g) + / 8 + + + +

    +

    + + j = r - y + +

    +

    + + k = g - y + +

    +
    + +
    +
    RGB to YJK+YAE (rounded)
    +

    + + y = + 2 + (4b + 2r + g) + / 16 + + + +

    +
    + +

    + Using these formula you can express exactly half of what 15-bit RGB allows; + 16384 out of 32768 colours (50%), and 8192 for YJK+YAE (25%). Each + representable RGB value maps to one unique YJK value and vice versa. If you + want to keep things simple, stop here and stick to these 16384 colours. +

    + + +

    Clipping

    + +

    + As we know though YJK mode can generate 19268 colours. Where did the remaining + 2884 colours go? YJK’s 17 bits allow for 131072 different values, but most of + these fall outside the range of 15-bit RGB. Values that are out of range are + clipped to [0, 31]. +

    + +
    +
    YJK to RGB (clipped)
    +

    + + r = + clamp(y + j, + 0, 31) + +

    +

    + + g = + clamp(y + k, + 0, 31) + +

    +

    + + b = + clamp( + + (5y - 2j - k) + / 4 + , + 0, 31) + + +

    +
    + +

    + As a side effect of this clipping, 2884 colours which could not be represented + by the previous YJK formulas become available. For example, take the colour + (0, 24, 31). Without clipping this could not be expressed, since applying the + previous RGB to YJK to RGB formula will yield (0, 24, 32). Due to the clipping + of the blue component though, this colour is available to us after all. +

    + +

    + If we calculate the number of colours affected by clipping, those which have + one component of either 0 or 31, there are 323 - 303 = 5768 + of them. Half of them are already covered by unclipped colours, 5768 / 2 = 2884, + meaning that all of the colours where any RGB component is 0 or 31 can be + represented. Some of these have only one YJK representation, others up to 3463. +

    + +

    + You may notice no RGB to YJK formula is specified. This is because although + colours whose components all fall within the [1, 30] range have a unique YJK + representation which can be determined with the earlier formula, colours with + one or more components of either 0 or 31 have multiple solutions. +

    + +

    + In order to deal with this, consider that an RGB value is a point in a + 3-dimensional (colour) space. For each potentially clipped component (0 or 31) + a ray in the clip direction extends from the point, where two rays form a + bounded plane, and three rays a volume. This colour space can be transformed + to YJK with a matrix, and all YJK values overlapping this point, ray, plane + or volume represent the same RGB colour. +

    + +

    + More on this in part 2, which will be a deep dive into the topic of image conversion. +

    + + +

    Drawing techniques

    + +

    + The YJK modes can be very nice for pixel art because of the wider range of colours + that you an express with 15-bit RGB. Ever wanted that particular pastel colour + that the V9938’s 9-bit RGB palette could not express? Or a smoother gradient? + The V9958 can do in YJK. Additionally the high colour count allows you to + create very colourful artwork. +

    + +

    + However on the flipside the colour restrictions of YJK make it a difficult mode + to work with. Below you will find some tips on how to work with the constraints + of the YJK modes. +

    + +

    + For pixel art I recommend to use the YJK+YAE mode. The availability of palette + colours greatly increases your flexibility to work around the colour clash + restrictions. The YJK mode is more suited for photographic material with + smoother gradients and lower contrast. +

    + +

    + The simplest technique is to approach the art like you would a SCREEN 5 image. + Draw the majority of the image using the 16 palette colours. Consider it a + layer on top of the YJK layer. Then “punch holes” through this layer by + introducing YJK colours in select places, adhering to the 4×1 pixel group + restrictions. The easiest way to do this is by simply using only a single + colour per pixel group. This already allows you to introduce many more colour + details without needing extra palette colours for them, while avoiding the + complexities of YJK. +

    + +

    + If you want to use multiple YJK colours within those 4 pixels, you need to pay + attention to the colours available in the specific J, K combination for that + quad. This requires a bit of planning, you’ll want to start by establishing a + few useful colour ramps to reuse them througout the art piece. Choose colours + from an Y ramp you like, for example using the tool at the top of this page, + and then note them in a little scratch area for easy colour picking. Then + whenever you need to draw e.g. some grassy bits, you can pick from the green + ramp you’ve noted, while being mindful of the 4×1 grid. +

    + +

    + For a more cartoony drawing style in YJK mode, you could draw the artwork in + grayscale, and then paint the colour on a separate colour modifier layer with + a blurry brush, so that there are no hard colour transitions. This will convert + to YJK well. +

    + +

    + In the end to which degree you want to apply these techniques depends on how much + time you want to invest. Please share your experiences and tips and tricks in the + msx.org + pixel art thread! +

    + + +

    Comparison to Y′UV

    + +

    + The YJK colour space is similar to the more common Y′UV + colour space, with 4×1 chroma subsampling like the Y′UV411 encoding. However + they differ in the weights applied to the different colour components. Where + Y′UV assigns weights to make the Y component express luma (brightness), YJK + has swapped the weights for green and blue, meaning that Y does not only affect + luma but also the amount of blue. +

    + +

    + This trait can be a bit unfortunate, as in natural scenes the amount of blue + decreases under (sun)light, and increases in shade. Y′UV would work better + in those situations. YJK is perhaps better suited for metallic, fluorescent + and pastel colours. +

    + +

    + In an interesting article “Issues + on YJK colour model implemented in Yamaha V9958 VDP chip”, Ricardo Cancho + Niemietz elaborates why this is not ideal from the perspective of the + luminance-chrominance colour model, and suggests that it must be an unintentional + mistake of the Yamaha engineers. Assuming though that Yamaha’s engineers were + smart people, and wouldn’t make a colour model inspired by YUV but deviate in + such a detail without reason, let’s try to find that reason. +

    + +

    + In YJK’s 4-byte encoding there are six bits for J and K, but only five for Y. + Due to this it can only represent half of the RGB colours, 214 + instead of the 215 that 15-bit RGB can produce. Ideally Y would have + 6 bits as well to get the full range, but this doesn’t fit in the byte encoding + (and certainly not in YJK+YAE). So which RGB colour component’s resolution do + we reduce by one bit? +

    + +

    + The most logical answer here is blue, just like in SCREEN 8. Because our eyes + are the least perceptive of blue, we won’t notice it as much. And indeed in YJK + blue is effectively only 4-bit, in YJK+YAE effectively 3-bit. Had Yamaha + gone for YUV, green would have gotten the least resolution, while it is the + most visible colour, causing more banding in it. From this perspective, YJK + spends its bits more effectively on the colours our eye perceive best. +

    + +

    + A final note on this; the Yamaha V9990 VDP has both an YJK and an YUV mode, + so on that VDP you can use the mode of your choosing. The YUV mode works + identical to YJK on the V9958 but with the weights for the G and B components + swapped. The weights do not perfectly match the real Y’UV but it’s pretty close. +

    + +

    Relevant links:

    + + + + +

    Grauw

    + + + + diff --git a/articles/yjk/matrix.js b/articles/yjk/matrix.js new file mode 100644 --- /dev/null +++ b/articles/yjk/matrix.js @@ -0,0 +1,126 @@ +import { Vector4 } from "./vector.js"; + +export class Matrix4x4 { + constructor(row0, row1, row2, row3) { + this.row0 = row0; + this.row1 = row1; + this.row2 = row2; + this.row3 = row3; + } + + static identity() { + return Matrix4x4.scale(Vector4.scalar(1)); + } + + static scale(vector) { + return new Matrix4x4( + new Vector4(vector.x, 0, 0, 0), + new Vector4(0, vector.y, 0, 0), + new Vector4(0, 0, vector.z, 0), + new Vector4(0, 0, 0, vector.w) + ); + } + + static translate(vector) { + return new Matrix4x4( + new Vector4(1, 0, 0, vector.x), + new Vector4(0, 1, 0, vector.y), + new Vector4(0, 0, 1, vector.z), + new Vector4(0, 0, 0, vector.w) + ); + } + + static rotateX(angle) { + return new Matrix4x4( + new Vector4(1, 0, 0, 0), + new Vector4(0, Math.cos(angle), -Math.sin(angle), 0), + new Vector4(0, Math.sin(angle), Math.cos(angle), 0), + new Vector4(0, 0, 0, 1) + ); + } + + static rotateY(angle) { + return new Matrix4x4( + new Vector4(Math.cos(angle), 0, -Math.sin(angle), 0), + new Vector4(0, 1, 0, 0), + new Vector4(Math.sin(angle), 0, Math.cos(angle), 0), + new Vector4(0, 0, 0, 1) + ); + } + + static rotateZ(angle) { + return new Matrix4x4( + new Vector4(Math.cos(angle), -Math.sin(angle), 0, 0), + new Vector4(Math.sin(angle), Math.cos(angle), 0, 0), + new Vector4(0, 0, 1, 0), + new Vector4(0, 0, 0, 1) + ); + } + + static rotateXYZ(vector) { + vector = vector.project(2 * Math.PI); + return Matrix4x4.rotateX(vector.x).multiply(Matrix4x4.rotateY(vector.y)).multiply(Matrix4x4.rotateZ(vector.z)); + } + + static orthogonal(width, height, depth, near) { + const far = near + depth; + return new Matrix4x4( + new Vector4(2 / width, 0, 0, 0), + new Vector4(0, 2 / height, 0, 0), + new Vector4(0, 0, -2 / depth, -(far + near) / depth), + new Vector4(0, 0, 0, 1) + ); + } + + static perspective(width, height, depth, near) { + const far = near + depth; + return new Matrix4x4( + new Vector4(2 * near / width, 0, 0, 0), + new Vector4(0, 2 * near / height, 0, 0), + new Vector4(0, 0, -(far + near) / depth, -2 * far * near / depth), + new Vector4(0, 0, -1, 0) + ); + } + + multiplyVector(vector) { + return new Vector4(this.row0.dot(vector), this.row1.dot(vector), this.row2.dot(vector), this.row3.dot(vector)); + } + + multiply(matrix) { + const transposed = matrix.transpose(); + return new Matrix4x4( + transposed.multiplyVector(this.row0), + transposed.multiplyVector(this.row1), + transposed.multiplyVector(this.row2), + transposed.multiplyVector(this.row3) + ); + } + + transpose() { + return new Matrix4x4( + new Vector4(this.row0.x, this.row1.x, this.row2.x, this.row3.x), + new Vector4(this.row0.y, this.row1.y, this.row2.y, this.row3.y), + new Vector4(this.row0.z, this.row1.z, this.row2.z, this.row3.z), + new Vector4(this.row0.w, this.row1.w, this.row2.w, this.row3.w) + ); + } + + project(scalar) { + const factor = Vector4.scalar(scalar / this.row3.w); + return new Matrix4x4(this.row0.multiply(factor), this.row1.multiply(factor), this.row2.multiply(factor), this.row3.multiply(factor)); + } + + toFloat32Array() { + const array = new Float32Array(16); + const matrix = this.transpose(); + matrix.row0.copyToArray(array, 0); + matrix.row1.copyToArray(array, 4); + matrix.row2.copyToArray(array, 8); + matrix.row3.copyToArray(array, 12); + return array; + } + + toString() { + return `({$this.row0}, {$this.row1}, {$this.row2}, {$this.row3})`; + } +} diff --git a/articles/yjk/vector.js b/articles/yjk/vector.js new file mode 100644 --- /dev/null +++ b/articles/yjk/vector.js @@ -0,0 +1,165 @@ +export function clamp(value, min, max) { + return Math.min(Math.max(value, min), max); +} + +export class Vector4 { + constructor(x, y, z, w) { + this.x = x; + this.y = y; + this.z = z; + this.w = w; + } + + static scalar(value) { + return new Vector4(value, value, value, value); + } + + transform(matrix) { + return matrix.multiplyVector(this); + } + + add(other) { + return new Vector4(this.x + other.x, this.y + other.y, this.z + other.z, this.w + other.w); + } + + addw(other) { + return new Vector4(this.x * other.w + other.x, this.y * other.w + other.y, this.z * other.w + other.z, this.w * other.w); + } + + subtract(other) { + return new Vector4(this.x - other.x, this.y - other.y, this.z - other.z, this.w - other.w); + } + + multiply(other) { + return new Vector4(this.x * other.x, this.y * other.y, this.z * other.z, this.w * other.w); + } + + average(other) { + return new Vector4( + this.x * other.w + other.x * this.w, + this.y * other.w + other.y * this.w, + this.z * other.w + other.z * this.w, + this.w * other.w + other.w * this.w + ); + } + + dot(other) { + return this.x * other.x + this.y * other.y + this.z * other.z + this.w * other.w; + } + + cross(other) { + if (this.w || other.w) + throw new Error("Cross product must use a 3D vector."); + return new Vector4(this.y * other.z - this.z * other.y, this.z * other.x - this.x * other.z, this.x * other.y - this.y * other.x, 0); + } + + magnitude() { + return Math.sqrt(this.magnitudeSquared()); + } + + magnitudeSquared() { + return this.dot(this); + } + + normalize() { + return this.multiply(Vector4.scalar(1 / this.magnitude())); + } + + isZero(threshold) { + return this.magnitudeSquared() <= (threshold || 0); + } + + power(exponent) { + return new Vector4(Math.pow(this.x, exponent.x), Math.pow(this.y, exponent.y), Math.pow(this.z, exponent.z), Math.pow(this.w, exponent.w)) + } + + project(scale) { + return new Vector4(this.x * scale / this.w, this.y * scale / this.w, this.z * scale / this.w, scale); + } + + min(other) { + return new Vector4(Math.min(this.x, other.x), Math.min(this.y, other.y), Math.min(this.z, other.z), Math.min(this.w, other.w)) + } + + max(other) { + return new Vector4(Math.max(this.x, other.x), Math.max(this.y, other.y), Math.max(this.z, other.z), Math.max(this.w, other.w)) + } + + clamp(min, max) { + return min.max(max.min(this)); + } + + saturate() { + return new Vector4(clamp(this.x, 0, this.w), clamp(this.y, 0, this.w), clamp(this.z, 0, this.w), this.w); + } + + floor() { + return new Vector4(Math.floor(this.x), Math.floor(this.y), Math.floor(this.z), Math.floor(this.w)); + } + + ceil() { + return new Vector4(Math.ceil(this.x), Math.ceil(this.y), Math.ceil(this.z), Math.ceil(this.w)); + } + + round() { + return new Vector4(Math.round(this.x), Math.round(this.y), Math.round(this.z), Math.round(this.w)); + } + + equals(other) { + return this.x * other.w == other.x * this.w && this.y * other.w == other.y * this.w && this.z * other.w == other.z * this.w; + } + + almostEquals(other) { + return Math.abs(this.x * other.w - other.x * this.w) < 0.01 && + Math.abs(this.y * other.w - other.y * this.w) < 0.01 && + Math.abs(this.z * other.w - other.z * this.w) < 0.01; + } + + isInRange() { + return this.x >= 0 && this.x <= this.w && this.y >= 0 && this.y <= this.w && this.z >= 0 && this.z <= this.w; + } + + isInInnerRange() { + return this.x > 0 && this.x < this.w && this.y > 0 && this.y < this.w && this.z > 0 && this.z < this.w; + } + + toRGB() { + return new Vector4(this.x + this.y, this.x + this.z, Math.floor(1.25 * this.x + -0.5 * this.y + -0.25 * this.z), this.w); + } + + toYJK() { + const y = Math.ceil(0.25 * this.x + 0.125 * this.y + 0.5 * this.z); + return new Vector4(y, this.x - y, this.y - y, this.w); + } + + toYJKYAE() { + const y = 2 * Math.ceil((0.25 * this.x + 0.125 * this.y + 0.5 * this.z) / 2); + return new Vector4(y, this.x - y, this.y - y, this.w); + } + + toString() { + return `(${this.x}, ${this.y}, ${this.z}, ${this.w})`; + } + + toFloat32Array() { + return Vector4.toFloat32Array(this); + } + + static toFloat32Array(...vectors) { + const array = new Float32Array(vectors.length * 4); + Vector4.copyToArray(array, ...vectors); + return array; + } + + static copyToArray(array, ...vectors) { + for (let i = 0; i < vectors.length; i++) + vectors[i].copyToArray(array, i * 4); + } + + copyToArray(array, offset) { + array[offset + 0] = this.x; + array[offset + 1] = this.y; + array[offset + 2] = this.z; + array[offset + 3] = this.w; + } +} diff --git a/articles/yjk/visualizer.js b/articles/yjk/visualizer.js new file mode 100644 --- /dev/null +++ b/articles/yjk/visualizer.js @@ -0,0 +1,207 @@ +import { Vector4, clamp } from "./vector.js"; + +export class Visualizer { + constructor(container) { + this.jkCanvas = container.querySelector("canvas.jk"); + this.yCanvas = container.querySelector("canvas.y"); + this.yInput = container.querySelector("input.y"); + this.jInput = container.querySelector("input.j"); + this.kInput = container.querySelector("input.k"); + this.rInput = container.querySelector("input.r"); + this.gInput = container.querySelector("input.g"); + this.bInput = container.querySelector("input.b"); + this.rangeCheckbox = container.querySelector("input.range"); + this.yaeCheckbox = container.querySelector("input.yae"); + this.editingInput = null; + this.yjk = new Vector4(16, 0, 0, 31); + this.gamma = 1.1; + this.clipRange = this.rangeCheckbox.checked; + this.yStep = this.yaeCheckbox.checked ? 2 : 1; + + this.setYJK(this.getYJKInput()); + this.update(); + + addSlideListener(this.jkCanvas, this.onJKCanvasClick.bind(this)); + addSlideListener(this.yCanvas, this.onYCanvasClick.bind(this)); + addArrowKeyListener(this.jkCanvas, this.onJKCanvasArrowKey.bind(this)); + addArrowKeyListener(this.yCanvas, this.onYCanvasArrowKey.bind(this)); + addNumberInputListener(this.yInput, this.onYJKInputChange.bind(this)); + addNumberInputListener(this.jInput, this.onYJKInputChange.bind(this)); + addNumberInputListener(this.kInput, this.onYJKInputChange.bind(this)); + addNumberInputListener(this.rInput, this.onRGBInputChange.bind(this)); + addNumberInputListener(this.gInput, this.onRGBInputChange.bind(this)); + addNumberInputListener(this.bInput, this.onRGBInputChange.bind(this)); + this.rangeCheckbox.addEventListener("change", this.onRangeCheckboxChange.bind(this)); + this.yaeCheckbox.addEventListener("change", this.onYAECheckboxChange.bind(this)); + } + + setYJK(yjk) { + this.yjk = new Vector4(yjk.x & -this.yStep, yjk.y, yjk.z, yjk.w); + } + + getYJKInput() { + return new Vector4( + clamp(Math.round(this.yInput.value), 0, 31), + clamp(Math.round(this.jInput.value), -32, 31), + clamp(Math.round(this.kInput.value), -32, 31), + 31 + ); + } + + getRGBInput() { + return new Vector4( + clamp(Math.round(this.rInput.value), 0, 31), + clamp(Math.round(this.gInput.value), 0, 31), + clamp(Math.round(this.bInput.value), 0, 31), + 31 + ); + } + + onJKCanvasClick(event) { + this.setYJK(new Vector4( + this.yjk.x, + clamp(Math.floor(event.offsetX * 64 / this.jkCanvas.width - 32), -32, 31), + clamp(Math.floor(event.offsetY * 64 / this.jkCanvas.height - 32), -32, 31), + this.yjk.w + )); + this.update(); + } + + onYCanvasClick(event) { + this.setYJK(new Vector4( + clamp(Math.floor(event.offsetY * 32 / this.yStep / this.yCanvas.height) * this.yStep, 0, 31), + this.yjk.y, + this.yjk.z, + this.yjk.w + )); + this.update(); + } + + onJKCanvasArrowKey(x, y) { + this.setYJK(new Vector4(this.yjk.x, clamp(this.yjk.y + x, -32, 31), clamp(this.yjk.z + y, -32, 31), this.yjk.w)); + this.update(); + } + + onYCanvasArrowKey(x, y) { + this.setYJK(new Vector4(clamp(this.yjk.x + y * this.yStep, 0, 31), this.yjk.y, this.yjk.z, this.yjk.w)); + this.update(); + } + + onYJKInputChange(event) { + this.editingInput = event.type == "input" ? event.target : null; + this.setYJK(this.getYJKInput()); + this.update(); + } + + onRGBInputChange(event) { + this.editingInput = event.type == "input" ? event.target : null; + this.setYJK(this.getRGBInput().toYJK()); + this.update(); + } + + onRangeCheckboxChange(event) { + this.clipRange = this.rangeCheckbox.checked; + this.update(); + } + + onYAECheckboxChange(event) { + this.yStep = this.yaeCheckbox.checked ? 2 : 1; + this.setYJK(this.yjk); + this.update(); + } + + update() { + this.updateJKCanvas(); + this.updateYCanvas(); + this.updateInputs(); + } + + updateJKCanvas() { + const width = this.jkCanvas.width / 64; + const height = this.jkCanvas.height / 64; + const context = this.jkCanvas.getContext("2d"); + context.clearRect(0, 0, this.jkCanvas.width, this.jkCanvas.height); + for (let k = -32; k < 32; ++k) { + for (let j = -32; j < 32; ++j) { + const rgb = new Vector4(this.yjk.x, j, k, this.yjk.w).toRGB(); + const a = this.clipRange && !rgb.isInRange() ? 0.5 : 1; + const rgb24 = rgb.saturate().power(Vector4.scalar(this.gamma)).project(255); + context.fillStyle = `rgba(${rgb24.x}, ${rgb24.y}, ${rgb24.z}, ${a})`; + context.fillRect((j + 32) * width, (k + 32) * height, width, height); + } + } + context.lineWidth = 2; + context.strokeStyle = "rgb(255, 255, 255)" + context.strokeRect((this.yjk.y + 32) * width, (this.yjk.z + 32) * height, width, height); + } + + updateYCanvas() { + const width = this.yCanvas.width; + const height = this.yCanvas.height / 32; + const context = this.yCanvas.getContext("2d"); + context.clearRect(0, 0, this.yCanvas.width, this.yCanvas.height); + for (let y = 0; y < 32; y += this.yStep) { + const rgb = new Vector4(y, this.yjk.y, this.yjk.z, this.yjk.w).toRGB(); + const a = this.clipRange && !rgb.isInRange() ? 0.5 : 1; + const rgb24 = rgb.saturate().power(Vector4.scalar(this.gamma)).project(255); + context.fillStyle = `rgba(${rgb24.x}, ${rgb24.y}, ${rgb24.z}, ${a})`; + context.fillRect(0, y * height, width, this.yStep * height); + } + context.lineWidth = 2; + context.strokeStyle = "rgb(255, 255, 255)" + context.strokeRect(1, this.yjk.x * height, width - 2, this.yStep * height); + } + + updateInputs() { + this.yInput.step = this.yStep; + if (this.yInput != this.editingInput) + this.yInput.value = this.yjk.x; + if (this.jInput != this.editingInput) + this.jInput.value = this.yjk.y; + if (this.kInput != this.editingInput) + this.kInput.value = this.yjk.z; + if (this.rInput != this.editingInput) + this.rInput.value = this.yjk.toRGB().x; + if (this.gInput != this.editingInput) + this.gInput.value = this.yjk.toRGB().y; + if (this.bInput != this.editingInput) + this.bInput.value = this.yjk.toRGB().z; + } +} + +function addSlideListener(element, listener) { + element.addEventListener("pointerdown", event => { + element.setPointerCapture(event.pointerId); + element.addEventListener("pointermove", listener); + listener(event); + }); + element.addEventListener("pointerup", event => { + element.releasePointerCapture(event.pointerId); + element.removeEventListener("pointermove", listener); + listener(event); + }); + element.addEventListener("pointercancel", event => { + element.releasePointerCapture(event.pointerId); + element.removeEventListener("pointermove", listener); + }); +} + +function addArrowKeyListener(element, listener) { + element.tabIndex = 0; + element.addEventListener("keydown", event => { + const x = event.code == "ArrowLeft" ? -1 : event.code == "ArrowRight" ? 1 : 0; + const y = event.code == "ArrowUp" ? -1 : event.code == "ArrowDown" ? 1 : 0; + if (x || y) { + listener(x, y); + event.preventDefault(); + } + }); +} + +function addNumberInputListener(element, listener) { + element.addEventListener("change", listener); + element.addEventListener("input", event => { + if (element.value && !isNaN(element.value)) + listener(event); + }); +} diff --git a/articles/yjk/yjk.css b/articles/yjk/yjk.css new file mode 100644 --- /dev/null +++ b/articles/yjk/yjk.css @@ -0,0 +1,77 @@ +.tool { + background-color: #5D5D7F; + box-shadow: 0 0 16px #3E3E55; + color: #EEF; + text-align: center; + border-radius: 7px; + margin: 8px 16px; + padding: 2px 2px; +} + +.tool.visualizer { + width: 512px; + float: right; + clear: both; +} + +.tool figcaption, +.tool p { + color: #EEF; + margin: 8px 0; +} + +.tool figcaption { + font-style: italic; +} + +.tool .charts { + display: flex; + justify-content: center; +} + +.tool canvas { + border-radius: 2px; + cursor: crosshair; + margin: 0 4px; +} + +.tool canvas:focus { + outline: none; +} + +.tool label { + user-select: none; +} + +.tool .controls { + display: flex; + justify-content: space-evenly; +} + +.tool .controls input[type=number] { + background-color: #7C7CAA; /* #5D5D7F; */ + color: #EEF; + border: 0; + padding: 3px; + text-align: center; + border-radius: 4px; + font-size: small; +} + +.tool .controls input[type=number]:invalid { + box-shadow: 0 0 1px 1px red; +} + +@media screen and (max-width: 896px) { + .tool { + float: none; + margin-left: auto; + margin-right: auto; + } +} + +@media screen and (max-width: 512px) { + .tool { + width: 384px; + } +} diff --git a/articles/yjk/yjk.js b/articles/yjk/yjk.js new file mode 100644 --- /dev/null +++ b/articles/yjk/yjk.js @@ -0,0 +1,188 @@ +import { Vector4 } from "./vector.js"; +import { Matrix4x4 } from "./matrix.js"; + +export const yjkToRGB = new Matrix4x4( + new Vector4(4, 4, 0, 0), + new Vector4(4, 0, 4, 0), + new Vector4(5, -2, -1, 0), + new Vector4(0, 0, 0, 4) +); + +export const rgbToYJK = new Matrix4x4( + new Vector4(2, 1, 4, 0), + new Vector4(6, -1, -4, 0), + new Vector4(-2, 7, -4, 0), + new Vector4(0, 0, 0, 8) +); + +export const yjkYAEToRGB = new Matrix4x4( + new Vector4(8, 4, 0, 0), + new Vector4(8, 0, 4, 0), + new Vector4(10, -2, -1, 0), + new Vector4(0, 0, 0, 4) +); + +export const rgbToYJKYAE = new Matrix4x4( + new Vector4(1, 0.5, 2, 0), + new Vector4(6, -1, -4, 0), + new Vector4(-2, 7, -4, 0), + new Vector4(0, 0, 0, 8) +); + +export const yuvToRGB = new Matrix4x4( + new Vector4(4, 4, 0, 0), + new Vector4(5, -2, -1, 0), + new Vector4(4, 0, 4, 0), + new Vector4(0, 0, 0, 4) +); + +export const rgbToYUV = new Matrix4x4( + new Vector4(2, 4, 1, 0), + new Vector4(6, -4, -1, 0), + new Vector4(-2, -4, 7, 0), + new Vector4(0, 0, 0, 8) +); + +export const yuvYAEToRGB = new Matrix4x4( + new Vector4(8, 4, 0, 0), + new Vector4(10, -2, -1, 0), + new Vector4(8, 0, 4, 0), + new Vector4(0, 0, 0, 4) +); + +export const rgbToYUVYAE = new Matrix4x4( + new Vector4(1, 2, 0.5, 0), + new Vector4(6, -4, -1, 0), + new Vector4(-2, -4, 7, 0), + new Vector4(0, 0, 0, 8) +); + +export const yuvToRGB601 = new Matrix4x4( + new Vector4(1, 0, 0.701 / 0.615, 0), + new Vector4(1, -0.114 * 0.886 / 0.436 / 0.587, -0.299 * 0.701 / 0.615 / 0.587, 0), + new Vector4(1, 0.886 / 0.436, 0, 0), + new Vector4(0, 0, 0, 1) +); + +export const rgbToYUV601 = new Matrix4x4( + new Vector4(0.299, 0.587, 0.114, 0), + new Vector4(-0.299 * 0.436 / 0.886, -0.587 * 0.436 / 0.886, 0.436, 0), + new Vector4(0.615, -0.587 * 0.615 / 0.701, -0.114 * 0.615 / 0.701, 0), + new Vector4(0, 0, 0, 1) +); + +export class Image { + constructor(width, height) { + this.width = width; + this.height = height; + } + + getPixel(x, y) { + throw new Error("Not implemented."); + } + + map(fn) { + const data = []; + for (let y = 0; y < this.height; ++y) + for (let x = 0; x < this.width; ++x) + data.push(fn(this.getPixel(x, y), x, y)); + return new ImageData(this.width, this.height, data); + } + + filter(filter) { + const data = []; + for (let y = 0; y < this.height; ++y) { + for (let x = 0; x < this.width; ++x) { + let value = new Vector4(0, 0, 0, 0); + for (let i = 0; i < filter.kernel.length; ++i) { + const ix = x + i - filter.center; + if (ix >= 0 && ix < this.width) + value = value.add(this.getPixel(ix, y).project(filter.kernel[i])); + } + data.push(value); + } + } + return new ImageData(this.width, this.height, data); + } + + subsampleChroma1(filter, toRGB) { + const filteredImage = this.filter(filter); + // const yjkToY601 = new Vector4(1.0285, 0.242, 0.5585, 0); + const yjkToY601 = rgbToYUV601.multiply(toRGB).row0; + const yjkLuma = yjk => yjk.dot(yjkToY601) / yjkToY601.x; + return this.map((yjk, x, y) => { + const yjkScaled = yjk.project(31); + const sample = filteredImage.getPixel(x & ~3, y).project(31); + return new Vector4(sample.x + yjkLuma(yjkScaled) - yjkLuma(sample), sample.y, sample.z, sample.w); + }); + } + + subsampleChroma2(filter, toRGB) { + const filteredImage = this.filter(filter); + const yjkLuminance = yjk => yjk.transform(toRGB).saturate().power(Vector4.scalar(1)).dot(rgbToYUV601.row0); + return this.map((yjk, x, y) => { + const yjkScaled = yjk.project(31); + const sample = filteredImage.getPixel(x & ~3, y).project(31); + const targetLuminance = yjkLuminance(yjkScaled); + let bestDistance = Infinity; + let bestY = 0; + for (let y = 0; y < 32; y++) { + const candidate = new Vector4(y, sample.y, sample.z, sample.w); + const distance = Math.abs(yjkLuminance(candidate) - targetLuminance); + if (distance < bestDistance) { + bestDistance = distance; + bestY = y; + } + } + return new Vector4(bestY, sample.y, sample.z, sample.w); + }); + } +} + +export class ImageData extends Image { + constructor(width, height, data) { + super(width, height); + if (data.length != width * height) + throw new Error("Incomplete data."); + this.data = data; + } + + getPixel(x, y) { + return this.data[y * this.width + x]; + } +} + +export class CanvasImage extends ImageData { + constructor(canvas, x, y, width, height) { + const imageData = canvas.getContext("2d").getImageData(x || 0, y || 0, width || canvas.width, height || canvas.height).data; + const data = []; + for (let i = 0; i < imageData.length; i += 4) { + data.push(new Vector4(imageData[i], imageData[i + 1], imageData[i + 2], 255)); + } + super(canvas.width, canvas.height, data); + } +} + +export class Filter { + constructor(kernel, center) { + if (center < 0 || center >= kernel.length) + throw new Error("Center outside kernel range."); + this.kernel = kernel; + this.center = center; + } + + normalize(w) { + const sum = this.kernel.reduce((accu, current) => accu + current); + return new Filter(this.kernel.map(value => value * w / sum), this.center); + } +} + +export class BoxFilter extends Filter { + constructor(size, center) { + super(new Array(size).fill(1), center || 0); + } +} + +export const identityFilter = new Filter([1], 0); +export const boxFilter = new BoxFilter(4, 0); +export const tentFilter = new Filter([1, 3, 4, 4, 5, 4, 4, 3, 1], 3);