ImageBitmap.webidl (15471B)
1 /* -*- Mode: IDL; tab-width: 2; indent-tabs-mode: nil; c-basic-offset: 2 -*- */ 2 /* This Source Code Form is subject to the terms of the Mozilla Public 3 * License, v. 2.0. If a copy of the MPL was not distributed with this file, 4 * You can obtain one at http://mozilla.org/MPL/2.0/. 5 * 6 * The origin of this IDL file is 7 * https://html.spec.whatwg.org/multipage/webappapis.html#images 8 * 9 * The origin of the extended IDL file is 10 * http://w3c.github.io/mediacapture-worker/#imagebitmap-extensions 11 */ 12 13 typedef (CanvasImageSource or 14 Blob or 15 CanvasRenderingContext2D or // This is out of spec. 16 ImageData) ImageBitmapSource; 17 18 [Exposed=(Window,Worker)] 19 interface ImageBitmap { 20 [Constant] 21 readonly attribute unsigned long width; 22 [Constant] 23 readonly attribute unsigned long height; 24 }; 25 26 // It's crucial that there be a way to explicitly dispose of ImageBitmaps 27 // since they refer to potentially large graphics resources. Some uses 28 // of this API proposal will result in repeated allocations of ImageBitmaps, 29 // and garbage collection will not reliably reclaim them quickly enough. 30 // Here we reuse close(), which also exists on another Transferable type, 31 // MessagePort. Potentially, all Transferable types should inherit from a 32 // new interface type "Closeable". 33 partial interface ImageBitmap { 34 // Dispose of all graphical resources associated with this ImageBitmap. 35 undefined close(); 36 }; 37 38 // ImageBitmap-extensions 39 // Bug 1141979 - [FoxEye] Extend ImageBitmap with interfaces to access its 40 // underlying image data 41 42 /* 43 * An image or a video frame is conceptually a two-dimensional array of data and 44 * each element in the array is called a pixel. The pixels are usually stored in 45 * a one-dimensional array and could be arranged in a variety of image formats. 46 * Developers need to know how the pixels are formatted so that they are able to 47 * process them. 48 * 49 * The image format describes how pixels in an image are arranged. A single 50 * pixel has at least one, but usually multiple pixel values. The range of a 51 * pixel value varies, which means different image formats use different data 52 * types to store a single pixel value. 53 * 54 * The most frequently used data type is 8-bit unsigned integer whose range is 55 * from 0 to 255, others could be 16-bit integer or 32-bit floating points and 56 * so forth. The number of pixel values of a single pixel is called the number 57 * of channels of the image format. Multiple pixel values of a pixel are used 58 * together to describe the captured property which could be color or depth 59 * information. For example, if the data is a color image in RGB color space, 60 * then it is a three-channel image format and a pixel is described by R, G and 61 * B three pixel values with range from 0 to 255. As another example, if the 62 * data is a gray image, then it is a single-channel image format with 8-bit 63 * unsigned integer data type and the pixel value describes the gray scale. For 64 * depth data, it is a single channel image format too, but the data type is 65 * 16-bit unsigned integer and the pixel value is the depth level. 66 * 67 * For those image formats whose pixels contain multiple pixel values, the pixel 68 * values might be arranged in one of the following ways: 69 * 1) Planar pixel layout: 70 * each channel has its pixel values stored consecutively in separated 71 * buffers (a.k.a. planes) and then all channel buffers are stored 72 * consecutively in memory. 73 * (Ex: RRRRRR......GGGGGG......BBBBBB......) 74 * 2) Interleaving pixel layout: 75 * each pixel has its pixel values from all channels stored together and 76 * interleaves all channels. 77 * (Ex: RGBRGBRGBRGBRGB......) 78 */ 79 80 81 /* 82 * The ImageBitmap extensions use this enumeration to negotiate the image format 83 * while 1) accessing the underlying data of an ImageBitmap and 84 * 2) creating a new ImageBitmap. 85 * 86 * For each format in this enumeration, we use a 2x2 small image (4 pixels) as 87 * example to illustrate the pixel layout. 88 * 89 * 2x2 image: +--------+--------+ 90 * | pixel1 | pixel2 | 91 * +--------+--------+ 92 * | pixel3 | pixel4 | 93 * +--------+--------+ 94 * 95 */ 96 enum ImageBitmapFormat { 97 /* 98 * Channel order: R, G, B, A 99 * Channel size: full rgba-chennels 100 * Pixel layout: interleaving rgba-channels 101 * Pixel layout illustration: 102 * [Plane1]: R1 G1 B1 A1 R2 G2 B2 A2 R3 G3 B3 A3 R4 G4 B4 A4 103 * Data type: 8-bit unsigned integer 104 */ 105 "RGBA32", 106 107 /* 108 * Channel order: B, G, R, A 109 * Channel size: full bgra-channels 110 * Pixel layout: interleaving bgra-channels 111 * Pixel layout illustration: 112 * [Plane1]: B1 G1 R1 A1 B2 G2 R2 A2 B3 G3 R3 A3 B4 G4 R4 A4 113 * Data type: 8-bit unsigned integer 114 */ 115 "BGRA32", 116 117 /* 118 * Channel order: R, G, B 119 * Channel size: full rgb-channels 120 * Pixel layout: interleaving rgb-channels 121 * Pixel layout illustration: 122 * [Plane1]: R1 G1 B1 R2 G2 B2 R3 G3 B3 R4 G4 B4 123 * Data type: 8-bit unsigned integer 124 */ 125 "RGB24", 126 127 /* 128 * Channel order: B, G, R 129 * Channel size: full bgr-channels 130 * Pixel layout: interleaving bgr-channels 131 * Pixel layout illustration: 132 * [Plane1]: B1 G1 R1 B2 G2 R2 B3 G3 R3 B4 G4 R4 133 * Data type: 8-bit unsigned integer 134 */ 135 "BGR24", 136 137 /* 138 * Channel order: GRAY 139 * Channel size: full gray-channel 140 * Pixel layout: planar gray-channel 141 * Pixel layout illustration: 142 * [Plane1]: GRAY1 GRAY2 GRAY3 GRAY4 143 * Data type: 8-bit unsigned integer 144 */ 145 "GRAY8", 146 147 /* 148 * Channel order: Y, U, V 149 * Channel size: full yuv-channels 150 * Pixel layout: planar yuv-channels 151 * Pixel layout illustration: 152 * [Plane1]: Y1 Y2 Y3 Y4 153 * [Plane2]: U1 U2 U3 U4 154 * [Plane3]: V1 V2 V3 V4 155 * Data type: 8-bit unsigned integer 156 */ 157 "YUV444P", 158 159 /* 160 * Channel order: Y, U, V 161 * Channel size: full y-channel, half uv-channels 162 * Pixel layout: planar yuv-channels 163 * Pixel layout illustration: 164 * [Plane1]: Y1 Y2 Y3 Y4 165 * [Plane2]: U1 U3 166 * [Plane3]: V1 V3 167 * Data type: 8-bit unsigned integer 168 */ 169 "YUV422P", 170 171 /* 172 * Channel order: Y, U, V 173 * Channel size: full y-channel, quarter uv-channels 174 * Pixel layout: planar yuv-channels 175 * Pixel layout illustration: 176 * [Plane1]: Y1 Y2 Y3 Y4 177 * [Plane2]: U1 178 * [Plane3]: V1 179 * Data type: 8-bit unsigned integer 180 */ 181 "YUV420P", 182 183 /* 184 * Channel order: Y, U, V 185 * Channel size: full y-channel, quarter uv-channels 186 * Pixel layout: planar y-channel, interleaving uv-channels 187 * Pixel layout illustration: 188 * [Plane1]: Y1 Y2 Y3 Y4 189 * [Plane2]: U1 V1 190 * Data type: 8-bit unsigned integer 191 */ 192 "YUV420SP_NV12", 193 194 /* 195 * Channel order: Y, V, U 196 * Channel size: full y-channel, quarter vu-channels 197 * Pixel layout: planar y-channel, interleaving vu-channels 198 * Pixel layout illustration: 199 * [Plane1]: Y1 Y2 Y3 Y4 200 * [Plane2]: V1 U1 201 * Data type: 8-bit unsigned integer 202 */ 203 "YUV420SP_NV21", 204 205 /* 206 * Channel order: H, S, V 207 * Channel size: full hsv-channels 208 * Pixel layout: interleaving hsv-channels 209 * Pixel layout illustration: 210 * [Plane1]: H1 S1 V1 H2 S2 V2 H3 S3 V3 211 * Data type: 32-bit floating point value 212 */ 213 "HSV", 214 215 /* 216 * Channel order: l, a, b 217 * Channel size: full lab-channels 218 * Pixel layout: interleaving lab-channels 219 * Pixel layout illustration: 220 * [Plane1]: l1 a1 b1 l2 a2 b2 l3 a3 b3 221 * Data type: 32-bit floating point value 222 */ 223 "Lab", 224 225 /* 226 * Channel order: DEPTH 227 * Channel size: full depth-channel 228 * Pixel layout: planar depth-channel 229 * Pixel layout illustration: 230 * [Plane1]: DEPTH1 DEPTH2 DEPTH3 DEPTH4 231 * Data type: 16-bit unsigned integer 232 */ 233 "DEPTH", 234 }; 235 236 enum ChannelPixelLayoutDataType { 237 "uint8", 238 "int8", 239 "uint16", 240 "int16", 241 "uint32", 242 "int32", 243 "float32", 244 "float64" 245 }; 246 247 /* 248 * Two concepts, ImagePixelLayout and ChannelPixelLayout, together generalize 249 * the variety of pixel layouts among image formats. 250 * 251 * The ChannelPixelLayout represents the pixel layout of a single channel in a 252 * certain image format and the ImagePixelLayout is just the collection of 253 * ChannelPixelLayouts. So, the ChannelPixelLayout is defined as a dictionary 254 * type with properties to describe the layout and the ImagePixelLayout is just 255 * an alias name to a sequence of ChannelPixelLayout objects. 256 * 257 * Since an image format is composed of at least one channel, an 258 * ImagePixelLayout object contains at least one ChannelPixelLayout object. 259 * 260 * Although an image or a video frame is a two-dimensional structure, its data 261 * is usually stored in a one-dimensional array in the row-major way and the 262 * ChannelPixelLayout objects use the following properties to describe the 263 * layout of pixel values in the buffer. 264 * 265 * 1) offset: 266 * denotes the beginning position of the channel's data relative to the 267 * beginning position of the one-dimensional array. 268 * 2) width & height: 269 * denote the width and height of the channel respectively. Each channel in 270 * an image format may have different height and width. 271 * 3) data type: 272 * denotes the format used to store one single pixel value. 273 * 4) stride: 274 * the number of bytes between the beginning two consecutive rows in memory. 275 * (The total bytes of each row plus the padding bytes of each raw.) 276 * 5) skip value: 277 * the value is zero for the planar pixel layout, and a positive integer for 278 * the interleaving pixel layout. (Describes how many bytes there are between 279 * two adjacent pixel values in this channel.) 280 */ 281 282 /* 283 * Example1: RGBA image, width = 620, height = 480, stride = 2560 284 * 285 * chanel_r: offset = 0, width = 620, height = 480, data type = uint8, stride = 2560, skip = 3 286 * chanel_g: offset = 1, width = 620, height = 480, data type = uint8, stride = 2560, skip = 3 287 * chanel_b: offset = 2, width = 620, height = 480, data type = uint8, stride = 2560, skip = 3 288 * chanel_a: offset = 3, width = 620, height = 480, data type = uint8, stride = 2560, skip = 3 289 * 290 * <---------------------------- stride ----------------------------> 291 * <---------------------- width x 4 ----------------------> 292 * [index] 01234 8 12 16 20 24 28 2479 2559 293 * |||||---|---|---|---|---|---|----------------------------|-------| 294 * [data] RGBARGBARGBARGBARGBAR___R___R... A%%%%%%%% 295 * [data] RGBARGBARGBARGBARGBAR___R___R... A%%%%%%%% 296 * [data] RGBARGBARGBARGBARGBAR___R___R... A%%%%%%%% 297 * ^^^ 298 * r-skip 299 */ 300 301 /* 302 * Example2: YUV420P image, width = 620, height = 480, stride = 640 303 * 304 * chanel_y: offset = 0, width = 620, height = 480, stride = 640, skip = 0 305 * chanel_u: offset = 307200, width = 310, height = 240, data type = uint8, stride = 320, skip = 0 306 * chanel_v: offset = 384000, width = 310, height = 240, data type = uint8, stride = 320, skip = 0 307 * 308 * <--------------------------- y-stride ---------------------------> 309 * <----------------------- y-width -----------------------> 310 * [index] 012345 619 639 311 * ||||||--------------------------------------------------|--------| 312 * [data] YYYYYYYYYYYYYYYYYYYYYYYYYYYYY... Y%%%%%%%%% 313 * [data] YYYYYYYYYYYYYYYYYYYYYYYYYYYYY... Y%%%%%%%%% 314 * [data] YYYYYYYYYYYYYYYYYYYYYYYYYYYYY... Y%%%%%%%%% 315 * [data] ...... 316 * <-------- u-stride ----------> 317 * <----- u-width -----> 318 * [index] 307200 307509 307519 319 * |-------------------|--------| 320 * [data] UUUUUUUUUU... U%%%%%%%%% 321 * [data] UUUUUUUUUU... U%%%%%%%%% 322 * [data] UUUUUUUUUU... U%%%%%%%%% 323 * [data] ...... 324 * <-------- v-stride ----------> 325 * <- --- v-width -----> 326 * [index] 384000 384309 384319 327 * |-------------------|--------| 328 * [data] VVVVVVVVVV... V%%%%%%%%% 329 * [data] VVVVVVVVVV... V%%%%%%%%% 330 * [data] VVVVVVVVVV... V%%%%%%%%% 331 * [data] ...... 332 */ 333 334 /* 335 * Example3: YUV420SP_NV12 image, width = 620, height = 480, stride = 640 336 * 337 * chanel_y: offset = 0, width = 620, height = 480, stride = 640, skip = 0 338 * chanel_u: offset = 307200, width = 310, height = 240, data type = uint8, stride = 640, skip = 1 339 * chanel_v: offset = 307201, width = 310, height = 240, data type = uint8, stride = 640, skip = 1 340 * 341 * <--------------------------- y-stride --------------------------> 342 * <----------------------- y-width ----------------------> 343 * [index] 012345 619 639 344 * ||||||-------------------------------------------------|--------| 345 * [data] YYYYYYYYYYYYYYYYYYYYYYYYYYYYY... Y%%%%%%%%% 346 * [data] YYYYYYYYYYYYYYYYYYYYYYYYYYYYY... Y%%%%%%%%% 347 * [data] YYYYYYYYYYYYYYYYYYYYYYYYYYYYY... Y%%%%%%%%% 348 * [data] ...... 349 * <--------------------- u-stride / v-stride --------------------> 350 * <------------------ u-width + v-width -----------------> 351 * [index] 307200(u-offset) 307819 307839 352 * |------------------------------------------------------|-------| 353 * [index] |307201(v-offset) |307820 | 354 * ||-----------------------------------------------------||------| 355 * [data] UVUVUVUVUVUVUVUVUVUVUVUVUVUVUV... UV%%%%%%% 356 * [data] UVUVUVUVUVUVUVUVUVUVUVUVUVUVUV... UV%%%%%%% 357 * [data] UVUVUVUVUVUVUVUVUVUVUVUVUVUVUV... UV%%%%%%% 358 * ^ ^ 359 * u-skip v-skip 360 */ 361 362 /* 363 * Example4: DEPTH image, width = 640, height = 480, stride = 1280 364 * 365 * chanel_d: offset = 0, width = 640, height = 480, data type = uint16, stride = 1280, skip = 0 366 * 367 * note: each DEPTH value uses two bytes 368 * 369 * <----------------------- d-stride ----------------------> 370 * <----------------------- d-width -----------------------> 371 * [index] 02468 1278 372 * |||||---------------------------------------------------| 373 * [data] DDDDDDDDDDDDDDDDDDDDDDDDDDDDD... D 374 * [data] DDDDDDDDDDDDDDDDDDDDDDDDDDDDD... D 375 * [data] DDDDDDDDDDDDDDDDDDDDDDDDDDDDD... D 376 * [data] ...... 377 */ 378 379 dictionary ChannelPixelLayout { 380 required unsigned long offset; 381 required unsigned long width; 382 required unsigned long height; 383 required ChannelPixelLayoutDataType dataType; 384 required unsigned long stride; 385 required unsigned long skip; 386 }; 387 388 typedef sequence<ChannelPixelLayout> ImagePixelLayout; 389 390 enum ImageOrientation { "none", "flipY", "from-image" }; 391 enum PremultiplyAlpha { "none", "premultiply", "default" }; 392 enum ColorSpaceConversion { "none", "default" }; 393 //enum ResizeQuality { "pixelated", "low", "medium", "high" }; 394 395 dictionary ImageBitmapOptions { 396 ImageOrientation imageOrientation = "none"; 397 PremultiplyAlpha premultiplyAlpha = "default"; 398 // options to be added bugs: 1363861 399 ColorSpaceConversion colorSpaceConversion = "default"; 400 [EnforceRange] unsigned long resizeWidth; 401 [EnforceRange] unsigned long resizeHeight; 402 //ResizeQuality resizeQuality = "low"; 403 };