index.html (20203B)
1 <!DOCTYPE html> 2 <html> 3 <head> 4 <title>CSS Writing Modes testing strategy</title> 5 <meta http-equiv='Content-Type' content='text/html;charset=utf-8'/> 6 <!-- 7 === NOTA BENE === 8 For the three scripts below, if your spec resides on dev.w3 you can check them 9 out in the same tree and use relative links so that they'll work offline, 10 --> 11 <script src='http://www.w3.org/Tools/respec/respec-w3c-common' class='remove' async></script> 12 <script class='remove'> 13 var respecConfig = { 14 specStatus: "unofficial", 15 shortName: "css3-writingmodes-test-strategy", 16 editors: [ 17 { 18 name: "Shinsuke Matsuki", mailto: "shinsuke.matsuki@access-company.com", 19 company: "ACCESS", companyURL: "" 20 }, 21 { 22 name: "Masataka Yakura", mailto: "masataka.yakura@gmail.com", 23 company: "", companyURL: "" 24 }, 25 ], 26 testSuiteURI: "http://test.csswg.org/suites/css3-writing-modes/nightly-unstable/", 27 }; 28 </script> 29 <style> 30 a.bibref, 31 #references dt { 32 text-transform: uppercase; 33 } 34 </style> 35 36 <style> 37 table 38 { 39 border-collapse:collapse; 40 } 41 table, td, th 42 { 43 border:1px solid black; 44 padding: 13px; 45 } 46 table 47 { 48 width: 100%; 49 } 50 img 51 { 52 width: 400px; 53 } 54 </style> 55 <style>/***************************************************************** 56 * ReSpec 3 CSS 57 * Robin Berjon - http://berjon.com/ 58 *****************************************************************/ 59 60 /* --- INLINES --- */ 61 em.rfc2119 { 62 text-transform: lowercase; 63 font-variant: small-caps; 64 font-style: normal; 65 color: #900; 66 } 67 68 h1 acronym, h2 acronym, h3 acronym, h4 acronym, h5 acronym, h6 acronym, a acronym, 69 h1 abbr, h2 abbr, h3 abbr, h4 abbr, h5 abbr, h6 abbr, a abbr { 70 border: none; 71 } 72 73 dfn { 74 font-weight: bold; 75 } 76 77 a.internalDFN { 78 color: inherit; 79 border-bottom: 1px solid #99c; 80 text-decoration: none; 81 } 82 83 a.externalDFN { 84 color: inherit; 85 border-bottom: 1px dotted #ccc; 86 text-decoration: none; 87 } 88 89 a.bibref { 90 text-decoration: none; 91 } 92 93 cite .bibref { 94 font-style: normal; 95 } 96 97 code { 98 color: #ff4500; 99 } 100 101 /* --- TOC --- */ 102 .toc a, .tof a { 103 text-decoration: none; 104 } 105 106 a .secno, a .figno { 107 color: #000; 108 } 109 110 ul.tof, ol.tof { 111 list-style: none outside none; 112 } 113 114 .caption { 115 margin-top: 0.5em; 116 font-style: italic; 117 } 118 119 /* --- TABLE --- */ 120 table.simple { 121 border-spacing: 0; 122 border-collapse: collapse; 123 border-bottom: 3px solid #005a9c; 124 } 125 126 .simple th { 127 background: #005a9c; 128 color: #fff; 129 padding: 3px 5px; 130 text-align: left; 131 } 132 133 .simple th[scope="row"] { 134 background: inherit; 135 color: inherit; 136 border-top: 1px solid #ddd; 137 } 138 139 .simple td { 140 padding: 3px 10px; 141 border-top: 1px solid #ddd; 142 } 143 144 .simple tr:nth-child(even) { 145 background: #f0f6ff; 146 } 147 148 /* --- DL --- */ 149 .section dd > p:first-child { 150 margin-top: 0; 151 } 152 153 .section dd > p:last-child { 154 margin-bottom: 0; 155 } 156 157 .section dd { 158 margin-bottom: 1em; 159 } 160 161 .section dl.attrs dd, .section dl.eldef dd { 162 margin-bottom: 0; 163 } 164 </style><link rel="stylesheet" href="https://www.w3.org/StyleSheets/TR/w3c-unofficial"><!--[if lt IE 9]><script src='https://www.w3.org/2008/site/js/html5shiv.js'></script><![endif]--> 165 166 </head> 167 <body> 168 <section id='abstract'> 169 <p> 170 This document is intended to be used as a guideline for the testing 171 activities related to the CSS Writing Modes spec [[!css3-writing-modes]]. Its main 172 goal is to provide an overview of the general testing areas, possible 173 caveats and testing aspects not immediately apparent from the spec. 174 Also, it provides a means of tracking the progress of the CSS Writing Modes 175 spec testing. 176 </p> 177 <p> 178 This document is not meant to replace the spec in determining the 179 normative and non-normative assertions to be tested, but rather 180 complement it. 181 </p> 182 </section> 183 184 <section> 185 <h2>Introduction</h2> 186 <p> 187 As CSS moved away from the monolithic development of CSS 2.1 to the 188 modular development of CSS 3, the number of proposed new features and 189 the complexity of the layout landscape have increased dramatically. 190 While this directly translates to increased flexibility and agility in 191 adopting and implementing new CSS features, it also increases the 192 complexity of testing CSS features and the need for coordinating the 193 testing efforts. Also, the need for testing coordination increases as 194 crowd-sourcing efforts like 195 <a href="http://testthewebforward.org/" target="_blank">Test the Web 196 Forward</a> present people less familiar with the processes and policies 197 of the W3C with the opportunity to contribute new tests. 198 </p> 199 <p> 200 Except when defining new behaviors or redefining old behaviors, the 201 implicit assumption for new CSS modules is that they play nicely with 202 other modules or properties defined in CSS 2.1 [[CSS21]]. As CSS 203 Writing Modes is a spec that touches many aspects of layout, styling and CSSOM, 204 it's not unreasonable to want to test the spec against these implicit 205 assumptions, too. 206 </p> 207 <p> 208 This testing strategy document is meant to complement the CSS Writing Modes 209 spec and the existing test suite by providing an overview of the testing 210 areas (especially the less apparent ones) and tracking the progress of 211 the testing activities against these test areas. 212 </p> 213 </section> 214 <section> 215 <h2>Goals</h2> 216 <p> 217 To ensure a comprehensive test suite with useful, high quality tests, a 218 number of goals are proposed. They range from process goals (how to 219 conduct testing) to implementation goals (how to write good tests). 220 </p> 221 <section> 222 <h3>Enabling easy test contribution</h3> 223 <p> 224 An important vector in successfully testing CSS Writing Modes is to 225 enable easy test contributions, both from W3C partners and from 226 non-W3C members that wish to contribute. This is achieved by clearly 227 marking and explaining the areas that need testing, linked to existing 228 tests and general testing progress. 229 </p> 230 </section> 231 <section> 232 <h3>Providing guidance on testing</h3> 233 <p> 234 In order to increase the quality of the test contributions, this 235 document offers a set of guidelines for conducting testing (see 236 <a href="#approach" class="sectionRef"></a>) and a testing progress 237 tracker to increase the surface coverage of tests (see 238 <a href="#test-progress-tracking" class="sectionRef"></a>). 239 </p> 240 </section> 241 <section> 242 <h3>Creating automation-friendly tests</h3> 243 <p> 244 In terms of actual tests produced for the CSS Writing Modes, the main goal 245 is to ensure that most tests are automatable (i.e. they're either 246 reftests or use <code>testharness.js</code>). Even where manual tests 247 are absolutely necessary they should be written so that they can be 248 easily automated – as there are on-going efforts to make 249 WebDriver [[webdriver]] automated tests a first class citized in W3C 250 testing. This means that even if a manual test requires user 251 interaction, the validation or PASS/FAIL conditions should still be 252 clear enough as to allow automatic validation if said interaction is 253 later automated. 254 </p> 255 256 <p> 257 In particular need a special font in the test of CSS Writing Modes. Since these are is assumed of many tests, people who try to create the test now, check to see whether any available font. 258 </p> 259 </section> 260 </section> 261 <section> 262 <h2>Risks and mitigation</h2> 263 <p> 264 There are a number of risks associated with creating a high-quality 265 test suite for CSS Writing Modes. The most important ones are listed below. 266 </p> 267 <section> 268 <h3>Implicit interactions with a lot of other specs (and commonly accepted 269 browser behavior)</h3> 270 <p> 271 The CSS Writing Modes spec introduces a significant change in the way layout 272 can be done and as a consequence, many of the assumptions that hold in 273 the context of CSS 2.1 must be re-validated in the context of CSS 274 Writing Modes. In the meanwhile, a lot of new layout modules have been 275 proposed, with various degrees of implementer support and maturity. 276 Also, non-CSS specific specs introduced concepts that affect how style 277 is propagated (e.g. Shadow DOM [[shadow-dom]]) or how elements are 278 rendered and interact with their containing documents (e.g. 279 <code><iframe seamless></code> in HTML 5 [[HTML5]]). 280 </p> 281 <p> 282 All the above factors increase the testing surface and the number of 283 the possible cases that might need an explicitly specified behavior in 284 order to ensure intuitive and predictable results as well as stable 285 interaction with widely used browser features (that might not be <em>yet</em> 286 so strictly specified). 287 </p> 288 <p> 289 In terms of specifying the expected behavior, the current approach is 290 to try and specify it for specs that are final or nearly-final and to 291 just make a note of the possible interactions and unspecified behaviors 292 in the case of specs that are still in flux. In exceptional cases, a 293 new spec might be created to cater for the needs of multiple specs 294 (e.g. the CSS Fragmentation spec [[css3-break]]). 295 </p> 296 </section> 297 <section> 298 <h3>Big number of tests required</h3> 299 <div class="note"> 300 Add here the estimation of tests required produced by 301 <a href="https://github.com/web-platform-tests/wpt/tree/master/tools/coverage" target="_blank">W3C test coverage</a> 302 tool. If possible, provide a better informed guess on the number of tests. 303 </div> 304 <p> 305 Given the complexity of the spec, a big number of tests will need to 306 be created to produce a test suite that can ensure interoperability 307 between implementations. 308 </p> 309 <p> 310 In this context, the main purpose of this document is to provide 311 useful informations for creating and contributing tests in an effective 312 manner in terms of coverage and test quality. 313 </p> 314 </section> 315 <section> 316 <h3>Special fonts required</h3> 317 <p> 318 For building reftest, several special fonts are necessary. 319 If could not get some fonts, a test will be manual or reftest waiting fonts. 320 </p> 321 </section> 322 </section> 323 <section> 324 <h2>Approach</h2> 325 <p> 326 As spec testing cannot be realistically separated from testing a 327 particular implementation (except for the very simple cases), the 328 approach proposed for testing is one that tries to first cover as many 329 areas as possible, instead of deep diving on a certain feature or aspect 330 of the spec first. A side benefit of this approach is that the spec 331 tests can be used at any time to gauge the level of support of a certain 332 implementation. 333 </p> 334 <p> 335 Having this <em>breadth-first</em> approach in mind, tests will be 336 created for the testing areas listed in <a href="#testing-areas" 337 class="sectionRef"></a>. Testing will be done in multiple passes, each 338 aimed at covering more specific edge-cases. 339 </p> 340 <p> 341 The selection of test data, in terms of parsing, rendering, choose only one typical values from the data set on the same effect (It is a test technique known "equivalence partitioning"). 342 </p> 343 344 </section> 345 <section> 346 <h2>Testing areas</h2> 347 <section> 348 <h3>Explicit testing areas</h3> 349 <p> 350 These are testing areas normatively defined by the spec. They cover 351 things explicitly or implicitly defined in the CSS Writing Modes spec. 352 Please note that while detailed, this list is not necessarily 353 exhaustive and normative behaviors may not be contained in it. 354 When in doubt, consult the CSS Writing Modes spec or ask a question on the 355 <a href="http://lists.w3.org/Archives/Public/www-style/">mailing 356 list</a>. 357 </p> 358 <p> 359 <h4>Overview</h4> 360 Below is the list of explicit testing areas: 361 <ol> 362 <li> 363 Proper parsing of the CSS properties and rules, rendering 364 according to the spec. 365 <ul> 366 <li><code>direction</code></li> 367 <li><code>unicode-bidi</code></li> 368 <li><code>writing-mode</code></li> 369 <li><code>text-orientation</code></li> 370 <li><code>caption-side</code></li> 371 <li><code>text-combine-upright</code></li> 372 </ul> 373 </li> 374 Test the following set of elements as a group to the 'Applies to' description. 375 <ul> 376 <li>inline element, inline block, replaced elements, block element, list-item 377 , table, inline-table, table-header-group, table-footer-group, table-cell, table-caption</li> 378 <li><code>display:none</code></li> 379 <li>inherit</li> 380 </ul> 381 Selected as a representative following four frequently used as 'replace element'. 382 <ul> 383 <li><code>button</code></li> 384 <li><code>input:text</code></li> 385 <li><code>select</code></li> 386 <li><code>text-area</code></li> 387 </ul> 388 </li> 389 <li> 390 Box related specifications that are affected as specified by <code>writing-mode</code>, <code>text-orientation</code>. The original definition about 'Box model' see [[CSS21]] for details. Calculation of the following in particular: 391 <ul> 392 <li><code>margin</code></li> 393 <li><code>border</code></li> 394 <li><code>padding</code></li> 395 </ul> 396 </li> 397 <li> 398 Compression of the glyph in horizontal-in-vertical(tate-chu-yoko) when using the text-transform [[!css3-text]] and OpenType properties. 399 </li> 400 <li> 401 Positioning, sizing and Auto-sizing of the box in orthogonal flows. 402 </li> 403 <li> 404 Text Baselines in vertical writing mode. 405 <ul> 406 <li>alphabetic</li> 407 <li>central</li> 408 </ul> 409 </li> 410 </ol> 411 </p> 412 <h3>Each Section</h3> 413 <ol> 414 <h3>Sec 2.Inline Direction and Bidirectionality</h3> 415 <p> 416 Parse, Rendering fundamental when combined the <code>direction</code> property and the <code>unicode-bidi</code> property. Applying Unicode bidirectional algorithm, behavior when the value is a newline. Placement of split inline-box. Calculation positioning of the box element. 417 </p> 418 <h3>Sec 3.Introduction to Vertical Text</h3> 419 <p> 420 Parse, Rendering fundamental for the <code>writing-mode</code> property. Character should be considered in particular the placement in vertical writing mode(Punctuation, etc. Onbiki in Japanese). Rendering for principal writing mode. Handling of child block in the case of line feed direction different from the container block. Converte value of the SVG in the writing-mode. 421 </p> 422 423 <h3>Sec 4.Inline-level Alignment</h3> 424 <p> 425 Consider baseline alphabetic, central in this specification. Baseline in vertical alignment for glyph, processing of dominant baseline. 426 <div class="note"> 427 'dominant baseline' testing in CSS21 side? 428 </div> 429 </p> 430 <h3>Sec 5.Introduction to Vertical Text Layout</h3> 431 <p> 432 Parse, Rendering fundamental for the <code>text-orientation</code> property. rendering when the vertical writing mode. Rules of Appendix C. 433 </p> 434 <h3>Sec 6.Abstract Box Terminology</h3> 435 <p> 436 Processing over and under in line-relative directions. left, right adding in vertical writing mode to line-box. The processing of before, after, start, end of each block-level-element, inline-level-element in flow-relative directions. 437 </p> 438 <h3>Sec 7.Abstract Box Layout</h3> 439 <p> 440 Layout rule that refers to property in the box when in vertical writing mode and margin collapsing. Sizing, auto-sizing of the writing modes in Orthogonal Flow. Margin calculation in the case of Flow-Relative. Position calculation in the case of Line-Relative. Properties that do not affect the WritingMode. rendering and parsing of <code>caption-side</code> property. 441 </p> 442 <h3>Sec 8.Page Flow: the page progression direction</h3> 443 <p> 444 It is a page feed flow specification of UA, is beyond the scope of the current test environment. 445 </p> 446 <h3>Sec 9.Glyph Composition</h3> 447 <p> 448 Parse, Rendering fundamental for the <code>text-combine-upright</code> property. rendering when the vertical writing mode. Interrupted by a box boundary in text run-rules. Glyph of centering in 1em box. Several algorithms to compress the 1em glyph of multiple. 449 </p> 450 </ol> 451 452 </section> 453 <section> 454 <h3>Implicit testing areas</h3> 455 <p> 456 These are testing areas either normatively defined in other specs 457 that explicitly refer to the CSS Writing Modes spec (e.g. [[css3-text]]) 458 or simply not explicitly defined, but implied by various aspects of 459 the spec (e.g. processing model, CSS 2.1 compliance, etc.). 460 Please note that while detailed, this list is not necessarily 461 exhaustive and normative behaviors may not be contained in it. 462 When in doubt, consult the CSS Writing Modes spec or ask a question on the 463 <a href="http://lists.w3.org/Archives/Public/www-style/">mailing 464 list</a>. 465 </p> 466 <p> 467 Below is the list of implicit testing areas: 468 <ol> 469 <li> 470 CSS Writing Modes and layout modules: 471 <ul> 472 <li><code>overflow</code></li> 473 <li><code>clip</code></li> 474 <li><code>line-height</code></li> 475 <li>the <code>alt</code> attribute </li> 476 </ul> 477 </li> 478 </ol> 479 </p> 480 </section> 481 </section> 482 <section> 483 <h2>People and responsibilities</h2> 484 <p> 485 Below is a list of people you should reach out to if you have any 486 questions related to this document or testing CSS Writing Modes in general: 487 <ul> 488 <li>fantasai – editor for CSS Writing Modes spec</li> 489 <li>Koji Ishii – editor for CSS Writing Modes spec</li> 490 <li>Rebecca Hauck – CSSWG testing owner</li> 491 </ul> 492 </p> 493 </section> 494 <section> 495 <h2>Test progress tracking</h2> 496 <p> 497 Currently test progress tracking is done via gitHub 498 <a href="https://github.com/w3c/csswg-test/issues/milestones?with_issues=yes">milestones</a> 499 and <a href="https://github.com/w3c/csswg-test/issues?milestone=9&state=open">issues</a>. 500 </p> 501 </section> 502 </body> 503 </html>