[10970010] |
WordPerfect
[10970020] |'''WordPerfect''' is a [[proprietary software|proprietary]] [[word processing]] application. [10970030] |At the height of its popularity in the late 1980s and early 1990s, it was the ''[[de facto]]'' standard word processor, but has since been eclipsed in sales by [[Microsoft Office Word|Microsoft Word]]. [10970040] |Although the [[MS-DOS]] and [[Microsoft Windows]] versions are best known, its popularity was based on the fact that it had been available for a wide variety of computers and operating systems, including [[Mac OS]], [[Linux]], the [[Apple IIe]], a separate version for the [[Apple IIgs]], most popular versions of [[Unix]], [[OpenVMS|VMS]], [[Data General]], [[System/370]], [[AmigaOS]], [[Atari ST]], [[OS/2]], and [[Nextstep|NeXTSTEP]]. [10970050] |==WordPerfect for DOS== [10970060] |WordPerfect was originally produced by [[Bruce Bastian]] and Dr. [[Alan Ashton (executive)|Alan Ashton]] who founded Satellite Software International, Inc. of [[Orem]], [[Utah]], which later renamed itself WordPerfect Corporation. [10970070] |Originally written for [[Data General]] minicomputers, in 1982 the developers ported the program to the IBM PC as WordPerfect 2.20, continuing the version numbering of the Data General series. [10970080] |The program's popularity took off with the introduction of WordPerfect 4.2 in 1986, with automatic paragraph numbering (important to the law office market), and the splitting of a lengthy footnote and its partial overflow to the bottom of the next page, as if it had been professionally typeset (valuable to both the law office and academic markets). [10970090] |WordPerfect 4.2 became the first program to overtake the original microcomputer word processor market leader, [[WordStar]], in a major application category on the [[DOS]] platform. [10970100] |In 1989, WordPerfect Corporation released the program's most successful version ever, WordPerfect 5.1 for DOS, which was the first version to include [[Macintosh]] style pull-down menus to supplement the traditional F-key combinations, as well as support for tables, a spreadsheet-like feature. [10970110] |The data format used by WordPerfect 5.1 was, for years, the most portable format in the world. [10970120] |All word processors could read (and convert) that format. [10970130] |Many conferences and magazines insisted that you shipped your documents in 5.1 format. [10970140] |Unlike previous DOS versions, WordPerfect 6.0 for DOS could switch between its traditional text-based editing mode and a graphical editing mode that [[WYSIWYG|showed the document as it would print out]], including fonts and text effects like bold, underline, and italics. [10970150] |The previous text-based versions used different colors or text color inversions to indicate various markups, and (starting with version 5.0) used a graphic mode only for an uneditable print preview that used generic fonts rather than the actual fonts that appeared on the printed page. [10970160] |===Key characteristics=== [10970170] |To this day, WordPerfect's three major characteristics that have differentiated from other market-leading word processors are its streaming code architecture, its Reveal Codes feature, and its unusually user-friendly macro/scripting language, PerfectScript. [10970180] |====Streaming code architecture==== [10970190] |A key to WordPerfect's design is its streaming code architecture that parallels the formatting features of [[HTML]] and [[Cascading Style Sheets]]. [10970200] |Documents are created much the same way that raw HTML pages are written, with text interspersed by tags that trigger treatment of data until a corresponding closing tag is encountered, at which point the settings active to the point of the opening tag resume control. [10970210] |As with HTML, tags can be nested. [10970220] |Some data structures are treated as objects within the stream as with HTML's treatment of graphic images, e.g., footnotes and styles, but the bulk of a WordPerfect document's data and formatting codes appear as a single continuous stream. [10970230] |====Styles and style libraries==== [10970240] |The addition of styles and style libraries in WP 5.0 provided greatly increased power and flexibility in formatting documents, while maintaining the streaming-code architecture of earlier versions. [10970250] |Prior to that, WordPerfect's only use of styles (a particular type of programming object) is the Opening Style, which contains the default settings for a document. [10970260] |====Reveal codes==== [10970270] |The Reveal Codes feature is a second editing screen that can be toggled open and closed at the bottom of the main editing screen. [10970280] |Text is displayed in Reveal Codes interspersed with tags and the occasional objects, with the tags and objects represented by named tokens. [10970290] |The scheme makes it far easier to untangle coding messes than with styles-based word processors, and object tokens can be clicked with a pointing device to directly open the configuration editor for the particular object type, e.g. clicking on a style token brings up the style editor with the particular style type displayed. [10970300] |WordPerfect users forced to change word processors by employers frequently complain on WordPerfect online forums that they are lost without Reveal Codes. [10970310] |Because of their style dependencies, efforts to create the equivalent of Reveal Codes in other word processors have produced disappointing results. [10970320] |Note that WordPerfect had this feature already in its DOS incarnations: it could be brought forward by pressing the keys 'Alt' and 'F3' together. [10970330] |====Macro languages==== [10970340] |WordPerfect for DOS was notable for its Alt-keystroke macro facility, which was expanded with the addition of macro libraries in WP 5.0 that also allowed for Ctrl-keystroke macros, and remapping of any key as a macro. [10970350] |This enabled any sequence of keystrokes to be recorded, saved, edited, and recalled. [10970360] |Macros could examine system data, make decisions, be chained together, and operate recursively until a defined 'stop' condition was met. [10970370] |This capability provided an amazingly powerful way to rearrange data and formatting codes within a document, where the same sequence of actions needed to be performed repetitively e.g. for tabular data. [10970380] |Macros can also be edited using WordPerfect Program Editor. [10970390] |Unfortunately, this facility could not easily be ported to the subsequent Windows versions. [10970400] |A new and even more powerful interpreted token-based macro recording and scripting language was introduced for both DOS and Windows 6.0 versions, and that became the basis of the language named PerfectScript in later versions. [10970410] |PerfectScript has remained the mainstay scripting language for WordPerfect users ever since. [10970420] |PerfectScript was specifically designed to be user-friendly, thus avoiding far less user-friendly methods of scripting languages implemented on other word processing programs that require education in advanced programming concepts such as Object Oriented Programming in order to produce useful yet sophisticated and powerful macros. [10970430] |===Function keys=== [10970440] |Like its mid-1980s competitor, [[MultiMate]], WordPerfect used almost every possible combination of [[function key]]s with Ctrl, Alt, and Shift modifiers. [10970450] |([[WordPerfect 4.1|See example help screen on this page]].) [10970460] |This was in contrast to [[WordStar]], which used only Ctrl, in conjunction with traditional typing keys. [10970470] |Many people still know and use the [[function key]] combinations from the DOS version, which were originally designed for Data General Dasher VDUs that supported 2 groups of 5 plain, shift, control, and control shift function keys. [10970480] |This was translated to the layout of the 1981 [[IBM PC keyboard]], with two columns of function keys at the left end of the keyboard, but worked even better with the 1984 PC AT keyboard with 3 groups of 4 function keys across the top of the keyboard. [10970490] |With the 1981 PC keyboard, the Tab key and the related F4 (''Indent'') functions were adjacent. [10970500] |This plethora of keystroke possibilities, combined with the developers' wish to keep the user interface free of "clutter" such as on-screen menus, made it necessary for most users to use a keyboard template showing each function. [10970510] |Infamously, WordPerfect used F3 instead of F1 for ''Help'', F1 instead of Esc for ''Cancel'', and Esc for ''Repeat'' (though a configuration option in later versions allowed these functions to be rotated to locations that later became more standard). [10970520] |===Printer drivers=== [10970530] |WordPerfect for DOS shipped with an impressive array of printer drivers - a feature that played an important role in its adoption - and also shipped with a [[printer driver]] editor called PTR, which features a flexible [[Macro (computer science)|macro language]] and allows technically-inclined users to customize and create printer drivers. [10970540] |Internally, WordPerfect used an extensive WordPerfect [[character set]] as its [[internal code]]. [10970550] |The precise meaning of the characters, although clearly defined and documented, can be overridden in its customizable printer drivers with PTR. [10970560] |The relationship between different type faces and styles, and between them and the various sections in the WordPerfect character set, were also described in the printer drivers and can be customized through PTR. [10970570] |===WordPerfect Library/Office=== [10970580] |WordPerfect Corporation produced a variety of ancillary and spin-off products. [10970590] |WordPerfect Library (introduced in 1986 and later renamed WordPerfect Office) was a package of network and stand-alone utilities for use with WordPerfect, primarily developed for offices running [[Novell NetWare]]. [10970600] |WordPerfect Library/Office included the DOS antecedents of what is now known as [[Novell GroupWise]], a shareable package of contact management, calendaring, and related word processing utilities. [10970610] |WordPerfect Library/Office a brand name later revived by Corel after it acquired ownership of WordPerfect and other programs still bundled under that product name as of this writing – included amongst other utilities a local area network (LAN) email facility and was the most popular such package in its day. [10970620] |====WordPerfect Shell==== [10970630] |The Library/Office bundle also included a noteworthy task-switching program that ran as a shell atop DOS, branded as WordPerfect Shell. [10970640] |Task-switchers were a popular application type for the DOS operating system because of its lack of multi-tasking, making it impractical to have many applications running at once. [10970650] |Task-switchers were programs that allocated available memory between open applications, allowing fast switching between open applications whose actions were suspended when the user switched to a different program. [10970660] |WordPerfect Shell 4.0, which was also bundled with the WordPerfect 6.x versions, had most functionality of the Windows 3.x shell but was far more versatile. [10970670] |Its automated memory management was superior to that of the Microsoft Windows shell, and Microsoft's product generally performed with far less frequent memory glitches when Windows was run as a program under Shell 4.0. [10970680] |The user interface for Shell is based on a hierarchical menu metaphor rather than the windows/folders/icons metaphor used by Microsoft. [10970690] |Shell 4.0's menu structures could be individually hot-keyed as pop-ups and its powerful menu editor allowed fast creation and editing of menu structures and menu items, with each menu item quickly configurable for entry of command lines and menu names. [10970700] |Shell 4.0 included 80 programmable clipboards, and the menu structures and menu items were also programmable using a scripting language whose scripts could themselves be chained to and from WordPerfect macros. [10970710] |The scripting language also included a keyboard buffer stuffing tool for control and operation of non-WordPerfect applications. [10970720] |Microsoft Windows had no answer to such powerful features other than a glitz of windows, icons, pointing devices, and an overwhelming marketing strategy. [10970730] |WordPerfect Shell was laid to rest along with many other popular DOS character-based tools inundated by Microsoft's marketing of Windows 95. [10970740] |Novell later licensed Shell 3.0 and 4.0 for free distribution. [10970750] |As of this writing it is still downloadable from the DataPerfect Users Group. [10970760] |WordPerfect Library/Office also included a Calculator, a flat-file database called Notebook that could be used by itself or in WordPerfect document merges, an exceptionally powerful relational database - [[DataPerfect]] - that retains a small but dedicated following despite having been dropped by WordPerfect Corporation in favour of Borland's Paradox as a companion of WP for Windows. [10970770] |Additional features continue to be added from time to time by DataPerfect's author, Lew Bastian - Bruce Bastian's older brother - a brilliant programmer who had written some of IBM's earliest disk-caching patents, and DataPerfect can now run as web server. [10970780] |LetterPerfect was a scaled down version of WordPerfect with the more advanced features removed but with file and (for the most part) keystroke compatibility. [10970790] |An implementation of Microsoft Visual Basic for Applications (VBA), introduced with WordPerfect for Windows 9.0, provides a full-featured development environment for building advanced custom WordPerfect solutions. [10970800] |These solutions are often created by corporate developers or programmers and may not be easily accessible to the typical WordPerfect user. [10970810] |For these users, PerfectScript is the better option. [10970820] |People who code scripts for WordPerfect use the Macros & Merges forum at WordPerfect Universe as their primary meeting ground. [10970830] |That site is a collaboration among other WordPerfect-related web site operators and others and functions as a portal to WordPerfect resources on the web. [10970840] |The site also maintains an extensive clip library for use in PerfectScript programming, has the Web's largest metalink library for locating online WordPerfect resources, and has the only peer-to-peer forum on the Web for DOS WordPerfect. [10970850] |The WordPerfect template and document file formats have remained remarkably stable since the WordPerfect 6.x DOS and Windows versions. [10970860] |Complete backward compatibility has been maintained and all WordPerfect versions since 6.0 have included a feature that stores any unrecognized codes in stream location represented in Reveal Codes by an "Unknown" token. [10970870] |Documents generated on newer versions can thus be edited in older versions with the codes retained. [10970880] |Then, upon being reopened in a newer version of WordPerfect, the "unknown" tokens regain their functionality. [10970890] |None of the newer WordPerfect features reflected in the file formats cause data loss when opened in older versions. [10970900] |==WordPerfect for Windows== [10970910] |===History=== [10970920] |WordPerfect was late in coming to market with a Windows version. [10970930] |The first mature version, WordPerfect 5.2 for Windows, was released in November [[1992]]. [10970940] |Prior to that, there was a WordPerfect 5.1 for Windows, introduced a year earlier. [10970950] |That version had to be installed from DOS and was largely unpopular due to serious stability issues. [10970960] |By the time WordPerfect 5.2 for Windows was introduced, [[Microsoft Word|Microsoft Word for Windows]] version 2 had been on the market for over a year and had received its third interim release, v2.0c. WordPerfect's function-key-centered user interface did not adapt well to the new paradigm of mouse and pull-down menus, especially with many of WordPerfect's standard key combinations pre-empted by incompatible keyboard shortcuts that Windows itself used (e.g. Alt-F4 became ''Exit Program'' as opposed to WordPerfect's ''Block Text''). [10970970] |The DOS version's impressive arsenal of finely tuned printer drivers was also rendered obsolete by Windows' use of its own printer device drivers. [10970980] |Internally, WordPerfect for Windows still used the WordPerfect character set as its internal code. [10970990] |This caused WordPerfect for Windows to be unable to support some languages — for example [[Chinese language|Chinese]] — that were natively supported by Windows. [10971000] |WordPerfect became part of an [[office suite]] when the company entered into a co-licensing agreement with [[Borland|Borland Software Corporation]] in 1993. [10971010] |The offerings were marketed as Borland Office, containing Windows versions of WordPerfect, [[Quattro Pro]], [[Borland Paradox]], and a LAN-based groupware package called WordPerfect Office (not to be confused with the complete applications suite of the same name later marketed by Corel) based on the WordPerfect Library for DOS. [10971020] |The WordPerfect product line was sold twice, first to [[Novell]] in June [[1994]], who then sold it to [[Corel]] in January [[1996]]. [10971030] |However, Novell kept the WordPerfect Office technology, incorporating it into its [[GroupWise]] messaging and collaboration product. [10971040] |Compounding WordPerfect's troubles were issues associated with the release of the first [[32-bit application|32-bit]] version, WordPerfect 7, intended for use on [[Windows 95]]. [10971050] |While it contained notable improvements over the [[16-bit application|16-bit]] WordPerfect for Windows 6.1, it was released in May [[1996]], nine months after the introduction of Windows 95 and Microsoft Office 95 (including [[Word for Windows|Word 95]]). [10971060] |The initial release suffered from notable stability problems. [10971070] |WordPerfect 7 also didn't have a Microsoft "Designed for Windows 95" logo. [10971080] |This was important to Windows 95 software purchasers as Microsoft set standards for application design, behavior, and interaction with the operating system. [10971090] |To make matters worse, the original release of WordPerfect 7 was incompatible with [[Windows NT]], hindering its adoption in academia. [10971100] |The "NT Enabled" version of WordPerfect 7, which Corel considered to be Service Pack 2, wasn't available until Q1-[[1997]], over 6 months after the introduction of [[Windows NT 4.0]], a year and a half after the introduction of Office 95 (which supported Windows NT out of the box), and shortly after the introduction of Office 97. [10971110] |Corel charged its customers to receive, what amounted to, a bug fix. [10971120] |While WordPerfect retained a majority of the retail shelf sales of word processors, Microsoft gained marketshare by including [[Microsoft Word|Word for Windows]] in its Windows product on new PCs. [10971130] |Microsoft gave discounts for Windows to OEMs who included Word on their PCs. [10971140] |When new PC buyers found Word installed on their new PC, Word began to dominate marketshare of desktop word processing. [10971150] |Amongst the remaining avid users of WordPerfect are many law firms and academics who favor the WordPerfect features such as macros and reveal codes. [10971160] |Corel now caters to these markets, with, for example, a major sale to the [[United States Department of Justice]] in 2005 . [10971170] |In November 2004, Novell filed an antitrust lawsuit against Microsoft for alleged anticompetitive behavior (viz, tying Word to sales of Windows) that Novell claims led to loss of WordPerfect market share . [10971180] |===Corel WordPerfect=== [10971190] |Since its acquisition by [[Corel]], WordPerfect for Windows has officially been known as '''Corel WordPerfect'''. [10971200] |== Unicode and Asian language editing == [10971210] |WordPerfect also lacks support for [[Unicode]]. [10971220] |The absence of support for [[Unicode]] limits its usefulness in many markets outside North America and Western Europe. [10971230] |Despite pleas from longtime users, this feature has not been implemented as of yet. [10971240] |For users in WordPerfect's traditional markets, the inability to deal with complex character sets, such as Asian language scripts, can cause difficulty when working on documents containing those characters. [10971250] |However, later versions have provided better compliance with interface conventions, file compatibility, and even Word interface emulation. [10971260] |== "Classic Mode" == [10971270] |Corel added "Classic Mode" in WordPerfect 11. [10971280] |==WordPerfect for Macintosh== [10971290] |Development of WordPerfect for Macintosh did not run parallel to versions for other operating systems, and used version numbers unconnected to contemporary releases for DOS, Windows, etc. [10971300] |The first release reminded users and reviewers of the DOS version, and was not especially successful in the marketplace. [10971310] |Version 2 was a total re-write, adhering more closely to Apple's UI guidelines. [10971320] |Version 3 took this further, making extensive use of the technologies Apple introduced in Systems 7.0–7.5, while remaining fast and capable of running well on older machines. [10971330] |Corel released version 3.5 in 1996, followed by the improved version 3.5e. [10971340] |It was never updated beyond that, and the product was eventually discontinued. [10971350] |[[As of 2004]], Corel has reiterated that the company has no plans to further develop WordPerfect for Macintosh (such as creating a native Mac OS X version). [10971360] |For several years, Corel allowed Mac users to download version 3.5e from their website free of charge, and some Mac users still use this version. [10971370] |The download is still available, along with the necessary OS 8/9/Classic Updater that slows scroll speed and restores functionality to the Style and Window menus. [10971380] |Like other Mac OS applications of its age, it requires the Classic environment on [[PowerPC]] Macs. [10971390] |While Intel Macs do not support Classic, emulators such as [[SheepShaver]], and [[vMac]] allow users to run WordPerfect and other Mac OS applications. [10971400] |Users wishing to use an up to date version of WordPerfect can run the Windows version through [[Boot Camp (software)|Boot Camp]] or a Windows emulator, and through [[Darwine]] or [[CrossOver Mac]] with mixed results. [10971410] |==WordPerfect for Linux== [10971420] |In 1995, WordPerfect 6.0 was made available for [[Linux]] as part of [[SCO Group|Caldera]]'s internet office package. [10971430] |In late 1997, a newer version was made available for download, but had to be purchased to be activated. [10971440] |Hoping to establish themselves in the nascent commercial Linux market, Corel also developed their [[Corel Linux|own distribution]] of Linux. [10971450] |Although the Linux distribution was fairly well-received, the response to WordPerfect for Linux was varied. [10971460] |Some Linux promoters appreciated the availability of a well-known, mainstream application for the OS. Developers of other Linux-compatible word processors questioned the need for another application in the category. [10971470] |Advocates of [[open-source software]] scoffed at its proprietary, closed-source nature, and questioned the viability of a commercial application in a market dominated by free software, such as [[OpenOffice.org]] and numerous others. [10971480] |The performance and stability of WordPerfect 9.0 (not a native Linux application like WP 6-8, but derived from the Windows version using the [[Wine (software)|Wine]] compatibility library) was highly criticized. [10971490] |WordPerfect failed to gain a large user base, and as part of Corel's change of strategic direction following a (non-voting) investment by Microsoft, WordPerfect for Linux was discontinued and their Linux distribution was sold to [[Xandros]]. [10971500] |In April 2004, Corel re-released WordPerfect 8.1 (the last Linux-native version) with some updates, as a "proof of concept" and to test the Linux market. [10971510] |[[As of 2005]], WordPerfect for Linux is not available for purchase. [10971520] |==Versions== [10971530] |(* - Part of [[WordPerfect Office]]) [10971540] |Known versions for VAX/VMS include 5.1, 5.3 and 7.1 , year of release unknown. [10971550] |Known versions for SUN include 6.0, requiring SunOS or Solaris 2, year of release unknown. [10971560] |Known versions for IBM System/370 include 4.2, released 1988. [10971570] |Known versions for OS/2 include 5.0, released 1989. [10971580] |Known versions for the DEC Rainbow 100 include version (?), released November 1983. [10971590] |In addition, versions of WordPerfect have also been available for Apricot, Atari ST, DEC Rainbow, Tandy 2000, TI Professional, Victor 9000, and Zenith Z-100 systems, as well as around 30 flavors of unix, including AT&T, NCR, SCO Xenix, Microport Unix, DEC Ultrix, Pyramid Tech Unix, Tru64, AIX, Motorola 8000, and HP9000 and SUN 3. [10971600] |==Current versions== [10971610] |On [[January 17]], [[2006]], Corel announced WordPerfect X3, the newest version of this office package. [10971620] |Corel is an original member of the [[OASIS (organization)|OASIS]] Technical Committee on the [[OpenDocument|Open Document Format]], and Paul Langille, a senior Corel developer, is one of the original four authors of the OpenDocument specification. [10971630] |In January 2006, subscribers to Corel's electronic newsletter were informed that WordPerfect 13 was scheduled for release later in 2006. [10971640] |The subsequent release of X3 (identified as "13" internally and in registry entries) has been met with generally positive reviews, due to new features including a unique PDF import capability, metadata removal tools, integrated search and online resources and other features. [10971650] |Version X3 was described by [[CNET]] in January, 2006 as a "winner", "a feature-packed productivity suite that's just as easy to use – and in many ways more innovative than – industry-goliath Microsoft Office 2003." [10971660] |CNET went on to describe X3 as "a solid upgrade for longtime users", but that "Die-hard Microsoft fans may want to wait to see what Redmond has up its sleeve with the radical changes expected within the upcoming Microsoft Office 12." [10971670] |While the notable if incremental enhancements of WordPerfect Office X3 have been well received by reviewers, a number of online forums have voiced concern about the future direction of WordPerfect, with longtime users complaining about certain usability and functionality issues that users have been asking to have fixed for the last few release versions. [10971680] |Although the released version of X3 does not support the [[Office Open XML|OOXML]] or [[OpenDocument]] formats, a beta has been released that supports both. [10971690] |Reports surfaced late in January 2006 that Apple's [[iWork]] had leapfrogged WordPerfect Office as the leading alternative to Microsoft Office. [10971700] |This claim was soon debunked after industry analyst Joe Wilcox described JupiterResearch usage surveys that showed WordPerfect as the No. 2 office suite behind Microsoft Office in the consumer, small and medium businesses, and enterprise markets with a roughly 15 percent share in each market. [10971710] |In April [[2008]] Corel released their WordPerfect Office X4 [[office suite]] containing the new X4 version of WordPerfect which includes support for [[PDF]], [[OpenDocument]] and [[Office Open XML]]. [10980010] |
Word sense disambiguation
[10980020] |In [[computational linguistics]], '''word sense disambiguation''' (WSD) is the process of identifying which [[word sense|sense]] of a [[word]] (having a number of distinct senses) is used in a given [[Sentence (linguistics)|sentence]]. [10980030] |For example, consider the word ''bass'', two distinct senses of which are: [10980040] |#a type of fish [10980050] |#tones of low frequency [10980060] |and the sentences: [10980070] |#''I went fishing for some sea bass'' [10980080] |#''The bass line of the song is very moving'' [10980090] |Explanation [10980100] |To a human it is obvious that the first sentence is using the word ''bass'' in the first sense above, and that in the second sentence it is being used in the second sense. [10980110] |Although this seems obvious to a human, developing [[algorithm]]s to replicate this human ability is a difficult task. [10980120] |==Difficulties== [10980130] |One problem with word sense disambiguation is deciding what the senses are. [10980140] |In cases like the word ''bass'' above, at least some senses are obviously different. [10980150] |In other cases, however, the different senses can be closely related (one meaning being a [[metaphor]]ical or [[metonymy|metonymic]] extension of another), and in such cases division of words into senses becomes much more difficult. [10980160] |Different dictionaries will provide different divisions of words into senses. [10980170] |One solution some researchers have used is to choose a particular dictionary, and just use its set of senses. [10980180] |Generally, however, research results using broad distinctions in senses have been much better than those using narrow, so most researchers ignore the fine-grained distinctions in their work. [10980190] |Another problem is inter-judge [[variance]]. [10980200] |WSD systems are normally tested by having their results on a task compared against those of a human. [10980210] |However, humans do not agree on the task at hand — give a list of senses and sentences, and humans will not always agree on which word belongs in which sense. [10980220] |A computer cannot be expected to give better performance on such a task than a human (indeed, since the human serves as the standard, the computer being better than the human is incoherent), so the human performance serves as an upper bound. [10980230] |Human performance, however, is much better on coarse-grained than fine-grained distinctions, so this again is why research on coarse-grained distinctions is most useful. [10980240] |==Approaches== [10980250] |As in all [[natural language processing]], there are two main approaches to WSD — deep approaches and shallow approaches. [10980260] |Deep approaches presume access to a comprehensive body of [[commonsense knowledge|world knowledge]]. [10980270] |Knowledge such as "you can go fishing for a type of fish, but not for low frequency sounds" and "songs have low frequency sounds as parts, but not types of fish" is then used to determine in which sense the word is used. [10980280] |These approaches are not very successful in practice, mainly because such a body of knowledge does not exist in computer-readable format outside of very limited domains. [10980290] |But if such knowledge did exist, they would be much more accurate than the shallow approaches. [10980300] |However, there is a long tradition in Computational Linguistics of trying such approaches in terms of coded knowledge, and in some cases it is hard to say clearly whether the knowledge involved is linguistic or world knowledge. [10980310] |The first attempt was that by Margaret Masterman and her colleagues at Cambridge Language Research Unit in England in the 1950s. [10980320] |This used as data a punched-card version of Roget's Thesaurus and its numbered "heads" as indicators of topics and looked for their repetitions in text, using a set intersection algorithm: it was not very successful (and is described in some detail in (Wilks, Y. et al., 1996) but had strong relationships to later work, especially Yarowsky's machine learning optimisation of a thesaurus method in the 1990s (see below). [10980330] |Shallow approaches don't try to understand the text. [10980340] |They just consider the surrounding words, using information like "if ''bass'' has words ''sea'' or ''fishing'' nearby, it probably is in the fish sense; if ''bass'' has the words ''music'' or ''song'' nearby, it is probably in the music sense." [10980350] |These rules can be automatically derived by the computer, using a training corpus of words tagged with their word senses. [10980360] |This approach, while theoretically not as powerful as deep approaches, gives superior results in practice, due to computers' limited world knowledge. [10980370] |It can, though, be confused by sentences like ''The dogs bark at the tree'', which contains the word ''bark'' near both ''tree'' and ''dogs''. [10980380] |These approaches normally work by defining a window of ''N'' content words around each word to be disambiguated in the corpus, and statistically analyzing those ''N'' surrounding words. [10980390] |Two shallow approaches used to train and then disambiguate are ''[[Naïve Bayes classifier]]s'' and ''[[decision tree]]s''. [10980400] |In recent research, kernel based methods such as [[support vector machine]]s have shown superior performance in [[supervised learning]]. [10980410] |But over the last few years, there hasn't been any major improvement in performance of any of these methods. [10980420] |It is instructive to compare the word sense disambiguation problem with the problem of [[part-of-speech tagging]]. [10980430] |Both involve disambiguating or tagging with words, be it with senses or parts of speech. [10980440] |However, algorithms used for one do not tend to work well for the other, mainly because the part of speech of a word is primarily determined by the immediately adjacent one to three words, whereas the sense of a word may be determined by words further away. [10980450] |The success rate for part-of-speech tagging algorithms is at present much higher than that for WSD, state-of-the art being around 95% accuracy or better, as compared to less than 75% accuracy in word sense disambiguation with supervised learning. [10980460] |These figures are typical for English, and may be very different from those for other languages. [10980470] |Another aspect of word sense disambiguation that differentiates it from part-of-speech tagging is the availability of training data. [10980480] |While it is relatively easy to assign parts of speech to text, training people to tag senses is far more difficult . [10980490] |While users can memorize all of the possible parts of speech a word can take, it is impossible for individuals to memorize all of the senses a word can take. [10980500] |Thus, many word sense disambiguation algorithms use [[semi-supervised learning]], which allows both labeled and unlabeled data. [10980510] |The [[Yarowsky algorithm]] was an early example of such an algorithm. [10980520] |Yarowsky’s [[Unsupervised learning|unsupervised algorithm]] uses the ‘One sense per collocation’ and the ‘One sense per discourse’ properties of human languages for word sense disambiguation. [10980530] |From observation, words tend to exhibit only one sense in most given discourse and in a given collocation. [10980540] |The corpus is initially untagged. [10980550] |The algorithm starts with a large corpus, in which it identifies examples of the given polysemous word, and stores all the relevant sentences as lines. [10980560] |For instance, Yarowsky uses the word ‘plant’ in his 1995 paper to demonstrate the algorithm. [10980570] |Assume that there are two possible senses of the word, the next step is to identify a small number of seed collocations representative of each sense, give each sense a label, i.e. sense A and B, then assign the appropriate label to all training examples containing the seed collocations. [10980580] |In this case, the words ‘life’ and ‘manufacturing’ are chosen as initial seed collocations for sense A and B respectively. [10980590] |The residual examples (85% - 98% according to Yarowsky) remain untagged. [10980600] |The algorithm should initially choose seed collocations representative that will distinguish sense A and B accurately and productively. [10980610] |This can be done by selecting seed words from a dictionary’s entry for that sense. [10980620] |The collocations tend to have stronger effect if they are adjacent to the target word, the effect weakens with distance. [10980630] |According to the criteria given in Yarowsky (1993), seed words that appear in the most reliable collocational relationships with the target word will be selected. [10980640] |The effect is much stronger for words in a predicate-argument relationship than for arbitrary associations at the same distance to the target word, and is much stronger for collocations with content words than with function words. [10980650] |Having said this, a collocation word can have several collocational relationships with the target word throughout the corpus. [10980660] |This could give the word different rankings or even different classifications. [10980670] |Alternatively, it can be done by identifying a single defining collocate for each class, and using for seeds only those contexts containing one of these defining words. [10980680] |A publicly available database called WordNet can be used as an automatic source for such defining terms. [10980690] |In addition, words that occur near the target word in great frequency can be selected as seed collocations representative. [10980700] |This approach is not fully automatic, a human judge must decide which word will be selected for each target word’s sense, the outputs will be reliable indicators of the senses. [10980710] |A decision-list algorithm is then used to identify other reliable collocations. [10980720] |This training algorithm calculates the probability P(Sense | Collocation), and the decision list is ranked by the log-likelihood ratio: '''Log( P(SenseA | Collocationi) / P(SenseB | Collocationi) )''' [10980730] |A [[smoothing]] algorithm will then be used to avoid 0 values. [10980740] |The decision-list algorithm resolves many problems in a large set of non-independent evidence source by using only the most reliable piece of evidence rather than the whole matching collocation set. [10980750] |The new resulting classifier will then be applied to the whole sample set. [10980760] |Add those examples in the residual that are tagged as A or B with probability above a reasonable threshold to the seed sets. [10980770] |Apply the decision-list algorithm and the above adding step iteratively. [10980780] |As more newly-learned collocations are added to the seed sets, the sense A or sense B set will grow, and the original residual will shrink. [10980790] |However, these collocations stay in the seed sets only if their probability of classification remains above the threshold, otherwise they are returned to the residual for later classification. [10980800] |At the end of each iteration, the ‘One sense per discourse’ property can be used to help preventing initially mistagged collocates and hence improving the purity of the seed sets. [10980810] |In order to avoid strong collocates becoming indicators for the wrong class, the class-inclusion threshold needs to be randomly altered. [10980820] |For the same purpose, after intermediate convergence the algorithm will also need to increase the width of the context window. [10980830] |The algorithm will continue to iterate until no more reliable collocations are found. [10980840] |The ‘One sense per discourse’ property can be used here for error correction. [10980850] |For a target word that has a binary sense partition, if the occurrences of the majority sense A exceed that of the minor sense B by a certain threshold, the minority ones will be relabeled as A. According to Yarowsky, for any sense to be clearly dominant, the occurrences of the target word should not be less than 4. [10980860] |When the algorithm converges on a stable residual set, a final decision list of the target word is obtained. [10980870] |The most reliable collocations are at the top of the new list instead of the original seed words. [10980880] |The original untagged corpus is then tagged with sense labels and probabilities. [10980890] |The final decision list may now be applied to new data, the collocation with the highest rank in the list is used to classify the new data. [10980900] |For example, if the highest ranking collocation of the target word in the new data set is of sense A, then the target word is classified as sense A. [10990010] |
XHTML
[10990020] |The '''''Extensible [[Hypertext]] Markup Language''''', or '''XHTML''', is a [[markup language]] that has the same depth of expression as [[HTML]], but also conforms to [[XML]] syntax. [10990030] |While HTML is an application of [[Standard Generalized Markup Language]] (SGML), a very flexible markup language, XHTML is an application of [[XML]], a more restrictive subset of SGML. [10990040] |Because they need to be [[XML#Well-formed_documents|well-formed]], true XHTML documents allow for automated processing to be performed using standard XML tools—unlike HTML, which requires a relatively complex, lenient, and generally custom [[parsing|parser]]. [10990050] |XHTML can be thought of as the intersection of HTML and XML in many respects, since it is a reformulation of HTML in XML. [10990060] |XHTML 1.0 became a [[World Wide Web Consortium]] (W3C) [[W3C recommendation|Recommendation]] on [[January 26]], [[2000]]. [10990070] |XHTML 1.1 became a W3C Recommendation on [[May 31]], [[2001]]. [10990080] |== Overview == [10990090] |XHTML is "a reformulation of the three HTML 4 document types as applications of XML 1.0". [10990100] |The [[W3C]] also continues to maintain the HTML 4.01 Recommendation and the specifications for [[HTML5]] and XHTML5 are being actively developed. [10990110] |In the current XHTML 1.0 Recommendation document, as published and revised to August 2002, the W3C comments that, "The XHTML family is the next step in the evolution of the Internet. [10990120] |By migrating to XHTML today, content developers can enter the XML world with all of its attendant benefits, while still remaining confident in their content's backward and future compatibility." [10990130] |=== Motivation === [10990140] |The need for a reformulated version of HTML was felt primarily because [[World Wide Web]] content now needs to be delivered to many devices (like [[Mobile computing|mobile device]]s) apart from traditional desktop [[computer]]s, where extra resources cannot be devoted to support the additional complexity of HTML syntax. [10990150] |In practice, however, HTML-supporting browsers for such constrained devices have emerged faster than XHTML support has been added to the desktop browser with the largest market share, [[Internet Explorer]]. [10990160] |Another goal for XHTML and XML was to reduce the demands on parsers and [[user agent]]s in general. [10990170] |With HTML, user agents increasingly took on the burden of "correcting" errant documents. [10990180] |Instead, XML requires user agents to give a "fatal" error when encountering malformed XML. [10990190] |In theory, this allows for vendors to produce leaner browsers, without the obligation to work around author errors. [10990200] |A side effect of this behavior is that those authoring XHTML documents and testing in conformant browsers should be more readily alerted to errors that may have gone otherwise unnoticed if the browser had attempted to render or ignore the malformed markup. [10990210] |A feature XHTML inherits from its XML underpinnings is XML [[Namespace (computer science)|namespaces]]. [10990220] |With namespaces, authors or communities of authors can define their own XML elements, attributes and content models to mix within XHTML documents. [10990230] |This is similar to the semantic flexibility of the class attribute in an [[HTML element]], but with fewer restrictions. [10990240] |Some W3C XML namespaces/schema that can be mixed with XHTML include [[MathML]] for semantic math markup, [[Scalable Vector Graphics]] for markup of vector graphics, and [[RDFa]] for embedding [[Resource Description Framework|RDF]] data. [10990250] |=== Relationship to HTML === [10990260] |HTML is the [[antecedent]] technology to XHTML. [10990270] |The changes from HTML to first-generation XHTML 1.0 are minor and are mainly to achieve conformance with XML. [10990280] |The most important change is the requirement that the document must be [[well-formed element|well-formed]] and that all [[HTML element|elements]] must be explicitly closed as required in XML. [10990290] |In XML, all element and attribute names are [[case-sensitive]], so the XHTML approach has been to define all tag names to be lowercase. [10990300] |This contrasts with some earlier established traditions which began around the time of HTML 2.0, when many used uppercase tags. [10990310] |In XHTML, all attribute values must be enclosed by quotes; either single (') or double (") quotes may be used. [10990320] |In contrast, this was sometimes optional in SGML-based HTML, where numeric or boolean attributes can omit quotes. [10990330] |All elements must also be explicitly closed, including empty (aka [[singleton]]) elements such as img and br. [10990340] |This can be done by adding a closing slash to the start tag, ''e.g.'', <img /> and <br />. [10990350] |Attribute minimization (e.g., <option selected>) is also prohibited, as the attribute selected contains no explicit value; instead this would be written as <option selected="selected">. [10990360] |HTML elements which are optional in the content model will not appear in the [[Document Object Model|DOM]] tree unless they are explicitly specified. [10990370] |For example, an XHTML page ''must'' have a <body> element, and a table will not have a <tbody> element unless the author specifies one. [10990380] |The XHTML 1.0 recommendation devotes a section to differences between HTML and XHTML.. [10990390] |The WHATWG wiki similarly considers differences that arise with the use of (X)HTML5.. [10990400] |Because XHTML and HTML are closely related technologies, sometimes they are written about and documented in parallel. [10990410] |In such circumstances, some authors conflate the two names by using a parenthetical notation, such as (X)HTML. [10990420] |This indicates that the documentation and principles can be considered to apply generally to both standards. [10990430] |=== Adoption === [10990440] |The similarities between HTML 4.01 and XHTML 1.0 led many web sites and content management systems to adopt the initial W3C XHTML 1.0 Recommendation. [10990450] |To aid authors in the transition, the W3C provided guidance on how to publish XHTML 1.0 documents in an HTML-compatible manner, and serve them to browsers that were not designed for XHTML. [10990460] |Such "HTML-compatible" content is sent using the HTML media type (text/html) rather than the official Internet media type for XHTML (application/xhtml+xml). [10990470] |When measuring the adoption of XHTML to that of regular HTML, therefore, it is important to distinguish whether it is media type usage or actual document contents that is being compared. [10990480] |Most web browsers have mature support for all of the possible XHTML media types. [10990490] |The notable exception is [[Internet Explorer]] by [[Microsoft]]; rather than rendering application/xhtml+xml content, a dialog box invites the user to save the content to disk instead. [10990500] |Both Internet Explorer 7 (released in 2006) and the initial beta version of Internet Explorer 8 (released in March 2008) exhibit this behaviour, and it is unclear whether this will be resolved in a future release. [10990510] |Whilst this remains the case, most web developers avoid using XHTML that isn’t HTML-compatible, so advantages of XML such as namespaces, faster parsing and smaller-footprint browsers do not benefit the user. [10990520] |Microsoft developer Chris Wilson explained in 2005 that IE7’s priorities were improved security and [[Cascading Style Sheet|CSS]] support, and that proper XHTML support would be difficult to graft onto IE’s compatibility-oriented HTML parser. [10990530] |Recently, notable developers have begun to question why Web authors ever made the leap into authoring in XHTML. [10990540] |In October 2006, HTML inventor and W3C chair [[Tim Berners-Lee]], explaining the motivation for the resumption of HTML (not XHTML) development, posted in his blog: "The attempt to get the world to switch to XML, including quotes around attribute values and slashes in empty tags and namespaces all at once didn't work. [10990550] |The large HTML-generating public did not move, largely because the browsers didn't complain." [10990560] |== Versions of XHTML == [10990570] |=== XHTML 1.0 === [10990580] |December 1998 saw the publication of a W3C Working Draft entitled ''Reformulating HTML in XML''. [10990590] |This introduced Voyager, the codename for a new markup language based on HTML 4 but adhering to the stricter syntax rules of XML. [10990600] |By February 1999 the specification had changed name to ''XHTML™ 1.0: The Extensible HyperText Markup Language'', and in January 2000 it was officially adopted as a W3C Recommendation. [10990610] |There are three formal [[Document Type Definition|DTDs]] for XHTML 1.0, corresponding to the three different versions of HTML 4.01: [10990620] |* '''XHTML 1.0 Strict''' is the equivalent to strict HTML 4.01, and includes elements and attributes that have not been marked deprecated in the HTML 4.01 specification. [10990630] |* '''XHTML 1.0 Transitional''' is the equivalent of HTML 4.01 Transitional, and includes the presentational elements (such as center, font and strike) excluded from the strict version. [10990640] |* '''XHTML 1.0 Frameset''' is the equivalent of HTML 4.01 Frameset, and allows for the definition of [[frameset|frameset documents]]—a common Web feature in the late 1990s. [10990650] |The second edition of XHTML 1.0 became a W3C Recommendation in August 2002. [10990660] |=== Modularization of XHTML === [10990670] |The initial draft of ''Modularization of XHTML'' became available in April 1999, and reached Recommendation status in April 2001. [10990680] |[[XHTML Modularization|Modularization]] provides an abstract collection of components through which XHTML can be subsetted and extended. [10990690] |The feature is intended to help XHTML extend it’s reach onto emerging platforms, such as mobile devices and Web-enabled televisions. [10990700] |The first XHTML Family Markup Languages to be developed with this technique were XHTML 1.1 and XHTML Basic 1.0. [10990710] |Another example is XHTML-Print (W3C Recommendation, September 2006), a language designed for printing from mobile devices to low-cost printers. [10990720] |In 2008 ''Modularization of XHTML'' is expected to be superseded by ''XHTML Modularization 1.1'', which adds an [[XML Schema (W3C)|XML Schema]] implementation. [10990730] |=== XHTML 1.1—Module-based XHTML === [10990740] |XHTML 1.1 evolved out of the work surrounding the initial ''Modularization of XHTML'' specification. [10990750] |The W3C released a first draft in September 1999; Recommendation status was reached in May 2001. [10990760] |The modules combined within XHTML 1.1 effectively recreate XHTML 1.0 Strict, with the addition of [[ruby character|ruby annotation]] elements (ruby, rbc, rtc, rb, rt and rp) to better support East-Asian languages. [10990770] |Other changes include removal of the lang attribute (in favour of xml:lang), and removal of the name attribute from the a and map elements. [10990780] |Although XHTML 1.1 is largely compatible with XHTML 1.0 and HTML 4, in August 2002 the W3C issued a formal Note advising that it should not be transmitted with the HTML media type. [10990790] |With limited browser support for the alternate application/xhtml+xml media type, XHTML 1.1 has so far proven unable to gain widespread use. [10990800] |XHTML 1.1 Second Edition is expected in the third quarter of 2008. [10990810] |=== XHTML Basic and XHTML-MP === [10990820] |To support constrained devices, ''[[XHTML Basic]]'' was created by the W3C; it reached Recommendation status in December 2000. [10990830] |XHTML Basic 1.0 is the most restrictive version of XHTML, providing a minimal set of features that even the most limited devices can be expected to support. [10990840] |The [[Open Mobile Alliance]] and it’s predecessor the WAP Forum released three specifications between 2001 and 2006 that extended XHTML Basic 1.0. [10990850] |Known as [[XHTML Mobile Profile]] or XHTML-MP, they were strongly focussed on uniting the differing markup languages used on [[mobile phone|mobile handsets]] at the time. [10990860] |All provide richer form controls than XHTML Basic 1.0, along with varying levels of scripting support. [10990870] |''XHTML Basic 1.1'' became a W3C Proposed Recommendation in June 2008, superseding XHTML-MP 1.2. [10990880] |XHTML Basic 1.1 is almost but not quite a subset of regular XHTML 1.1. [10990890] |The most notable addition over XHTML 1.1 is the inputmode attribute—also found in XHTML-MP 1.2—which provides hints to help browsers improve form entry. [10990900] |=== XHTML 1.2 === [10990910] |The XHTML 2 Working Group is considering the creation a new language based on XHTML 1.1. [10990920] |If XHTML 1.2 is created, it will include [[WAI-ARIA]] and role attributes to better support accessible web applications, and improved [[Semantic Web]] support through [[RDFa]]. [10990930] |The inputmode attribute from XHTML Basic 1.1, along with the target attribute (for specifying [[Framing (World Wide Web)|frame]] targets) may also be present. [10990940] |=== XHTML 2.0 === [10990950] |Between August 2002 and July 2006 the W3C released the first eight Working Drafts of XHTML 2.0, a new version of XHTML able to make a clean break from the past by discarding the requirement of backward compatibility. [10990960] |This lack of compatibility with XHTML 1.x and HTML 4 caused some early controversy in the web developer community. [10990970] |Some parts of the language (such as the role and RDFa attributes) were subsequently split out of the specification and worked on as separate modules, partially to help make the transition from XHTML 1.x to XHTML 2.0 smoother. [10990980] |A ninth draft of XHTML 2.0 is expected to appear in 2008. [10990990] |New features introduced by XHTML 2.0 include: [10991000] |* HTML forms will be replaced by [[XForms]], an XML-based user input specification allowing forms to be displayed appropriately for different rendering devices. [10991010] |* HTML frames will be replaced by [[XFrames]]. [10991020] |* The [[DOM Events]] will be replaced by [[XML Events]], which uses the XML [[Document Object Model]]. [10991030] |* A new list element type, the nl element type, will be included to specifically designate a list as a navigation list. [10991040] |This will be useful in creating nested menus, which are currently created by a wide variety of means like nested unordered lists or nested definition lists. [10991050] |* Any element will be able to act as a [[hyperlink]], e.g.,
  • Articles
  • , similar to [[XLink]]. [10991060] |However, XLink itself is not compatible with XHTML due to design differences. [10991070] |* Any element will be able to reference alternative media with the src attribute, e.g.,

    London Bridge

    is the same as

    London Bridge

    . [10991080] |* The alt attribute of the img element has been removed: alternative text will be given in the content of the img element, much like the object element, e.g., HMS Audacious. [10991090] |* A single heading element (h) will be added. [10991100] |The level of these headings are determined by the depth of the nesting. [10991110] |This allows the use of headings to be infinite, rather than limiting use to six levels deep. [10991120] |* The remaining presentational elements i, b and tt, still allowed in XHTML 1.x (even Strict), will be absent from XHTML 2.0. [10991130] |The only somewhat presentational elements remaining will be sup and sub for superscript and subscript respectively, because they have significant non-presentational uses and are required by certain languages. [10991140] |All other tags are meant to be [[semantic]] instead (e.g. for strong or bolded text) while allowing the user agent to control the presentation of elements via CSS. [10991150] |* The addition of RDF triple with the property and about attributes to facilitate the conversion from XHTML to RDF/XML. [10991160] |=== HTML 5—Vocabulary and APIs for HTML and XHTML === [10991170] |[[HTML 5]] initially grew independently of the W3C, through a loose group of browser manufacturers and other interested parties calling themselves the [[WHATWG]], or Web Hypertext Application Technology Working Group. [10991180] |The WHATWG announced the existence of an open mailing list in June 2004, along with a website bearing the strapline “Maintaining and evolving HTML since 2004.” [10991190] |The key motive of the group was to create a platform for dynamic web applications; they considered XHTML 2.0 to be too document-centric, and not suitable for the creation of forum sites or online shops. [10991200] |In April 2007, the Mozilla Foundation and Opera Software joined Apple in requesting that the newly rechartered HTML Working Group of the W3C adopt the work, under the name of HTML 5. [10991210] |The group resolved to do this the following month, and the First Public Working Draft of HTML 5 was issued by the W3C in January 2008. [10991220] |The most recent W3C Working Draft was published in June 2008. [10991230] |HTML 5 has both a regular text/html serialization and an XML serialization, which is known as XHTML 5. [10991240] |In addition to the markup language, the specification includes a number of [[application programming interfaces]]. [10991250] |The [[Document Object Model]] is extended with APIs for editing, drag-and-drop, data storage and network communication. [10991260] |The language can be considered more compatible with HTML 4 and XHTML 1.x than XHTML 2.0, due to the decision to keep the existing HTML form elements and events model. [10991270] |It adds many new elements not found in XHTML 1.x, however, such as section and aside. [10991280] |(The XHTML 1.2 equivalent of these structural elements would be and .) [10991290] |The specification is expected to add WAI-ARIA support in a future draft. [10991300] |There is currently no indication as to whether HTML 5 will support RDFa, or be limited just to [[microformats]]. [10991310] |== Valid XHTML documents == [10991320] |An XHTML document that conforms to an XHTML specification is said to be ''valid''. [10991330] |Validity assures consistency in document code, which in turn eases processing, but does not necessarily ensure consistent rendering by browsers. [10991340] |A document can be checked for validity with the [[W3C Markup Validation Service]]. [10991350] |In practice, many web development programs such as [[Dreamweaver]] provide code validation based on the [[W3C]] standards. [10991360] |=== DOCTYPEs === [10991370] |In order to validate an XHTML document, a [[Document Type Declaration]], or ''DOCTYPE'', may be used. [10991380] |A DOCTYPE declares to the browser which [[Document Type Definition]] (DTD) the document conforms to. [10991390] |A Document Type Declaration should be placed before the [[root element]]. [10991400] |The [[system identifier]] part of the DOCTYPE, which in these examples is the [[Uniform Resource Locator|URL]] that begins with ''http://'', need only point to a copy of the DTD to use if the validator cannot locate one based on the [[public identifier]] (the other quoted string). [10991410] |It does not need to be the specific URL that is in these examples; in fact, authors are encouraged to use local copies of the DTD files when possible. [10991420] |The public identifier, however, must be character-for-character the same as in the examples. [10991430] |These are the most common XHTML Document Type Declarations: [10991440] |;XHTML 1.0 Strict [10991450] |:<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN""http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> [10991460] |;XHTML 1.0 Transitional [10991470] |:<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN""http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> [10991480] |;XHTML 1.0 Frameset [10991490] |:<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Frameset//EN""http://www.w3.org/TR/xhtml1/DTD/xhtml1-frameset.dtd"> [10991500] |;XHTML 1.1 [10991510] |:<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN""http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> [10991520] |;HTML 5 [10991530] |HTML5 does not require a doctype, and HTML 5 validation is not DTD-based. [10991540] |;XHTML 2.0 [10991550] |XHTML 2.0, [[As of April 2008]], is in a draft phase. [10991560] |If an XHTML 2.0 Recommendation is published with the same document type declaration as in the current Working Draft, the declaration will appear as: [10991570] |:<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 2.0//EN"
    "http://www.w3.org/MarkUp/DTD/xhtml2.dtd">
    [10991580] |A placeholder DTD schema exists at the corresponding URI, though it currently only includes the character reference entities from previous recommendations. [10991590] |XHTML 2 contemplates both a version attribute and an xsi:schemalocation attribute on the root HTML element that could possibly serve as a substitute for any DOCTYPE declaration. [10991600] |==== XML namespaces and schemas ==== [10991610] |In addition to the DOCTYPE, all XHTML elements must be in the appropriate [[XML namespace]] for the version being used. [10991620] |This is usually done by declaring a default namespace on the root element using xmlns="namespace" as in the example below. [10991630] |For XHTML 1.0, XHTML 1.1 and HTML5, this is [10991640] |:<html xmlns="http://www.w3.org/1999/xhtml"> [10991650] |XHTML 2.0 requires both a namespace and an [[W3C XML Schema|XML Schema]] instance declaration. [10991660] |These might be declared as [10991670] |:<html xmlns="http://www.w3.org/2002/06/xhtml2/"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.w3.org/2002/06/xhtml2/ http://www.w3.org/MarkUp/SCHEMA/xhtml2.xsd">
    [10991680] |This example for XHTML 2.0 also demonstrates the use of multiple namespaces within a document. [10991690] |The first xmlns default namespace declaration indicates that elements and attributes whose names have no XML namespace prefix fall within the XHTML 2.0 namespace. [10991700] |The second namespace prefix declaration xmlns:xsi indicates that any elements or attributes prefixed with the xsi: refer to the XMLSchema-Instance namespace. [10991710] |Through this namespace mechanism XML documents allow the use of a mixture of elements and attributes taken from various XML vocabularies while avoiding the potential for clashes of naming between items from independently developed vocabularies. [10991720] |Similar to the case of DOCTYPE above, the actual URL to the [[W3C XML Schema|XML Schema]] file can be changed, as long as the [[Universal Resource Identifier]] (URI) before it (which indicates the XHTML 2.0 namespace) remains the same. [10991730] |The namespace URI is intended to be a persistent and universally unique identifier for the particular version of the specification. [10991740] |If treated as a URL, the actual content located at the site is of no significance. [10991750] |==== XML Declaration ==== [10991760] |A [[character encoding]] may be specified at the beginning of an XHTML document in the XML declaration when the document is served using the application/xhtml+xml MIME type. [10991770] |(If an XML document lacks encoding specification, an XML parser assumes that the encoding is [[UTF-8]] or [[UTF-16]], unless the encoding has already been determined by a higher protocol.) [10991780] |For example: [10991790] |: [10991800] |The declaration may be optionally omitted because it declares as its encoding the default encoding. [10991810] |However, if the document instead makes use of XML 1.1 or another character encoding, a declaration is necessary. [10991820] |[[Internet Explorer]] prior to version 7 enters [[quirks mode]] if it encounters an XML declaration in a document served as text/html. [10991830] |=== Common errors === [10991840] |Some of the most common errors in the usage of XHTML are: [10991850] |* Failing to realize that documents won’t be treated as XHTML unless they are served with an appropriate XML [[Internet media type|MIME type]] [10991860] |* Not closing empty elements (elements without closing tags in HTML4) [10991870] |** Incorrect: [10991880] |** Correct: [10991890] |Note that any of these are acceptable in XHTML: <br></br>, <br/> and <br />. [10991900] |Older HTML-only browsers interpreting it as HTML will generally accept <br> and <br />. [10991910] |* Not closing non-empty elements [10991920] |** Incorrect:

    This is a paragraph.

    This is another paragraph. [10991930] |** Correct:

    This is a paragraph.

    This is another paragraph.

    [10991940] |* Improperly nesting elements (Note that this would also be invalid in [[HTML]]) [10991950] |** Incorrect: This is some text. [10991960] |** Correct: This is some text. [10991970] |* Not putting quotation marks around attribute values [10991980] |** Incorrect: [10991990] |** Correct: [10992000] |** Correct: [10992010] |* Using the ampersand character outside of entities [10992020] |** Incorrect: Cars & Trucks [10992030] |** Correct: Cars &amp; Trucks [10992040] |* Using the ampersand outside of entities in [[Uniform Resource Locator|URL]]s (Note that this would also be invalid in [[HTML]]) [10992050] |** Incorrect: News [10992060] |** Correct: News [10992070] |* Failing to recognize that XHTML elements and attributes are case sensitive [10992080] |** Incorrect:

    The Best Page Ever

    [10992090] |** Correct:

    The Best Page Ever

    [10992100] |* Using attribute minimization [10992110] |** Incorrect: [10992120] |** Correct: [10992130] |* Mis-using CDATA, script-comments and xml-comments when embedding scripts and stylesheets. [10992140] |** This problem can be avoided altogether by putting all script and stylesheet information into separate files and referring to them as follows in the XHTML head element. [10992150] |
     <link rel="stylesheet" href="/style/screen.css" type="text/css" /> <script type="text/javascript" src="/script/site.js"></script> 
    [10992160] |::Note: The format <script …></script>, rather than the more concise <script … />, is required for HTML compatibility when served as MIME type text/html. [10992170] |** If an author chooses to include script or style data inline within an XHTML document, different approaches are recommended depending whether the author intends to serve the page as application/xhtml+xml and target only fully conformant browsers, or serve the page as text/html and try to obtain usability in Internet Explorer 6 and other non-conformant browsers. [10992180] |== Backward compatibility == [10992190] |XHTML 1.x documents are mostly backward compatible with HTML 4 user agents when the appropriate guidelines are followed. [10992200] |XHTML 1.1 is essentially compatible, although the elements for [[ruby character|ruby annotiation]] are not part of the HTML 4 specification and thus generally ignored by HTML 4 browsers. [10992210] |Later XHTML 1.x modules such as those for the role attribute, [[RDFa]] and [[WAI-ARIA]] degrade gracefully in a similar manner. [10992220] |HTML 5 and XHTML 2 are significantly less compatible, although this can be mitigated to some degree through the use of scripting. [10992230] |(This can be simple one-liners, such as the use of “document.createElement()” to register a new HTML element within Internet Explorer, or complete JavaScript frameworks, such as the [[FormFaces]] implementation of [[XForms]].) [10992240] |=== Examples === [10992250] |The followings are examples of XHTML 1.0 Strict. [10992260] |Both of them have the same visual output. [10992270] |The former one follows the [http://www.w3.org/TR/xhtml1/#guidelines HTML Compatibility Guidelines] in Appendix C of the XHTML 1.0 Specification while the latter one breaks backward compatibility but provides cleaner codes. [10992280] |Example 1. XHTML 1.0 Example

    This is an example of an XHTML 1.0 Strict document.Valid XHTML 1.0 Strict

    [10992290] |Example 2. XHTML 1.0 Example

    This is an example of an XHTML 1.0 Strict document.Valid XHTML 1.0 Strict

    [10992300] |Notes: [10992310] |# For further information on the media type recommendation, please refer to [http://www.w3.org/TR/2002/NOTE-xhtml-media-types-20020801/ XHTML Media Types], a W3C Note issued on 2002-08-01. [10992320] |# The "loadpdf" function is actually a workaround for Internet Explorer. [10992330] |It can be replaced by adding within . [10992340] |# The img element does not get a name attribute in the [http://www.w3.org/TR/xhtml1/dtds.html#a_dtd_XHTML-1.0-Strict XHTML 1.0 Strict DTD]. [10992350] |Use id instead. [11000010] |
    XML
    [11000020] |The '''Extensible Markup Language''' ('''XML''') is a general-purpose ''specification'' for creating custom [[markup language]]s. [11000030] |It is classified as an [[extensible language]] because it allows its users to define their own elements. [11000040] |Its primary purpose is to facilitate the sharing of structured data across different information systems, particularly via the [[Internet]], and it is used both to encode documents and to [[Serialization|serialize]] data. [11000050] |In the latter context, it is comparable with other text-based serialization languages such as [[JSON]] and [[YAML]]. [11000060] |It started as a simplified subset of the [[Standard Generalized Markup Language]] (SGML), and is designed to be relatively human-legible. [11000070] |By adding [[semantics|semantic]] constraints, application languages can be implemented in XML. [11000080] |These include [[XHTML]], [[RSS (file format)|RSS]], [[MathML]], [[GraphML]], [[Scalable Vector Graphics]], [[MusicXML]], and thousands of others. [11000090] |Moreover, XML is sometimes used as the [[specification language]] for such application languages. [11000100] |XML is [[W3C recommendation|recommended]] by the [[World Wide Web Consortium]] (W3C). [11000110] |It is a fee-free [[open standard]]. [11000120] |The recommendation specifies both the [[lexical grammar]] and the requirements for [[parsing]]. [11000130] |==Well-formed and valid XML documents== [11000140] |There are two levels of correctness of an XML document: [11000150] |* '''Well-formed'''. [11000160] |A well-formed document conforms to all of XML's syntax rules. [11000170] |For example, if a start-tag appears without a corresponding end-tag, it is not ''well-formed''. [11000180] |A document that is not well-formed is not considered to be XML; a ''conforming parser'' is not allowed to process it. [11000190] |* '''Valid'''. [11000200] |A valid document additionally conforms to some semantic rules. [11000210] |These rules are either user-defined, or included as an [[XML schema]] or [[Document Type Definition|DTD]]. [11000220] |For example, if a document contains an undefined element, then it is not ''valid''; a ''validating parser'' is not allowed to process it. [11000230] |==Well-formed documents: XML syntax== [11000240] |As long as only [[well-formed element|well-formedness]] is required, XML is a generic framework for storing any amount of text or any data whose structure can be represented as a [[Tree (data structure)|tree]]. [11000250] |The only indispensable syntactical requirement is that the document has exactly one '''root element''' (alternatively called the '''document element'''). [11000260] |This means that the text must be enclosed between a root start-tag and a corresponding end-tag. [11000270] |The following is a "well-formed" XML document: This is a book.... [11000280] |The root element can be preceded by an optional '''XML declaration'''. [11000290] |This element states what version of XML is in use (normally 1.0); it may also contain information about [[character encoding]] and external dependencies. [11000300] |The specification ''requires'' that [[Data Processor|processors]] of XML support the pan-[[Unicode]] [[character encoding]]s [[UTF-8]] and [[UTF-16/UCS-2|UTF-16]] ([[UTF-32/UCS-4|UTF-32]] is not mandatory). [11000310] |The use of more limited encodings, such as those based on [[ISO/IEC 8859]], is acknowledged and is widely used and supported. [11000320] |'''Comments''' can be placed anywhere in the tree, including in the text if the content of the element is text or #PCDATA. [11000330] |XML comments start with '''<!--''' and end with '''-->'''. [11000340] |Two dashes (--) may not appear anywhere in the text of the comment. [11000350] | [11000360] |In any meaningful application, additional markup is used to structure the contents of the XML document. [11000370] |The text enclosed by the root tags may contain an arbitrary number of XML elements. [11000380] |The basic syntax for one '''element''' is: Content The two instances of »name« are referred to as the '''start-tag''' and '''end-tag''', respectively. [11000390] |Here, »content« is some text which may again contain XML elements. [11000400] |So, a generic XML document contains a [[tree (data structure)|tree-based data structure]]. [11000410] |Here is an example of a structured XML document: [11000420] | Basic bread Flour Yeast Water Salt Mix all ingredients together. Knead thoroughly. Cover with a cloth, and leave for one hour in warm room. Knead again. Place in a bread baking tin. Cover with a cloth, and leave for one hour in warm room. Bake in the oven at 180(degrees)C for 30 minutes. [11000430] |Attribute values must always be quoted, using single or double quotes; and each attribute name must appear only once in any element. [11000440] |XML requires that elements be properly nested — elements may never overlap, and so must be closed in the opposite order to which they are opened. [11000450] |For example, this fragment of code below cannot be part of a well-formed XML document because the ''title'' and ''author'' elements are closed in the wrong order: Book on Logic<author>Aristotle One way of writing the same information in a way which could be incorporated into a well-formed XML document is as follows: Book on Logic Aristotle [11000460] |XML provides special syntax for representing an element with empty content. [11000470] |Instead of writing a start-tag followed immediately by an end-tag, a document may contain an empty-element tag. [11000480] |An empty-element tag resembles a start-tag but contains a slash just before the closing angle bracket. [11000490] |The following three examples are equivalent in XML: An empty-element may contain attributes: [11000500] |===Entity references=== [11000510] |An [[SGML entity|entity]] in XML is a named body of data, usually text. [11000520] |Entities are often used to represent single characters that cannot easily be entered on the keyboard; they are also used to represent pieces of standard ("boilerplate") text that occur in many documents, especially if there is a need to allow such text to be changed in one place only. [11000530] |Special characters can be represented either using [[SGML entity|entity]] references, or by means of [[numeric character reference]]s. [11000540] |An example of a numeric character reference is "&#x20AC;", which refers to the [[Euro symbol]] by means of its [[Unicode]] codepoint in [[hexadecimal]]. [11000550] |An entity reference is a [[placeholder]] that represents that entity. [11000560] |It consists of the entity's name preceded by an [[ampersand]] ("&") and followed by a [[semicolon]] (";"). [11000570] |XML has five [[predeclared]] entities: [11000580] |* &amp; (& or "ampersand") [11000590] |* &lt; (< or "less than") [11000600] |* &gt; (> or "greater than") [11000610] |* &apos; (' or "apostrophe") [11000620] |* &quot; (" or "quotation mark") [11000630] |Here is an example using a predeclared XML entity to represent the ampersand in the name "AT&T": AT&T Additional entities (beyond the predefined ones) can be declared in the document's [[XML#DTD|Document Type Definition (DTD)]]. [11000640] |A basic example of doing so in a minimal internal DTD follows. [11000650] |Declared entities can describe single characters or pieces of text, and can reference each other. ]> ©right-notice; [11000660] |When viewed in a suitable browser, the XML document above appears as: [11000670] |Copyright © 2006, XYZ Enterprises [11000680] |====Numeric character references==== [11000690] |Numeric character references look like entity references, but instead of a name, they contain the "[[number sign|#]]" character followed by a number. [11000700] |The number (in decimal or "x"-prefixed [[hexadecimal]]) represents a Unicode code point. [11000710] |Unlike entity references, they are neither predeclared nor do they need to be declared in the document's DTD. [11000720] |They have typically been used to represent characters that are not easily encodable, such as an [[Arabic language|Arabic]] character in a document produced on a European computer. [11000730] |The ampersand in the "AT&T" example could also be [[escape character|escaped]] like this (decimal 38 and hexadecimal 26 both represent the Unicode code point for the "&" character): AT&T AT&T [11000740] |Similarly, in the previous example, notice that “©” is used to generate the “©” symbol. [11000750] |See also [[numeric character reference]]s. [11000760] |===Well-formed documents=== [11000770] |In XML, a [[well-formed element|well-formed]] document must conform to the following rules, among others: [11000780] |* Non-empty elements are [[delimiter|delimited]] by both a start-tag and an end-tag. [11000790] |* Empty elements may be marked with an empty-element (self-closing) tag, such as <IAmEmpty />. [11000800] |This is equal to <IAmEmpty></IAmEmpty>. [11000810] |* All attribute values are quoted with either single (') or double (") quotes. [11000820] |Single quotes close a single quote and double quotes close a double quote. [11000830] |* Tags may be nested but must not overlap. [11000840] |Each non-root element must be completely contained in another element. [11000850] |* The document complies with its declared character encoding. [11000860] |The encoding may be declared or implied externally, such as in "Content-Type" headers when a document is transported via [[Hypertext Transfer Protocol|HTTP]], or internally, using explicit markup at the very beginning of the document. [11000870] |When no such declaration exists, a Unicode encoding is assumed, as defined by a Unicode [[Byte-order mark|Byte Order Mark]] before the document's first character. [11000880] |If the mark does not exist, UTF-8 encoding is assumed. [11000890] |Element names are case-sensitive. [11000900] |For example, the following is a well-formed matching pair: [11000910] |:<Step> ... </Step> [11000920] |whereas this is not [11000930] |:<Step> ... </step> [11000940] |By carefully choosing the names of the XML elements one may convey the meaning of the data in the [[Markup language|markup]]. [11000950] |This increases human readability while retaining the rigor needed for software parsing. [11000960] |Choosing meaningful names implies the [[semantics]] of elements and attributes to a human reader without reference to external documentation. [11000970] |However, this can lead to verbosity, which complicates [[authoring]] and increases [[file size]]. [11000980] |===Automatic verification=== [11000990] |It is relatively simple to verify that a document is well-formed or validated XML, because the rules of well-formedness and validation of XML are designed for portability of tools. [11001000] |The idea is that any tool designed to work with XML files will be able to work with XML files written in any XML language (or XML application). [11001010] |Here are some examples of ways to verify XML documents: [11001020] |* load it into an XML-capable browser, such as [[Mozilla Firefox|Firefox]] or [[Internet Explorer]] [11001030] |* use a tool like xmlwf (usually bundled with [[Expat (XML)|expat]]) [11001040] |* parse the document, for instance in [[Ruby programming language|Ruby]]: [11001050] |irb> require "rexml/document" irb> include REXML irb> doc = Document.new(File.new("test.xml")).root [11001060] |==Valid documents: XML semantics== [11001070] |By leaving the names, allowable hierarchy, and meanings of the elements and attributes open and definable by a customizable ''[[XML schema|schema]] or [[Document Type Definition|DTD]]'', XML provides a syntactic foundation for the creation of purpose-specific, XML-based markup languages. [11001080] |The general syntax of such languages is rigid — documents must adhere to the general rules of XML, ensuring that all XML-aware software can at least read and understand the relative arrangement of information within them. [11001090] |The schema merely supplements the syntax rules with a set of constraints. [11001100] |Schemas typically restrict element and attribute names and their allowable containment hierarchies, such as only allowing an element named 'birthday' to contain one element named 'month' and one element named 'day', each of which has to contain only character data. [11001110] |The constraints in a schema may also include [[data type]] assignments that affect how information is processed; for example, the 'month' element's character data may be defined as being a month according to a particular schema language's conventions, perhaps meaning that it must not only be formatted a certain way, but also must not be processed as if it were some other type of data. [11001120] |An XML document that complies with a particular schema/DTD, in addition to being well-formed, is said to be '''valid'''. [11001130] |An XML schema is a description of a type of XML document, typically expressed in terms of [[constraints]] on the structure and content of documents of that type, above and beyond the basic constraints imposed by XML itself. [11001140] |A number of standard and proprietary XML schema languages have emerged for the purpose of formally expressing such schemas, and some of these languages are XML-based, themselves. [11001150] |Before the advent of generalised data description languages such as SGML and XML, software designers had to define special [[file format]]s or small languages to share data between programs. [11001160] |This required writing detailed [[specifications]] and special-purpose parsers and [[writer]]s. [11001170] |XML's regular structure and strict parsing rules allow software designers to leave parsing to standard tools, and since XML provides a general, [[data model]]-oriented framework for the development of application-specific languages, software designers need only concentrate on the development of rules for their data, at relatively high levels of abstraction. [11001180] |Well-tested tools exist to [[validate]] an XML document "against" a schema: the tool automatically [[verify|verifies]] whether the document conforms to constraints expressed in the schema. [11001190] |Some of these validation tools are included in XML parsers, and some are packaged separately. [11001200] |Other usages of schemas exist: XML editors, for instance, can use schemas to support the editing process (by suggesting valid elements and attributes names, etc). [11001210] |====DTD==== [11001220] |The oldest schema format for XML is the [[Document Type Definition]] (DTD), inherited from SGML. [11001230] |While DTD support is ubiquitous due to its inclusion in the XML 1.0 standard, it is seen as limited for the following reasons: [11001240] |* It has no support for newer [[feature]]s of XML, most importantly [[XML Namespace|namespaces]]. [11001250] |* It lacks expressiveness. [11001260] |Certain formal aspects of an XML document cannot be captured in a DTD. [11001270] |* It uses a custom non-XML syntax, inherited from [[SGML]], to describe the schema. [11001280] |DTD is still used in many applications because it is considered the easiest to read and write. [11001290] |====XML Schema==== [11001300] |A newer [[XML schema]] language, described by the W3C as the successor of DTDs, is [[XML Schema (W3C)|XML Schema]], or more informally referred to by the [[acronym and initialism|initialism]] for XML Schema instances, XSD (XML Schema Definition). [11001310] |XSDs are far more powerful than DTDs in describing XML languages. [11001320] |They use a rich [[datatype|datatyping]] system, allow for more detailed constraints on an XML document's logical structure, and must be processed in a more robust validation framework. [11001330] |XSDs also use an XML-based format, which makes it possible to use ordinary XML tools to help process them, although XSD implementations require much more than just the ability to read XML. [11001340] |Criticisms of XSD include the following: [11001350] |*The specification is very large, which makes it difficult to understand and implement. [11001360] |*The XML-based syntax leads to verbosity in schema descriptions, which makes XSDs harder to read and write. [11001370] |*Schema validation can be an expensive addition to XML parsing, especially for high volume systems. [11001380] |*The modeling capabilities are very limited, with no ability to allow attributes to influence [[content model]]s. [11001390] |*The [[type derivation model]] is very limited, in particular that derivation by extension is rarely useful. [11001400] |*Database-related [[data transfer]] is supported with ideas such as [[nillability]], but the requirements of industrial publishing are under-supported. [11001410] |*The key/[[keyref]]/[[uniqueness]] mechanisms are not type-aware. [11001420] |*The [[PSVI]] concept (Post Schema Validation Infoset) does not have a standard XML representation or [[Application Programming Interface]], thus it works against [[vendor independence]] unless revalidation is performed. [11001430] |====RELAX NG==== [11001440] |Another popular schema language for XML is [[RELAX NG]]. [11001450] |Initially specified by [[OASIS (organization)|OASIS]], RELAX NG is now also an ISO international standard (as part of [[DSDL]]). [11001460] |It has two formats: an XML based syntax and a non-XML compact syntax. [11001470] |The compact syntax aims to increase readability and writability but, since there is a well-defined way to translate the compact syntax to the XML syntax and back again by means of [[James Clark (XML expert)|James Clark]]'s [http://www.thaiopensource.com/relaxng/trang.html Trang conversion tool], the advantage of using standard XML tools is not lost. [11001480] |RELAX NG has a simpler definition and validation framework than XML Schema, making it easier to use and implement. [11001490] |It also has the ability to use [[datatype]] framework [[plug-in]]s; a RELAX NG schema author, for example, can require values in an XML document to conform to definitions in XML Schema Datatypes. [11001500] |====ISO DSDL and other schema languages==== [11001510] |The ISO [[DSDL]] (Document Schema Description Languages) standard brings together a comprehensive set of small schema languages, each targeted at specific problems. [11001520] |DSDL includes [[RELAX NG]] full and compact syntax, [[Schematron]] assertion language, and languages for defining datatypes, character repertoire constraints, renaming and entity expansion, and namespace-based [[routing]] of document fragments to different validators. [11001530] |DSDL schema languages do not have the vendor support of XML Schemas yet, and are to some extent a grassroots reaction of industrial publishers to the lack of utility of XML Schemas for [[publishing]]. [11001540] |Some schema languages not only describe the structure of a particular XML format but also offer limited facilities to influence processing of individual XML files that conform to this format. [11001550] |DTDs and XSDs both have this ability; they can for instance provide attribute defaults. [11001560] |RELAX NG and Schematron intentionally do not provide these; for example the [[infoset]] augmentation facility. [11001570] |=== International use === [11001580] |XML supports the direct use of almost any Unicode character in element names, attributes, comments, character data, and processing instructions (other than the ones that have special symbolic meaning in XML itself, such as the open corner bracket, "<"). [11001590] |Therefore, the following is a well-formed XML document, even though it includes both [[Chinese character|Chinese]] and [[Cyrillic alphabet|Cyrillic]] characters: <俄語>Данные [11001600] |== Displaying XML on the web == [11001610] |XML documents do not carry information about how to display the data. [11001620] |Without using [[Cascading Style Sheets|CSS]] or [[Extensible Stylesheet Language|XSL]], a generic XML document is rendered as raw XML text by most [[web browser]]s. [11001630] |Some display it with 'handles' (e.g. + and - signs in the margin) that allow parts of the structure to be expanded or collapsed with mouse-clicks. [11001640] |In order to style the rendering in a browser with CSS, the XML document must include a reference to the [[stylesheet]]: [11001650] |Note that this is different from specifying such a stylesheet in HTML, which uses the <link> element. [11001660] |[[Extensible Stylesheet Language]] (XSL) can be used to alter the format of XML data, either into HTML or other formats that are suitable for a browser to display. [11001670] |To specify [[client-side]] [[XSL Transformations|XSL Transformation]] (XSLT), the following processing instruction is required in the XML: [11001680] |Client-side XSLT is supported by many [[web browser]]s. [11001690] |Alternatively, one may use XSL to convert XML into a displayable format ''on the [[Server (computing)|server]]'' rather than being dependent on the [[end-user]]'s browser capabilities. [11001700] |The end-user is not aware of what has gone on 'behind the scenes'; all they see is well-formatted, displayable data. [11001710] |See the XSLT article for [[XSL Transformations#XSLT examples|an example of server-side XSLT in action]]. [11001720] |== XML extensions == [11001730] |* '''[[XPath]]''' makes it possible to refer to individual parts of an XML document. [11001740] |This provides random access to XML data for other technologies, including XSLT, XSL-FO, XQuery etc. [11001750] |XPath expressions can refer to all or part of the text, data and values in XML elements, attributes, processing instructions, comments etc. [11001760] |They can also access the names of elements and attributes. [11001770] |XPaths can be used in both valid and well-formed XML, with and without defined namespaces. [11001780] |*'''[[XInclude]]''' defines the ability for XML files to include all or part of an external file. [11001790] |When processing is complete, the final XML infoset has no XInclude elements, but instead has copied the documents or parts thereof into the final infoset. [11001800] |It uses XPath to refer to a portion of the document for partial inclusions. [11001810] |*'''[[XQuery]]''' is to XML and [[XML database|XML Databases]] what [[SQL]] and [[PL/SQL]] are to [[relational databases]]: ways to access, manipulate and return XML. [11001820] |*'''[[XML Namespace]]s''' enable the same document to contain XML elements and attributes taken from different vocabularies, without any [[naming collision]]s occurring. [11001830] |*'''[[XML Signature]]''' defines the syntax and processing rules for creating [[digital signatures]] on XML content. [11001840] |*'''[[XML Encryption]]''' defines the syntax and processing rules for [[encryption|encrypting]] XML content. [11001850] |* '''[[XPointer]]''' is a system for addressing components of XML-based internet media. [11001860] |XML files may be served with a variety of [[MIME|Media types]]. [11001870] |RFC 3023 defines the types "application/xml" and "text/xml", which say only that the data is in XML, and nothing about its semantics. [11001880] |The use of "text/xml" has been criticized as a potential source of encoding problems but is now in the process of being deprecated. [11001890] |RFC 3023 also recommends that XML-based languages be given media types beginning in "application/" and ending in "+xml"; for example "application/atom+xml" for [[Atom (standard)|Atom]]. [11001900] |This page discusses further [[XML and MIME]]. [11001910] |== Processing XML files == [11001920] |Three traditional techniques for processing XML files are: [11001930] |* Using a programming language and the [[Simple API for XML|SAX]] [[Application programming interface|API]]. [11001940] |* Using a programming language and the [[DOM (XML API)|DOM]] [[Application programming interface|API]]. [11001950] |* Using a transformation engine and a filter [11001960] |More recent and emerging techniques for processing XML files are: [11001970] |* Pull Parsing [11001980] |* Non-Extractive Parsing (i.e. in-situ parsing) [11001990] |* Data binding [11002000] |=== Simple API for XML (SAX)=== [11002010] |[[Simple API for XML|SAX]] is a [[lexical analysis|lexical]], [[Event-driven programming|event-driven]] interface in which a document is read serially and its contents are reported as "[[callback]]s" to various [[method (computer science)|method]]s on a [[event handler|handler object]] of the user's design. [11002020] |SAX is fast and efficient to implement, but difficult to use for extracting information at random from the XML, since it tends to burden the application author with keeping track of what part of the document is being processed. [11002030] |It is better suited to situations in which certain types of information are always handled the same way, no matter where they occur in the document. [11002040] |=== DOM === [11002050] |[[Document Object Model|DOM]] is an [[interface]]-oriented [[Application Programming Interface]] that allows for navigation of the entire document as if it were a tree of "[[Node (computer science)|Node]]" [[Object (computer science)|object]]s representing the document's contents. [11002060] |A DOM document can be created by a parser, or can be generated manually by users (with limitations). [11002070] |Data types in DOM Nodes are abstract; implementations provide their own [[programming]] language-specific [[binding]]s. [11002080] |DOM implementations tend to be [[memory]] intensive, as they generally require the entire document to be loaded into memory and constructed as a tree of objects before access is allowed. [11002090] |DOM is supported in Java by several packages that usually come with the standard libraries. [11002100] |As the DOM specification is regulated by the [[World Wide Web Consortium]], the main interfaces (Node, Document, etc.) are in the package org.w3c.dom.*, as well as some of the events and interfaces for other capabilities like serialization (output). [11002110] |The package com.sun.org.apache.xml.internal.serialize.* provides the serialization (output capacities) by implementing the appropriate interfaces, while the javax.xml.parsers.* package parses data to create DOM XML documents for manipulation. [11002120] |=== Non-extractive XML Processing API === [11002130] |Non-extractive XML Processing API is a new and emerging category of parsers that aim to overcome the fundamental limitations of DOM and SAX. [11002140] |The most representative is VTD-XML, which abolishes the object-oriented modeling of XML hierarchy and instead uses 64-bit Virtual Token Descriptors (encoding offsets, lengths, depths, and types) of XML tokens. [11002150] |VTD-XML's approach enables a number of interesting features/enhancements, such as high performance, low memory usage [8], ASIC implementation [9], incremental update [10], and native XML indexing [11] [12]. [11002160] |=== Transformation engines and filters === [11002170] |A [[filter (software)|filter]] in the [[Extensible Stylesheet Language]] (XSL) family can transform an XML file for [[display]]ing or [[Computer printer|print]]ing. [11002180] |* '''[[XSL-FO]]''' is a declarative, XML-based [[page layout]] language. [11002190] |An XSL-FO processor can be used to convert an XSL-FO document into another non-XML format, such as [[PDF]]. [11002200] |* '''[[XSLT]]''' is a declarative, XML-based document transformation language. [11002210] |An XSLT processor can use an XSLT ''stylesheet'' as a guide for the conversion of the data tree represented by one XML document into another tree that can then be [[serialize]]d as XML, HTML, plain text, or any other format supported by the processor. [11002220] |* '''[[XQuery]]''' is a W3C language for [[query]]ing, constructing and transforming XML data. [11002230] |* '''[[XPath]]''' is a DOM-like node tree [[data model]] and [[path expression]] language for selecting data within XML documents. [11002240] |XSL-FO, XSLT and XQuery all make use of XPath. [11002250] |XPath also includes a useful [[function library]]. [11002260] |=== Pull parsing === [11002270] |Pull parsing treats the document as a series of items which are read in sequence using the Iterator design pattern. [11002280] |This allows for writing of [[recursive descent parser|recursive-descent parsers]] in which the structure of the code performing the parsing mirrors the structure of the XML being parsed, and intermediate parsed results can be used and accessed as local variables within the methods performing the parsing, or passed down (as method parameters) into lower-level methods, or returned (as method return values) to higher-level methods. [11002290] |Examples of pull parsers include [[StAX]] in the [[Java (programming language)|Java]] programming language, SimpleXML in PHP and System.Xml.XmlReader in .NET. [11002300] |A pull parser creates an iterator that sequentially visits the various elements, attributes, and data in an XML document. [11002310] |Code which uses this 'iterator' can test the current item (to tell, for example, whether it is a start or end element, or text), and inspect its attributes (local name, namespace, values of XML attributes, value of text, etc.), and can also move the iterator to the 'next' item. [11002320] |The code can thus extract information from the document as it traverses it. [11002330] |The recursive-descent approach tends to lend itself to keeping data as typed local variables in the code doing the parsing, while SAX, for instance, typically requires a parser to manually maintain intermediate data within a stack of elements which are parent elements of the element being parsed. [11002340] |Pull-parsing code can be more straightforward to understand and maintain than SAX parsing code. [11002350] |=== Non-extractive XML Processing API === [11002360] |Non-extractive XML Processing API is a new and emerging category of parsers that aim to overcome the fundamental limitations of DOM and SAX. [11002370] |The most representative is [[VTD-XML]], which abolishes the object-oriented modeling of XML hierarchy and instead uses 64-bit Virtual Token Descriptors (encoding offsets, lengths, depths, and types) of XML tokens. [11002380] |VTD-XML's approach enables a number of interesting features/enhancements, such as high performance, low memory usage , ASIC implementation , incremental update , and native XML indexing . [11002390] |=== Data binding === [11002400] |Another form of XML Processing API is [[XML data binding|data binding]], where XML data is made available as a custom, strongly typed programming language data structure, in contrast to the interface-oriented DOM. [11002410] |Example data binding systems include the [[Java (programming language)|Java]] [[Architecture]] for XML Binding ([[JAXB]]). [11002420] |=== Specific XML applications and editors === [11002430] |The [[Native and foreign format|native file format]] of [[OpenOffice.org]], [[AbiWord]], and [[Apple Computer|Apple]]'s [[iWork]] applications is XML. [11002440] |Some parts of [[Microsoft Office]] 2007 are also able to edit XML files with a user-supplied schema (but not a DTD), and Microsoft has released a file format compatibility kit for Office 2003 that allows previous versions of Office to save in the new XML based format. [11002450] |There are dozens of other [[XML editor]]s available. [11002460] |==History== [11002470] |The versatility of [[SGML]] for dynamic information display was understood by early digital media publishers in the late 1980s prior to the rise of the Internet. [11002480] |By the mid-1990s some practitioners of SGML had gained experience with the then-new [[World Wide Web]], and believed that SGML offered solutions to some of the problems the Web was likely to face as it grew. [11002490] |[[Dan Connolly]] added SGML to the list of W3C's activities when he joined the staff in 1995; work began in mid-1996 when [[Jon Bosak]] developed a charter and recruited collaborators. [11002500] |Bosak was well connected in the small community of people who had experience both in SGML and the Web. [11002510] |He received support in his efforts from [[Microsoft]]. [11002520] |XML was compiled by a [[working group]] of eleven members, supported by an (approximately) 150-member Interest Group. [11002530] |Technical debate took place on the Interest Group mailing list and issues were resolved by consensus or, when that failed, majority vote of the Working Group. [11002540] |A record of design decisions and their rationales was compiled by [[Michael Sperberg-McQueen]] on December 4th 1997. [11002550] |[[James Clark (XML expert)|James Clark]] served as Technical Lead of the Working Group, notably contributing the empty-element "" syntax and the name "XML". [11002560] |Other names that had been put forward for consideration included "MAGMA" (Minimal Architecture for Generalized Markup Applications), "SLIM" (Structured Language for Internet Markup) and "MGML" (Minimal Generalized Markup Language). [11002570] |The co-editors of the specification were originally [[Tim Bray]] and [[Michael Sperberg-McQueen]]. [11002580] |Halfway through the project Bray accepted a consulting engagement with [[Netscape Communications Corporation|Netscape]], provoking vociferous protests from Microsoft. [11002590] |Bray was temporarily asked to resign the editorship. [11002600] |This led to intense dispute in the Working Group, eventually solved by the appointment of Microsoft's [[Jean Paoli]] as a third co-editor. [11002610] |The XML Working Group never met face-to-face; the design was accomplished using a combination of email and weekly teleconferences. [11002620] |The major design decisions were reached in twenty weeks of intense work between July and November of 1996, when the first Working Draft of an XML specification was published. [11002630] |Further design work continued through [[1997]], and XML 1.0 became a [[W3C]] Recommendation on [[February 10]], [[1998]]. [11002640] |XML 1.0 achieved the Working Group's goals of Internet usability, general-purpose usability, SGML compatibility, facilitation of easy development of processing software, minimization of optional features, legibility, formality, conciseness, and ease of authoring. [11002650] |Like its antecedent SGML, XML allows for some redundant syntactic constructs and includes repetition of element identifiers. [11002660] |In these respects, terseness was not considered essential in its structure. [11002670] |===Sources=== [11002680] |XML is a profile of an ISO standard [[SGML]], and most of XML comes from SGML unchanged. [11002690] |From SGML comes the separation of logical and physical structures (elements and entities), the availability of grammar-based validation (DTDs), the separation of data and metadata (elements and attributes), mixed content, the separation of processing from representation (processing instructions), and the default angle-bracket syntax. [11002700] |Removed were the SGML Declaration (XML has a fixed delimiter set and adopts [[Unicode]] as the document [[Character encoding|character set]]). [11002710] |Other sources of technology for XML were the [[Text Encoding Initiative]] (TEI), which defined a profile of SGML for use as a 'transfer syntax'; [[HTML]], in which elements were synchronous with their resource, the separation of document character set from resource encoding, the xml:lang attribute, and the [[HTTP]] notion that metadata accompanied the resource rather than being needed at the declaration of a link; and the Extended Reference Concrete Syntax (ERCS), from which XML 1.0's naming rules were taken, and which had introduced hexadecimal numeric character references and the concept of references to make available all Unicode characters. [11002720] |Ideas that developed during discussion which were novel in XML, were the algorithm for encoding detection and the encoding header, the processing instruction target, the xml:space attribute, and the new close delimiter for empty-element tags. [11002730] |===Versions=== [11002740] |There are two current versions of XML. [11002750] |The first, ''XML 1.0'', was initially defined in [[1998]]. [11002760] |It has undergone minor revisions since then, without being given a new version number, and is currently in its fourth edition, as published on [[August 16]], [[2006]]. [11002770] |It is widely implemented and still recommended for general use. [11002780] |The second, ''XML 1.1'', was initially published on [[February 4]], [[2004]], the same day as XML 1.0 Third Edition, and is currently in its second edition, as published on [[August 16]], [[2006]]. [11002790] |It contains features — some contentious — that are intended to make XML easier to use in certain cases - mainly enabling the use of line-ending characters used on [[EBCDIC]] platforms, and the use of scripts and characters absent from Unicode 2.0. [11002800] |XML 1.1 is not very widely implemented and is recommended for use only by those who need its unique features. [11002810] |XML 1.0 and XML 1.1 differ in the requirements of characters used for element and attribute names: XML 1.0 only allows characters which are defined in Unicode 2.0, which includes most world scripts, but excludes those which were added in later Unicode versions. [11002820] |Among the excluded scripts are [[Mongolian language|Mongolian]], [[Cambodian language|Cambodian]], [[Amharic]], [[Burmese language|Burmese]], and others. [11002830] |Almost any Unicode character can be used in the character data and attribute values of an XML 1.1 document, even if the character is not defined, aside from having a code point, in the current version of Unicode. [11002840] |The approach in XML 1.1 is that only certain characters are forbidden, and everything else is allowed, whereas in XML 1.0, only certain characters are explicitly allowed, thus XML 1.0 cannot accommodate the addition of characters in future versions of Unicode. [11002850] |In character data and attribute values, XML 1.1 allows the use of more [[control character]]s than XML 1.0, but, for "robustness", most of the control characters introduced in XML 1.1 must be expressed as numeric character references. [11002860] |Among the supported control characters in XML 1.1 are two line break codes that must be treated as whitespace. [11002870] |Whitespace characters are the only control codes that can be written directly. [11002880] |There are also discussions on an XML 2.0, although it remains to be seen if such will ever come about. [http://www.textuality.com/xml/xmlSW.html XML-SW] (SW for [[skunk works]]), written by one of the original developers of XML, contains some proposals for what an XML 2.0 might look like: elimination of DTDs from syntax, integration of [[namespace (computer science)#XML|namespace]]s, [[XML Base]] and [[XML Information Set]] (''infoset'') into the base standard. [11002890] |The World Wide Web Consortium also has an XML Binary Characterization Working Group doing preliminary research into use cases and properties for a binary encoding of the XML infoset. [11002900] |The working group is not chartered to produce any official standards. [11002910] |Since XML is by definition text-based, ITU-T and ISO are using the name ''[[Fast Infoset]][http://asn1.elibel.tm.fr/xml/finf.htm]'' for their own binary infoset to avoid confusion (see ITU-T Rec. X.891 | ISO/IEC 24824-1). [11002920] |===Patent claims=== [11002930] |In October 2005 the small company [[Scientigo]] publicly asserted that two of its patents, and , apply to the use of XML. [11002940] |The patents cover the "modeling, storage and transfer [of data] in a particular ''non-hierarchical'', non-integrated neutral form", according to their applications, which were filed in 1997 and 1999. [11002950] |Scientigo CEO [[Doyal Bryant]] expressed a desire to "monetize" the patents but stated that the company was "not interested in having us against the world." [11002960] |He said that Scientigo was discussing the patents with several large corporations. [11002970] |XML users and independent experts responded to Scientigo's claims with widespread skepticism and criticism. [11002980] |Some derided the company as a [[patent troll]]. [11002990] |[[Tim Bray]] described any claims that the patents covered XML as "ridiculous on the face of it". [11003000] |== Critique of XML == [11003010] |Commentators have offered various critiques of XML, suggesting circumstances where XML provides both advantages and potential disadvantages. [11003020] |===Advantages of XML=== [11003030] |* It is text-based. [11003040] |* It supports [[Unicode]], allowing almost any information in any written human language to be communicated. [11003050] |* It can represent common [[computer science]] [[data structure]]s: [[record (computer science)|record]]s, [[List (computing)|list]]s and [[tree data structure|tree]]s. [11003060] |* Its [[self-documenting]] format describes [[structure]] and [[field name]]s as well as specific values. [11003070] |* The strict [[syntax]] and [[parsing]] requirements make the necessary [[parser|parsing algorithms]] extremely simple, efficient, and consistent. [11003080] |* XML is heavily used as a format for [[document storage]] and processing, both online and offline. [11003090] |* It is based on [[international standards]]. [11003100] |* It can be updated incrementally. [11003110] |* It allows validation using schema languages such as [[XSD]] and [[Schematron]], which makes effective unit-testing, firewalls, acceptance testing, contractual specification and software construction easier. [11003120] |* The [[hierarchy|hierarchical]] structure is suitable for most (but not all) types of documents. [11003130] |* It is platform-independent, thus relatively immune to changes in technology. [11003140] |* Forward and backward compatibility are relatively easy to maintain despite changes in DTD or Schema. [11003150] |* Its predecessor, [[SGML]], has been in use since [[1986]], so there is extensive experience and software available. [11003160] |===Disadvantages of XML=== [11003170] |* XML syntax is redundant or large relative to binary representations of similar data, especially with [[Table (information)|tabular]] data. [11003180] |* The redundancy may affect application efficiency through higher storage, transmission and processing costs. [11003190] |* XML syntax is verbose, especially for human readers, relative to other alternative 'text-based' data transmission formats. [11003200] |* The [[hierarchical model]] for representation is limited in comparison to an [[object oriented]] [[Graph (mathematics)|graph]]. [11003210] |* Expressing overlapping (non-hierarchical) node relationships requires extra effort. [11003220] |* XML namespaces are problematic to use and namespace support can be difficult to correctly implement in an XML parser. [11003230] |* XML is commonly depicted as "[[self-documenting]]" but this depiction ignores critical ambiguities. [11003240] |* The distinction between content and attributes in XML seems unnatural to some and makes designing XML data structures harder. [11003250] |== Standardization == [11003260] |In addition to the [[ISO]] standards mentioned above, other related document include [11003270] |* ISO/IEC 8825-4:2002 ''Information technology -- ASN.1 encoding rules: XML Encoding Rules (XER)'' [11003280] |* ISO/IEC 8825-5:2004 ''Information technology -- ASN.1 encoding rules: Mapping W3C XML schema definitions into ASN.1'' [11003290] |* ISO/IEC 9075-14:2006 ''Information technology -- Database languages -- SQL -- Part 14: XML-Related Specifications (SQL/XML)'' [11003300] |* ISO 10303-28:2007 ''Industrial automation systems and integration -- Product data representation and exchange -- Part 28: Implementation methods: XML representations of EXPRESS schemas and data, using XML schemas'' [11003310] |* ISO/IEC 13250-3:2007 ''Information technology -- Topic Maps -- Part 3: XML syntax'' [11003320] |* ISO/IEC 13522-5:1997 ''Information technology -- Coding of multimedia and hypermedia information -- Part 5: Support for base-level interactive applications'' [11003330] |* ISO/IEC 13522-8:2001 ''Information technology -- Coding of multimedia and hypermedia information -- Part 8: XML notation for ISO/IEC 13522-5'' [11003340] |* ISO/IEC 18056:2007 ''Information technology -- Telecommunications and information exchange between systems -- XML Protocol for Computer Supported Telecommunications Applications (CSTA) Phase III'' [11003350] |* ISO/IEC 19503:2005 ''Information technology -- XML Metadata Interchange (XMI)'' [11003360] |* ISO/IEC 19776-1:2005 ''Information technology -- Computer graphics, image processing and environmental data representation -- Extensible 3D (X3D) encodings -- Part 1: Extensible Markup Language (XML) encoding [11003370] |* ISO/IEC 22537:2006 ''Information technology -- ECMAScript for XML (E4X) specification'' [11003380] |* ISO 22643:2003 ''Space data and information transfer systems -- Data entity dictionary specification language (DEDSL) -- XML/DTD Syntax'' [11003390] |* ISO/IEC 23001-1:2006 ''Information technology -- MPEG systems technologies -- Part 1: Binary MPEG format for XML'' [11003400] |* ISO 24531:2007 ''Intelligent transport systems -- System architecture, taxonomy and terminology -- Using XML in ITS standards, data registries and data dictionaries''