Chinese word-segmented writing

<h2 id="history">History</h2>
In <a href="/facts/Ancient_China/TNqRSvQ4">ancient China</a>, texts were written without punctuation marks, which led to the reader needing to spend a considerable amount of time finding the boundary of a sentence. It was not until the early 1900s when the present punctuation marks were adopted.<a class="footnote-ref" id="fnref:3" href="#fn:3">3</a>
In the 1950s, there was a proposal for the employment of word-segmented writing in a discussion among the Chinese linguists, however it was not passed.<a class="footnote-ref" id="fnref:4" href="#fn:4">4</a>
In 1987, the idea of Chinese word-segmented writing was put forward again by Chen Liwei in an international conference on Chinese information processing.<a class="footnote-ref" id="fnref:5" href="#fn:5">5</a>
Chinese word-segmented writing was first put into application no later than 1998, when a paper entitled Written Chinese Word Segmentation Revisited: Ten advantages of word-segmented writing was published in a key academic journal in China.<a class="footnote-ref" id="fnref:6" href="#fn:6">6</a> The whole paper, seven pages altogether, was written word-segmentedly, with the abstract presented as:

摘要: 单词 的 切分 对 现代 汉语 的 运用、研究 和 计算机 信息 处理 等 都 具有 相当 重要 的 意义。本文 阐述 书面 汉语 分词 连写 的 十 大 好处 , 并 讨论 一些 实施 方面 的 问题。文章 全文 分词 连写。

In 2018, a one-paragraph short article was published on Wikiversity entitled Word segmentation of Hanzi,<a class="footnote-ref" id="fnref:7" href="#fn:7">7</a> with the Chinese text word-segmented as follows:

历史上，中国古文 是 没有 标点符号的。读者 需要 付出 额外的 精力 专注于 断句，而且 稍有差池 便会 造成 误读。所谓 差之毫厘 失之千里。引入 标点符号 是 一次 重大的 文字改革，使得 汉字文本的 阅读效率 有了 很大的 提高。但 中文的 改革 才 刚刚 起步， 远未达到 尽善尽美的 程度。至少 在 阅读效率 方面 仍然 存在着 一个 显而易见的 障碍 - 断词 （汉字的 分词连写）。

The first book written in word segmentation was 语言理论 (Language theories) published in 2000.<a class="footnote-ref" id="fnref:8" href="#fn:8">8</a>

<h2 id="chinese-character-word-segmented-writing">Chinese character word-segmented writing</h2>
Chinese is usually written in Chinese characters, so Chinese word segmented writing mainly refers to the segmentation of Chinese character text. The following are some methods or skills.

<h3>Textual context</h3>
The most important purpose of word-segmented writing is to express the intended meaning of the writer accurately and clearly.
For example, the traditional non-word-segmented text "乒乓球拍卖完了。" has two possible meanings, which can be expressed in word-segmented writing as "乒乓 球拍 卖完了。" (Ping pong bats are sold out) and "乒乓球 拍卖 完了。" (The ping pong balls have been auctioned). The author is to make a selection to correctly express the intended meaning without ambiguity.<a class="footnote-ref" id="fnref:9" href="#fn:9">9</a>

<h3>Dictionaries</h3>
If not sure whether a character string is a legal word, the writer can check its existence in a reliable word dictionary, such as <a href="/facts/Xiandai_Hanyu_Cidian/h68uLEHW">Xiandai Hanyu Cidian</a><a class="footnote-ref" id="fnref:10" href="#fn:10">10</a> and <a href="/facts/CEDICT/XQDeeR8v">CEDICT</a>. Or check whether it is a linguistically qualified word according to lexical, morphological and syntactical knowledge.<a class="footnote-ref" id="fnref:11" href="#fn:11">11</a>

<h3>Prosody</h3>
In spoken language, there is usually a pause between two words (and pause is not allowed within a word), so it is natural to put a pause (represented by a space) between the words in written language.
Methods to identify word boundaries can also be found in <a href="/facts/Word/sD4KyoLF">Word#Word boundaries</a>.

<h3>Whitespace</h3>
The space between two words should be set at half the width of a Chinese character, shorter than the distance between two lines. Because the average length of a Chinese word is about 2 characters, if a space is of full width of a Chinese character, longer than the inter-line distance, the lines of words will appear scattered, not compact. 
<a class="footnote-ref" id="fnref:12" href="#fn:12">12</a>

<h3>Proper noun marker</h3>
To further help the reader, the proper nouns should be marked as well, such as by underlines.<a class="footnote-ref" id="fnref:13" href="#fn:13">13</a> In fact this is already done in the Holy Bible (Union Version with modern punctuation).<a class="footnote-ref" id="fnref:14" href="#fn:14">14</a>

<h2 id="pinyin-segmentation">Pinyin segmentation</h2>
<a href="/facts/Pinyin/pUwjfMXg">Pinyin</a> is usually used to mark the pronunciation of Chinese characters, but in elementary Chinese teaching or teaching Chinese as a foreign language, Pinyin is sometimes used to express Chinese directly. Therefore, Pinyin writing is also a kind of Chinese writing, and it can also be an important reference for Chinese character word segmentation.<a class="footnote-ref" id="fnref:15" href="#fn:15">15</a> "Basic Rules of Chinese Pinyin Orthography" is the Chinese national standard for Pinyin writing and word segmentation. Its main content "5. General rules" is excerpted as follows:<a class="footnote-ref" id="fnref:16" href="#fn:16">16</a>

<h3>Pinyin orthography</h3>
The general rules are<a class="footnote-ref" id="fnref:17" href="#fn:17">17</a>

<ol><li>Use words as the basic writing units for Pinyin expressions. For example: rén (人, person), pǎo (跑, run), māmɑ (妈妈, mother), yuèdú (阅读, read), túshūɡuǎn (图书馆, library).</li>
<li>A two-syllable and three-syllable expression of a concept is written consecutively (without spaces). For example: huánbǎo (环保, environmental protection), ɡōnɡɡuān (公关, public relations), chánɡyònɡcí (常用词, commonly-used words), duìbuqǐ (对不起, sorry).</li>
<li>Names with four or more syllables that represent a concept are written-segmentedly by words or syllables (segments divided by speech pauses inside the phrase). Those that cannot be divided into words or syllables are written consecutively. For example: wúfènɡ ɡānɡɡuǎn (无缝钢管, seamless steel pipe), huánjìnɡ bǎohù guīhuà (环境保护规划, environmental protection planning), Zhōnɡɡuó Shèhuì Kēxuéyuàn (中国社会科学院, Chinese Academy of Social Sciences), yánjiūshēnɡyuàn (研究生院, graduate school), hónɡshízìhuì (红十字会, Red Cross Society)</li>
<li>Single-syllable repeating words are to be written consecutively; double-syllable repeating words are written separately. For example: rénrén (人人, everyone), kànkɑn (看看, look), hónɡhónɡ de (红红的, very red), yánjiū yánjiū (研究研究, research research), xuěbái xuěbái (雪白雪白, snow white snow white). Repeating words in AABB structure are written consecutively. For example: láiláiwǎnɡwǎnɡ (来来往往, coming and going), qīnɡqīnɡchǔchǔ (清清楚楚, crystal clear), fānɡfānɡmiànmiàn (方方面面, all aspects).</li>
<li>Monosyllabic prefixes (副 vice, 总 general/chief, 非 non, 反 anti, 超 super, 老 old, 阿 A, 可 able, 无 non, 半 semi, etc.) or monosyllable suffixes (子 zi, 儿 er, 头 man, 性 -ity, 者 person, 员 member, 家 expert, 手 specialist, 化 -ize, 们 plural, etc.) are written consecutively with the main word. For example: fùbùzhǎnɡ (副部长, vice minister), zǒnɡɡōnɡchénɡshī (总工程师, chief engineer), fùzǒnɡɡōnɡchénɡshī (副总工程师, vice chief engineer), fēijīnshǔ (非金属, non-metallic), kēxuéxìnɡ (科学性, scientific / scientificity), chénɡwùyuán (乘务员, flight attendant), xiàndàihuà (现代化, modernization), háizimen (孩子们, children).</li>
<li>For the convenience of reading and understanding, a hyphen can be used between some parallel words or morphemes, or in some abbreviations. For example: bā-jiǔ tiān (八九天, eight or nine days), rén-jī duìhuà (人机对话, human-computer dialogue), Jīnɡ-Zànɡ Gāosù Gōnɡlù (京藏高速公路, Beijing-Tibet Expressway).</li></ol>
In addition to the general rules, there are specific rules for nouns, verbs, adjectives, pronouns, numerals, quantifiers, adverbs, prepositions, conjunctions, auxiliary words, interjections, onomatopoeias, idioms, sayings, as well as names of people and places.

<h3>Example</h3>
Below is an example with a longer text from the Chinese version of the United Nations Universal Declaration of Human Rights:<a class="footnote-ref" id="fnref:18" href="#fn:18">18</a>
Article 1 of the Universal Declaration of Human Rights in <a href="/facts/Simplified_Chinese_characters/UcAD6xXs">simplified Chinese characters</a>:<a class="footnote-ref" id="fnref:19" href="#fn:19">19</a>

人人生而自由，在尊严和权利上一律平等。他们赋有理性和良心，并应以兄弟关系的精神相对待。
The pinyin transcription can be word-segmented into
Rénrén shēng ér zìyóu, zài zūnyán hé quánlì shàng yīlǜ píngděng. Tāmen fùyǒu lǐxìng hé liángxīn, bìng yīng yǐ xiōngdì guānxì de jīngshén xiāng duìdài.
Accordingly, the Chinese character text can be segmented into
人人 生 而 自由，在 尊严 和 权利 上 一律 平等。 他们 赋有 理性 和 良心， 并 应 以 兄弟 关系 的 精神 相 对待。

<h2 id="computer-based-word-segmentation">Computer-based word segmentation</h2>
Before word-segmented writing is popularized, computer-based word segmentation is often used for language information processing. The quality is getting better and better. But it still needs post-editing by human beings. And it will never be as reliable as word segmentation by the author personally.<a class="footnote-ref" id="fnref:20" href="#fn:20">20</a><a class="footnote-ref" id="fnref:21" href="#fn:21">21</a>

<h2 id="references">References</h2>

<ol>
<li id="fn:1">Chen, Liwei (陈力为) (1996). 汉语书面语的分词问题- - 一个有关全民的信息化问题 [Written Chinese Word Segmentation: An issue relevant to national information technology]. 中文信息学报 中文信息学报 [Journal of Chinese Information Processing] (in Simplified Chinese). 10 (1): 11–13. <a href="汉语书面语的分词问题- - 一个有关全民的信息化问题中文信息学报" target="_blank">汉语书面语的分词问题- - 一个有关全民的信息化问题中文信息学报</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></li>
<li id="fn:2">Zhang, Xiaoheng (张小衡) (1998). 也谈汉语书面语的分词问题——分词连写十大好处 [Written Chinese Word-Segmentation Revisited: Ten advantages of word-segmented writing]. 中文信息学报 中文信息学报 [Journal of Chinese Information Processing] (in Simplified Chinese). 12 (3): 57–63. <a href="也谈汉语书面语的分词问题——分词连写十大好处中文信息学报" target="_blank">也谈汉语书面语的分词问题——分词连写十大好处中文信息学报</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></li>
<li id="fn:3">Chen 1996, p. 12. - Chen, Liwei (陈力为) (1996). 汉语书面语的分词问题- - 一个有关全民的信息化问题 [Written Chinese Word Segmentation: An issue relevant to national information technology]. 中文信息学报 中文信息学报 [Journal of Chinese Information Processing] (in Simplified Chinese). 10 (1): 11–13. <a href="http://jcip.cipsc.org.cn/CN/Y1996/V10/I1/11" target="_blank">http://jcip.cipsc.org.cn/CN/Y1996/V10/I1/11</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></li>
<li id="fn:4">Chen 1996, p. 12. - Chen, Liwei (陈力为) (1996). 汉语书面语的分词问题- - 一个有关全民的信息化问题 [Written Chinese Word Segmentation: An issue relevant to national information technology]. 中文信息学报 中文信息学报 [Journal of Chinese Information Processing] (in Simplified Chinese). 10 (1): 11–13. <a href="http://jcip.cipsc.org.cn/CN/Y1996/V10/I1/11" target="_blank">http://jcip.cipsc.org.cn/CN/Y1996/V10/I1/11</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></li>
<li id="fn:5">Chen, Liwei (陈力为) (1987). 当前中文信息处理 中的几个问题及其发展前景 [Some issues in Chinese information processing and their perspective development]. 计算机世界 [Chinese Computer World] (in Chinese). 21 (34). <a href="当前中文信息处理 中的几个问题及其发展前景计算机世界" target="_blank">当前中文信息处理 中的几个问题及其发展前景计算机世界</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></li>
<li id="fn:6">Zhang 1998, pp. 57–63. - Zhang, Xiaoheng (张小衡) (1998). 也谈汉语书面语的分词问题——分词连写十大好处 [Written Chinese Word-Segmentation Revisited: Ten advantages of word-segmented writing]. 中文信息学报 中文信息学报 [Journal of Chinese Information Processing] (in Simplified Chinese). 12 (3): 57–63. <a href="http://jcip.cipsc.org.cn/CN/Y1998/V12/I3/58" target="_blank">http://jcip.cipsc.org.cn/CN/Y1998/V12/I3/58</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></li>
<li id="fn:7">"English-Chinese/Word segmentation of Hanzi - Wikiversity". <a href="https://en.wikiversity.org/wiki/English-Chinese/Word_segmentation_of_Hanzi" target="_blank">https://en.wikiversity.org/wiki/English-Chinese/Word_segmentation_of_Hanzi</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></li>
<li id="fn:8">Peng, Zerun (彭泽润、李葆嘉 eds) (2000). 语言理论 [Language theories] (in Chinese). Changsha: 中南大学出版社 (Central South University Press). ISBN 978-7-810-61342-2. <a href="语言理论978-7-810-61342-2" target="_blank">语言理论978-7-810-61342-2</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></li>
<li id="fn:9">Chen 1996, p. 12. - Chen, Liwei (陈力为) (1996). 汉语书面语的分词问题- - 一个有关全民的信息化问题 [Written Chinese Word Segmentation: An issue relevant to national information technology]. 中文信息学报 中文信息学报 [Journal of Chinese Information Processing] (in Simplified Chinese). 10 (1): 11–13. <a href="http://jcip.cipsc.org.cn/CN/Y1996/V10/I1/11" target="_blank">http://jcip.cipsc.org.cn/CN/Y1996/V10/I1/11</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></li>
<li id="fn:10">"教育部《重編國語辭典修訂本》2021". <a href="https://dict.revised.moe.edu.tw/index.jsp?la=0&powerMode=0" target="_blank">https://dict.revised.moe.edu.tw/index.jsp?la=0&powerMode=0</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></li>
<li id="fn:11">Zhang 1998, p. 61. - Zhang, Xiaoheng (张小衡) (1998). 也谈汉语书面语的分词问题——分词连写十大好处 [Written Chinese Word-Segmentation Revisited: Ten advantages of word-segmented writing]. 中文信息学报 中文信息学报 [Journal of Chinese Information Processing] (in Simplified Chinese). 12 (3): 57–63. <a href="http://jcip.cipsc.org.cn/CN/Y1998/V12/I3/58" target="_blank">http://jcip.cipsc.org.cn/CN/Y1998/V12/I3/58</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></li>
<li id="fn:12">Zhang 1998, p. 62. - Zhang, Xiaoheng (张小衡) (1998). 也谈汉语书面语的分词问题——分词连写十大好处 [Written Chinese Word-Segmentation Revisited: Ten advantages of word-segmented writing]. 中文信息学报 中文信息学报 [Journal of Chinese Information Processing] (in Simplified Chinese). 12 (3): 57–63. <a href="http://jcip.cipsc.org.cn/CN/Y1998/V12/I3/58" target="_blank">http://jcip.cipsc.org.cn/CN/Y1998/V12/I3/58</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></li>
<li id="fn:13">Chen 1996, p. 12. - Chen, Liwei (陈力为) (1996). 汉语书面语的分词问题- - 一个有关全民的信息化问题 [Written Chinese Word Segmentation: An issue relevant to national information technology]. 中文信息学报 中文信息学报 [Journal of Chinese Information Processing] (in Simplified Chinese). 10 (1): 11–13. <a href="http://jcip.cipsc.org.cn/CN/Y1996/V10/I1/11" target="_blank">http://jcip.cipsc.org.cn/CN/Y1996/V10/I1/11</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></li>
<li id="fn:14">Chinese Baptist Press, Hong Kong (translation) (1998). 聖經 現代標點和合本 (Holy Bible, Union Version with modern punctuation) (in Chinese). Hong Kong: Chinese Baptist Press (浸信會出版社). ISBN 962-933-101-2. <a href="聖經 現代標點和合本 (Holy Bible, Union Version with modern punctuation)962-933-101-2" target="_blank">聖經 現代標點和合本 (Holy Bible, Union Version with modern punctuation)962-933-101-2</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></li>
<li id="fn:15">Arcodia, Giorgio Francesco; Basciano, Bianca (2021). Chinese Linguistics. Oxford: Oxford University Press. pp. 84–85. ISBN 978-0-19-884784-7. <a href="978-0-19-884784-7" target="_blank">978-0-19-884784-7</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></li>
<li id="fn:16">国家语委 (2012). 漢語拼音正字法基本規則 (PDF) (in Chinese). <a href="漢語拼音正字法基本規則" target="_blank">漢語拼音正字法基本規則</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></li>
<li id="fn:17">国家语委 2012. - 国家语委 (2012). 漢語拼音正字法基本規則 (PDF) (in Chinese). <a href="http://www.moe.gov.cn/ewebeditor/uploadfile/2015/01/13/20150113091717604.pdf" target="_blank">http://www.moe.gov.cn/ewebeditor/uploadfile/2015/01/13/20150113091717604.pdf</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></li>
<li id="fn:18">"Universal Declaration of Human Rights - Chinese, Mandarin (Simplified)". unicode.org. <a href="https://www.ohchr.org/en/human-rights/universal-declaration/translations/chinese" target="_blank">https://www.ohchr.org/en/human-rights/universal-declaration/translations/chinese</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></li>
<li id="fn:19">"Universal Declaration of Human Rights - Chinese, Mandarin (Simplified)". unicode.org. <a href="https://unicode.org/udhr/d/udhr_cmn_hans.html" target="_blank">https://unicode.org/udhr/d/udhr_cmn_hans.html</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></li>
<li id="fn:20">"Chinese Word Segmentation". <a href="http://nlpprogress.com/chinese/chinese_word_segmentation.html" target="_blank">http://nlpprogress.com/chinese/chinese_word_segmentation.html</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></li>
<li id="fn:21">Zhang 1998, p. 57. - Zhang, Xiaoheng (张小衡) (1998). 也谈汉语书面语的分词问题——分词连写十大好处 [Written Chinese Word-Segmentation Revisited: Ten advantages of word-segmented writing]. 中文信息学报 中文信息学报 [Journal of Chinese Information Processing] (in Simplified Chinese). 12 (3): 57–63. <a href="http://jcip.cipsc.org.cn/CN/Y1998/V12/I3/58" target="_blank">http://jcip.cipsc.org.cn/CN/Y1998/V12/I3/58</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></li>
</ol>

Chinese word-segmented writing open-in-new

Chinese word-segmented writing