16f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* 26f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org****************************************************************************** 36f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org* 46f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org* Copyright (C) 2000-2013, International Business Machines 56f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org* Corporation and others. All Rights Reserved. 66f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org* 76f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org****************************************************************************** 86f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org* file name: ucnvmbcs.h 96f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org* encoding: US-ASCII 106f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org* tab size: 8 (not used) 116f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org* indentation:4 126f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org* 136f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org* created on: 2000jul07 146f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org* created by: Markus W. Scherer 156f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org*/ 166f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 176f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#ifndef __UCNVMBCS_H__ 186f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define __UCNVMBCS_H__ 196f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 206f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#include "unicode/utypes.h" 216f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 226f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#if !UCONFIG_NO_CONVERSION 236f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 246f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#include "unicode/ucnv.h" 256f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#include "ucnv_cnv.h" 266f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#include "ucnv_ext.h" 276f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 286f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** 296f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * ICU conversion (.cnv) data file structure, following the usual UDataInfo 306f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * header. 316f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 326f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Format version: 6.2 336f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 346f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * struct UConverterStaticData -- struct containing the converter name, IBM CCSID, 356f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * min/max bytes per character, etc. 366f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * see ucnv_bld.h 376f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 386f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -------------------- 396f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 406f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * The static data is followed by conversionType-specific data structures. 416f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * At the moment, there are only variations of MBCS converters. They all have 426f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * the same toUnicode structures, while the fromUnicode structures for SBCS 436f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * differ from those for other MBCS-style converters. 446f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 456f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * _MBCSHeader.version 5 is optional and not backward-compatible 466f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * (as usual for changes in the major version field). 476f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 486f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Versions 5.m work like versions 4.m except: 496f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * - The _MBCSHeader has variable length (and is always longer than in version 4). 506f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * See the struct _MBCSHeader further description below. 516f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * - There is a set of flags which indicate further incompatible changes. 526f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * (Reader code must reject the file if it does not recognize them all.) 536f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * - In particular, one of these flags indicates that most of the fromUnicode 546f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * data is missing and must be reconstituted from the toUnicode data 556f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * and from the utf8Friendly mbcsIndex at load time. 566f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * (This only works with a utf8Friendly table.) 576f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * In this case, makeconv may increase maxFastUChar automatically to U+FFFF. 586f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 596f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * The first of these versions is 5.3, which is like 4.3 except for the differences above. 606f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 616f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * When possible, makeconv continues to generate version 4.m files. 626f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 636f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * _MBCSHeader.version 5.4/4.4 supports "good one-way" mappings (|4) 646f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * in the extension tables (fromUTableValues bit 30). See ucnv_ext.h for details. 656f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 666f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * _MBCSHeader.version 4.3 optionally modifies the fromUnicode data structures 676f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * slightly and optionally adds a table for conversion to MBCS (non-SBCS) 686f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * charsets. 696f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 706f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * The modifications are to make the data utf8Friendly. Not every 4.3 file 716f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * file contains utf8Friendly data. 726f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It is utf8Friendly if _MBCSHeader.version[2]!=0. 736f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * In this case, the data structures are utf8Friendly up to the code point 746f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * maxFastUChar=((_MBCSHeader.version[2]<<8)|0xff) 756f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 766f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * A utf8Friendly file has fromUnicode stage 3 entries for code points up to 776f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * maxFastUChar allocated in blocks of 64 for indexing with the 6 bits from 786f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * a UTF-8 trail byte. ASCII is allocated linearly with 128 contiguous entries. 796f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 806f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * In addition, a utf8Friendly MBCS file contains an additional 816f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint16_t mbcsIndex[(maxFastUChar+1)>>6]; 826f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * which replaces the stage 1 and 2 tables for indexing with bits from the 836f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * UTF-8 lead byte and middle trail byte. Unlike the older MBCS stage 2 table, 846f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * the mbcsIndex does not contain roundtrip flags. Therefore, all fallbacks 856f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * from code points up to maxFastUChar (and roundtrips to 0x00) are moved to 866f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * the extension data structure. This also allows for faster roundtrip 876f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * conversion from UTF-16. 886f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 896f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * SBCS files do not contain an additional sbcsIndex[] array because the 906f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * proportional size increase would be noticeable, but the runtime 916f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * code builds one for the code point range for which the runtime conversion 926f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * code is optimized. 936f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 946f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * For SBCS, maxFastUChar should be at least U+0FFF. The initial makeconv 956f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * implementation sets it to U+1FFF. Because the sbcsIndex is not stored in 966f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * the file, a larger maxFastUChar only affects stage 3 block allocation size 976f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * and is free in empty blocks. (Larger blocks with sparse contents cause larger 986f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * files.) U+1FFF includes almost all of the small scripts. 996f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * U+0FFF covers UTF-8 two-byte sequences and three-byte sequences starting with 1006f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 0xe0. This includes most scripts with legacy SBCS charsets. 1016f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * The initial runtime implementation using 4.3 files only builds an sbcsIndex 1026f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * for code points up to U+0FFF. 1036f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1046f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * For MBCS, maxFastUChar should be at least U+D7FF (=initial value). 1056f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * This boundary is convenient because practically all of the commonly used 1066f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * characters are below it, and because it is the boundary to surrogate 1076f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * code points, above which special handling is necessary anyway. 1086f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * (Surrogate pair assembly for UTF-16, validity checking for UTF-8.) 1096f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1106f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * maxFastUChar could be up to U+FFFF to cover the whole BMP, which could be 1116f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * useful especially for conversion from UTF-8 when the input can be assumed 1126f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * to be valid, because the surrogate range would then not have to be 1136f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * checked. 1146f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * (With maxFastUChar=0xffff, makeconv would have to check for mbcsIndex value 1156f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * overflow because with the all-unassigned block 0 and nearly full mappings 1166f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * from the BMP it is theoretically possible that an index into stage 3 1176f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * exceeds 16 bits.) 1186f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1196f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * _MBCSHeader.version 4.2 adds an optional conversion extension data structure. 1206f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * If it is present, then an ICU version reading header versions 4.0 or 4.1 1216f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * will be able to use the base table and ignore the extension. 1226f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1236f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * The unicodeMask in the static data is part of the base table data structure. 1246f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Especially, the UCNV_HAS_SUPPLEMENTARY flag determines the length of the 1256f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * fromUnicode stage 1 array. 1266f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * The static data unicodeMask refers only to the base table's properties if 1276f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * a base table is included. 1286f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * In an extension-only file, the static data unicodeMask is 0. 1296f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * The extension data indexes have a separate field with the unicodeMask flags. 1306f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1316f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * MBCS-style data structure following the static data. 1326f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Offsets are counted in bytes from the beginning of the MBCS header structure. 1336f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Details about usage in comments in ucnvmbcs.c. 1346f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1356f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * struct _MBCSHeader (see the definition in this header file below) 1366f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * contains 32-bit fields as follows: 1376f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 8 values: 1386f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 0 uint8_t[4] MBCS version in UVersionInfo format (currently 4.3.x.0) 1396f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1 uint32_t countStates 1406f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 2 uint32_t countToUFallbacks 1416f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 3 uint32_t offsetToUCodeUnits 1426f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 4 uint32_t offsetFromUTable 1436f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 5 uint32_t offsetFromUBytes 1446f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 6 uint32_t flags, bits: 1456f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 31.. 8 offsetExtension -- _MBCSHeader.version 4.2 (ICU 2.8) and higher 1466f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 0 for older versions and if 1476f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * there is not extension structure 1486f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 7.. 0 outputType 1496f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 7 uint32_t fromUBytesLength -- _MBCSHeader.version 4.1 (ICU 2.4) and higher 1506f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * counts bytes in fromUBytes[] 1516f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1526f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * New and required in version 5: 1536f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 8 uint32_t options, bits: 1546f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 31..16 reserved for flags that can be added without breaking 1556f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * backward compatibility 1566f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 15.. 6 reserved for flags whose addition will break 1576f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * backward compatibility 1586f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 6 MBCS_OPT_FROM_U -- if set, 1596f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * then most of the fromUnicode data is omitted; 1606f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * fullStage2Length is present and the missing 1616f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * bottom part of stage 2 must be reconstituted from 1626f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * the toUnicode data; 1636f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * stage 3 is missing completely as well; 1646f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * not used for SBCS tables 1656f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 5.. 0 length of the _MBCSHeader (number of uint32_t) 1666f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1676f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * New and optional in version 5: 1686f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 9 uint32_t fullStage2Length: used if MBCS_OPT_FROM_U is set 1696f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * specifies the full length of stage 2 1706f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * including the omitted part 1716f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1726f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * if(outputType==MBCS_OUTPUT_EXT_ONLY) { 1736f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- base table name for extension-only table 1746f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * char baseTableName[variable]; -- with NUL plus padding for 4-alignment 1756f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1766f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- all _MBCSHeader fields except for version and flags are 0 1776f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } else { 1786f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- normal base table with optional extension 1796f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1806f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * int32_t stateTable[countStates][256]; 1816f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1826f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * struct _MBCSToUFallback { (fallbacks are sorted by offset) 1836f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint32_t offset; 1846f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * UChar32 codePoint; 1856f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } toUFallbacks[countToUFallbacks]; 1866f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1876f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint16_t unicodeCodeUnits[(offsetFromUTable-offsetToUCodeUnits)/2]; 1886f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * (padded to an even number of units) 1896f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1906f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- stage 1 tables 1916f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * if(staticData.unicodeMask&UCNV_HAS_SUPPLEMENTARY) { 1926f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- stage 1 table for all of Unicode 1936f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint16_t fromUTable[0x440]; (32-bit-aligned) 1946f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } else { 1956f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- BMP-only tables have a smaller stage 1 table 1966f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint16_t fromUTable[0x40]; (32-bit-aligned) 1976f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } 1986f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1996f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- stage 2 tables 2006f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * length determined by top of stage 1 and bottom of stage 3 tables 2016f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * if(outputType==MBCS_OUTPUT_1) { 2026f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- SBCS: pure indexes 2036f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint16_t stage 2 indexes[?]; 2046f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } else { 2056f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- DBCS, MBCS, EBCDIC_STATEFUL, ...: roundtrip flags and indexes 2066f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint32_t stage 2 flags and indexes[?]; 2076f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * if(options&MBCS_OPT_NO_FROM_U) { 2086f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * stage 2 really has length fullStage2Length 2096f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * and the omitted lower part must be reconstituted from 2106f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * the toUnicode data 2116f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } 2126f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } 2136f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 2146f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- stage 3 tables with byte results 2156f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * if(outputType==MBCS_OUTPUT_1) { 2166f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- SBCS: each 16-bit result contains flags and the result byte, see ucnvmbcs.c 2176f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint16_t fromUBytes[fromUBytesLength/2]; 2186f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } else if(!(options&MBCS_OPT_NO_FROM_U)) { 2196f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- DBCS, MBCS, EBCDIC_STATEFUL, ... 2/3/4 bytes result, see ucnvmbcs.c 2206f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint8_t fromUBytes[fromUBytesLength]; or 2216f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint16_t fromUBytes[fromUBytesLength/2]; or 2226f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint32_t fromUBytes[fromUBytesLength/4]; 2236f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } else { 2246f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * fromUBytes[] must be reconstituted from the toUnicode data 2256f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } 2266f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 2276f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- optional utf8Friendly mbcsIndex -- _MBCSHeader.version 4.3 (ICU 3.8) and higher 2286f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * if(outputType!=MBCS_OUTPUT_1 && 2296f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * _MBCSHeader.version[1]>=3 && 2306f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * (maxFastUChar=_MBCSHeader.version[2])!=0 2316f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * ) { 2326f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * maxFastUChar=(maxFastUChar<<8)|0xff; 2336f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * uint16_t mbcsIndex[(maxFastUChar+1)>>6]; 2346f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } 2356f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * } 2366f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 2376f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -- extension table, details see ucnv_ext.h 2386f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * int32_t indexes[>=32]; ... 2396f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 2406f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2416f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* MBCS converter data and state -------------------------------------------- */ 2426f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2436f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgenum { 2446f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_MAX_STATE_COUNT=128 2456f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org}; 2466f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2476f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** 2486f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * MBCS action codes for conversions to Unicode. 2496f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * These values are in bits 23..20 of the state table entries. 2506f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 2516f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgenum { 2526f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_STATE_VALID_DIRECT_16, 2536f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_STATE_VALID_DIRECT_20, 2546f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2556f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_STATE_FALLBACK_DIRECT_16, 2566f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_STATE_FALLBACK_DIRECT_20, 2576f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2586f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_STATE_VALID_16, 2596f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_STATE_VALID_16_PAIR, 2606f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2616f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_STATE_UNASSIGNED, 2626f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_STATE_ILLEGAL, 2636f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2646f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_STATE_CHANGE_ONLY 2656f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org}; 2666f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2676f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* Macros for state table entries */ 2686f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_TRANSITION(state, offset) (int32_t)(((int32_t)(state)<<24L)|(offset)) 2696f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_TRANSITION_SET_OFFSET(entry, offset) (int32_t)(((entry)&0xff000000)|(offset)) 2706f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_TRANSITION_ADD_OFFSET(entry, offset) (int32_t)((entry)+(offset)) 2716f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2726f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_FINAL(state, action, value) (int32_t)(0x80000000|((int32_t)(state)<<24L)|((action)<<20L)|(value)) 2736f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_SET_FINAL(entry) (int32_t)((entry)|0x80000000) 2746f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_FINAL_SET_ACTION(entry, action) (int32_t)(((entry)&0xff0fffff)|((int32_t)(action)<<20L)) 2756f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_FINAL_SET_VALUE(entry, value) (int32_t)(((entry)&0xfff00000)|(value)) 2766f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_FINAL_SET_ACTION_VALUE(entry, action, value) (int32_t)(((entry)&0xff000000)|((int32_t)(action)<<20L)|(value)) 2776f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2786f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_SET_STATE(entry, state) (int32_t)(((entry)&0x80ffffff)|((int32_t)(state)<<24L)) 2796f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2806f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_STATE(entry) ((((uint32_t)entry)>>24)&0x7f) 2816f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2826f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_IS_TRANSITION(entry) ((entry)>=0) 2836f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_IS_FINAL(entry) ((entry)<0) 2846f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2856f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_TRANSITION_STATE(entry) (((uint32_t)entry)>>24) 2866f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_TRANSITION_OFFSET(entry) ((entry)&0xffffff) 2876f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2886f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_FINAL_STATE(entry) ((((uint32_t)entry)>>24)&0x7f) 2896f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_FINAL_IS_VALID_DIRECT_16(entry) ((entry)<(int32_t)0x80100000) 2906f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_FINAL_ACTION(entry) ((((uint32_t)entry)>>20)&0xf) 2916f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_FINAL_VALUE(entry) ((entry)&0xfffff) 2926f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_ENTRY_FINAL_VALUE_16(entry) (uint16_t)(entry) 2936f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2946f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define IS_ASCII_ROUNDTRIP(b, asciiRoundtrips) (((asciiRoundtrips) & (1<<((b)>>2)))!=0) 2956f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2966f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* single-byte fromUnicode: get the 16-bit result word */ 2976f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_SINGLE_RESULT_FROM_U(table, results, c) (results)[ (table)[ (table)[(c)>>10] +(((c)>>4)&0x3f) ] +((c)&0xf) ] 2986f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 2996f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* single-byte fromUnicode using the sbcsIndex */ 3006f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define SBCS_RESULT_FROM_LOW_BMP(table, results, c) (results)[ (table)[(c)>>6] +((c)&0x3f) ] 3016f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3026f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* single-byte fromUTF8 using the sbcsIndex; l and t must be masked externally; can be l=0 and t<=0x7f */ 3036f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define SBCS_RESULT_FROM_UTF8(table, results, l, t) (results)[ (table)[l] +(t) ] 3046f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3056f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* multi-byte fromUnicode: get the 32-bit stage 2 entry */ 3066f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_STAGE_2_FROM_U(table, c) ((const uint32_t *)(table))[ (table)[(c)>>10] +(((c)>>4)&0x3f) ] 3076f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_FROM_U_IS_ROUNDTRIP(stage2Entry, c) ( ((stage2Entry) & ((uint32_t)1<< (16+((c)&0xf)) )) !=0) 3086f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3096f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_VALUE_2_FROM_STAGE_2(bytes, stage2Entry, c) ((uint16_t *)(bytes))[16*(uint32_t)(uint16_t)(stage2Entry)+((c)&0xf)] 3106f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_VALUE_4_FROM_STAGE_2(bytes, stage2Entry, c) ((uint32_t *)(bytes))[16*(uint32_t)(uint16_t)(stage2Entry)+((c)&0xf)] 3116f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3126f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define MBCS_POINTER_3_FROM_STAGE_2(bytes, stage2Entry, c) ((bytes)+(16*(uint32_t)(uint16_t)(stage2Entry)+((c)&0xf))*3) 3136f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3146f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* double-byte fromUnicode using the mbcsIndex */ 3156f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define DBCS_RESULT_FROM_MOST_BMP(table, results, c) (results)[ (table)[(c)>>6] +((c)&0x3f) ] 3166f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3176f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* double-byte fromUTF8 using the mbcsIndex; l and t1 combined into lt1; lt1 and t2 must be masked externally */ 3186f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define DBCS_RESULT_FROM_UTF8(table, results, lt1, t2) (results)[ (table)[lt1] +(t2) ] 3196f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3206f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3216f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** 3226f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * MBCS output types for conversions from Unicode. 3236f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * These per-converter types determine the storage method in stage 3 of the lookup table, 3246f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * mostly how many bytes are stored per entry. 3256f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 3266f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgenum { 3276f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OUTPUT_1, /* 0 */ 3286f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OUTPUT_2, /* 1 */ 3296f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OUTPUT_3, /* 2 */ 3306f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OUTPUT_4, /* 3 */ 3316f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3326f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OUTPUT_3_EUC=8, /* 8 */ 3336f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OUTPUT_4_EUC, /* 9 */ 3346f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3356f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OUTPUT_2_SISO=12, /* c */ 3366f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OUTPUT_2_HZ, /* d */ 3376f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3386f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OUTPUT_EXT_ONLY, /* e */ 3396f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3406f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OUTPUT_COUNT, 3416f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3426f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OUTPUT_DBCS_ONLY=0xdb /* runtime-only type for DBCS-only handling of SISO tables */ 3436f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org}; 3446f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3456f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** 3466f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Fallbacks to Unicode are stored outside the normal state table and code point structures 3476f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * in a vector of items of this type. They are sorted by offset. 3486f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 3496f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgtypedef struct { 3506f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint32_t offset; 3516f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UChar32 codePoint; 3526f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org} _MBCSToUFallback; 3536f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3546f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** Constants for fast and UTF-8-friendly conversion. */ 3556f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgenum { 3566f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org SBCS_FAST_MAX=0x0fff, /* maximum code point with UTF-8-friendly SBCS runtime code, see makeconv SBCS_UTF8_MAX */ 3576f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org SBCS_FAST_LIMIT=SBCS_FAST_MAX+1, /* =0x1000 */ 3586f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_FAST_MAX=0xd7ff, /* maximum code point with UTF-8-friendly MBCS runtime code, see makeconv MBCS_UTF8_MAX */ 3596f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_FAST_LIMIT=MBCS_FAST_MAX+1 /* =0xd800 */ 3606f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org}; 3616f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3626f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** 3636f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * This is the MBCS part of the UConverterTable union (a runtime data structure). 3646f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It keeps all the per-converter data and points into the loaded mapping tables. 3656f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 3666f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * utf8Friendly data structures added with _MBCSHeader.version 4.3 3676f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 3686f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgtypedef struct UConverterMBCSTable { 3696f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* toUnicode */ 3706f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint8_t countStates, dbcsOnlyState, stateTableOwned; 3716f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint32_t countToUFallbacks; 3726f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3736f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org const int32_t (*stateTable)/*[countStates]*/[256]; 3746f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org int32_t (*swapLFNLStateTable)/*[countStates]*/[256]; /* for swaplfnl */ 3756f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org const uint16_t *unicodeCodeUnits/*[countUnicodeResults]*/; 3766f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org const _MBCSToUFallback *toUFallbacks; 3776f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3786f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* fromUnicode */ 3796f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org const uint16_t *fromUnicodeTable; 3806f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org const uint16_t *mbcsIndex; /* for fast conversion from most of BMP to MBCS (utf8Friendly data) */ 3816f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint16_t sbcsIndex[SBCS_FAST_LIMIT>>6]; /* for fast conversion from low BMP to SBCS (utf8Friendly data) */ 3826f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org const uint8_t *fromUnicodeBytes; 3836f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint8_t *swapLFNLFromUnicodeBytes; /* for swaplfnl */ 3846f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint32_t fromUBytesLength; 3856f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint8_t outputType, unicodeMask; 3866f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UBool utf8Friendly; /* for utf8Friendly data */ 3876f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UChar maxFastUChar; /* for utf8Friendly data */ 3886f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3896f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* roundtrips */ 3906f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint32_t asciiRoundtrips; 3916f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3926f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* reconstituted data that was omitted from the .cnv file */ 3936f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint8_t *reconstitutedData; 3946f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3956f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* converter name for swaplfnl */ 3966f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org char *swapLFNLName; 3976f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 3986f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* extension data */ 3996f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org struct UConverterSharedData *baseSharedData; 4006f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org const int32_t *extIndexes; 4016f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org} UConverterMBCSTable; 4026f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 4036f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define UCNV_MBCS_TABLE_INITIALIZER { \ 4046f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* toUnicode */ \ 4056f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 0, 0, 0, \ 4066f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 0, \ 4076f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org \ 4086f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL, \ 4096f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL, \ 4106f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL, \ 4116f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL, \ 4126f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org \ 4136f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* fromUnicode */ \ 4146f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL, \ 4156f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL, \ 4166f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org { 0 }, \ 4176f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL, \ 4186f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL, \ 4196f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 0, \ 4206f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 0, 0, \ 4216f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org FALSE, \ 4226f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 0, \ 4236f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org \ 4246f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* roundtrips */ \ 4256f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 0, \ 4266f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org \ 4276f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* reconstituted data that was omitted from the .cnv file */ \ 4286f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL, \ 4296f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org \ 4306f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* converter name for swaplfnl */ \ 4316f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL, \ 4326f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org \ 4336f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* extension data */ \ 4346f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL, \ 4356f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org NULL \ 4366f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org} 4376f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 4386f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgenum { 4396f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OPT_LENGTH_MASK=0x3f, 4406f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OPT_NO_FROM_U=0x40, 4416f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* 4426f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * If any of the following options bits are set, 4436f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * then the file must be rejected. 4446f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 4456f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OPT_INCOMPATIBLE_MASK=0xffc0, 4466f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* 4476f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Remove bits from this mask as more options are recognized 4486f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * by all implementations that use this constant. 4496f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 4506f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_OPT_UNKNOWN_INCOMPATIBLE_MASK=0xff80 4516f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org}; 4526f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 4536f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgenum { 4546f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_HEADER_V4_LENGTH=8, 4556f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org MBCS_HEADER_V5_MIN_LENGTH=9 4566f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org}; 4576f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 4586f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** 4596f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * MBCS data header. See data format description above. 4606f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 4616f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgtypedef struct { 4626f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UVersionInfo version; 4636f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint32_t countStates, 4646f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org countToUFallbacks, 4656f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org offsetToUCodeUnits, 4666f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org offsetFromUTable, 4676f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org offsetFromUBytes, 4686f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org flags, 4696f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org fromUBytesLength; 4706f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 4716f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* new and required in version 5 */ 4726f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint32_t options; 4736f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 4746f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org /* new and optional in version 5; used if options&MBCS_OPT_NO_FROM_U */ 4756f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint32_t fullStage2Length; /* number of 32-bit units */ 4766f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org} _MBCSHeader; 4776f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 4786f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define UCNV_MBCS_HEADER_INITIALIZER { { 0 }, 0, 0, 0, 0, 0, 0, 0, 0, 0 } 4796f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 4806f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* 4816f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * This is a simple version of _MBCSGetNextUChar() that is used 4826f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * by other converter implementations. 4836f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It only returns an "assigned" result if it consumes the entire input. 4846f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It does not use state from the converter, nor error codes. 4856f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It does not handle the EBCDIC swaplfnl option (set in UConverter). 4866f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It handles conversion extensions but not GB 18030. 4876f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 4886f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Return value: 4896f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * U+fffe unassigned 4906f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * U+ffff illegal 4916f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * otherwise the Unicode code point 4926f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 4936f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgU_CFUNC UChar32 4946f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgucnv_MBCSSimpleGetNextUChar(UConverterSharedData *sharedData, 4956f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org const char *source, int32_t length, 4966f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UBool useFallback); 4976f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 4986f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** 4996f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * This version of _MBCSSimpleGetNextUChar() is optimized for single-byte, single-state codepages. 5006f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It does not handle the EBCDIC swaplfnl option (set in UConverter). 5016f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It does not handle conversion extensions (_extToU()). 5026f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 5036f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgU_CFUNC UChar32 5046f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgucnv_MBCSSingleSimpleGetNextUChar(UConverterSharedData *sharedData, 5056f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org uint8_t b, UBool useFallback); 5066f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 5076f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** 5086f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * This macro version of _MBCSSingleSimpleGetNextUChar() gets a code point from a byte. 5096f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It works for single-byte, single-state codepages that only map 5106f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * to and from BMP code points, and it always 5116f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * returns fallback values. 5126f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 5136f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define _MBCS_SINGLE_SIMPLE_GET_NEXT_BMP(sharedData, b) \ 5146f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org (UChar)MBCS_ENTRY_FINAL_VALUE_16((sharedData)->mbcs.stateTable[0][(uint8_t)(b)]) 5156f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 5166f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** 5176f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * This is an internal function that allows other converter implementations 5186f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * to check whether a byte is a lead byte. 5196f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 5206f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgU_CFUNC UBool 5216f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgucnv_MBCSIsLeadByte(UConverterSharedData *sharedData, char byte); 5226f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 5236f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** This is a macro version of _MBCSIsLeadByte(). */ 5246f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#define _MBCS_IS_LEAD_BYTE(sharedData, byte) \ 5256f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org (UBool)MBCS_ENTRY_IS_TRANSITION((sharedData)->mbcs.stateTable[0][(uint8_t)(byte)]) 5266f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 5276f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* 5286f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * This is another simple conversion function for internal use by other 5296f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * conversion implementations. 5306f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It does not use the converter state nor call callbacks. 5316f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It does not handle the EBCDIC swaplfnl option (set in UConverter). 5326f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It handles conversion extensions but not GB 18030. 5336f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 5346f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It converts one single Unicode code point into codepage bytes, encoded 5356f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * as one 32-bit value. The function returns the number of bytes in *pValue: 5366f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 1..4 the number of bytes in *pValue 5376f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 0 unassigned (*pValue undefined) 5386f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * -1 illegal (currently not used, *pValue undefined) 5396f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 5406f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * *pValue will contain the resulting bytes with the last byte in bits 7..0, 5416f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * the second to last byte in bits 15..8, etc. 5426f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Currently, the function assumes but does not check that 0<=c<=0x10ffff. 5436f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 5446f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgU_CFUNC int32_t 5456f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgucnv_MBCSFromUChar32(UConverterSharedData *sharedData, 5466f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UChar32 c, uint32_t *pValue, 5476f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UBool useFallback); 5486f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 5496f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** 5506f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * This version of _MBCSFromUChar32() is optimized for single-byte codepages. 5516f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It does not handle the EBCDIC swaplfnl option (set in UConverter). 5526f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * 5536f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * It returns the codepage byte for the code point, or -1 if it is unassigned. 5546f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 5556f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgU_CFUNC int32_t 5566f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgucnv_MBCSSingleFromUChar32(UConverterSharedData *sharedData, 5576f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UChar32 c, 5586f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UBool useFallback); 5596f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 5606f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/** 5616f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * SBCS, DBCS, and EBCDIC_STATEFUL are replaced by MBCS, but 5626f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * we cheat a little about the type, returning the old types if appropriate. 5636f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 5646f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgU_CFUNC UConverterType 5656f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgucnv_MBCSGetType(const UConverter* converter); 5666f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 5676f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgU_CFUNC void 5686f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgucnv_MBCSFromUnicodeWithOffsets(UConverterFromUnicodeArgs *pArgs, 5696f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UErrorCode *pErrorCode); 5706f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgU_CFUNC void 5716f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgucnv_MBCSToUnicodeWithOffsets(UConverterToUnicodeArgs *pArgs, 5726f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UErrorCode *pErrorCode); 5736f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 5746f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* 5756f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Internal function returning a UnicodeSet for toUnicode() conversion. 5766f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Currently only used for ISO-2022-CN, and only handles roundtrip mappings. 5776f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * In the future, if we add support for fallback sets, this function 5786f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * needs to be updated. 5796f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Handles extensions. 5806f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Does not empty the set first. 5816f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 5826f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgU_CFUNC void 5836f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgucnv_MBCSGetUnicodeSetForUnicode(const UConverterSharedData *sharedData, 5846f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org const USetAdder *sa, 5856f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UConverterUnicodeSet which, 5866f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UErrorCode *pErrorCode); 5876f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 5886f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org/* 5896f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Same as ucnv_MBCSGetUnicodeSetForUnicode() but 5906f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * the set can be filtered by encoding scheme. 5916f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * Used by stateful converters which share regular conversion tables 5926f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org * but only use a subset of their mappings. 5936f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org */ 5946f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgU_CFUNC void 5956f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.orgucnv_MBCSGetFilteredUnicodeSetForUnicode(const UConverterSharedData *sharedData, 5966f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org const USetAdder *sa, 5976f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UConverterUnicodeSet which, 5986f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UConverterSetFilter filter, 5996f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org UErrorCode *pErrorCode); 6006f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 6016f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#endif 6026f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org 6036f31ac30b9092fd02a8c97e5216cf53f3e4fae4jshin@chromium.org#endif 604