maxdz Software GmbH

mdz_unicode Overview and Reference

Summary: Basically, conversion of Unicode strings between ANSI, UTF8, UTF16-LE, UTF16-BE, UTF32-LE and UTF32-BE formats are possible. This all with ANSI C89/90 conforming code, without any further dependencies!

mdz_unicode Advantages

1. High portability: the whole code conforms to ANSI C 89/90 Standard. Multithreading/asynchronous part is POSIX compatible (under UNIX/Linux).

2. Little dependencies: basically, mdz_unicode functions are only dependent on standard C-library memory-management/access functions. Multithreading part is dependent on POSIX pthreads API (under UNIX/Linux) and old process control/synchronization API (from Windows 2000). It means you can use library in your code without any further dependencies except standard platform libraries/APIs.

3. Extended error-checking: all functions preserve internal error-code pointing the problem. It is possible to use strict error-checking (when all preserved error-codes should be MDZ_ERROR_NONE) or "relaxed"-checking - when only returned mdz_false will indicate error.

4. Extended control: string do only explicit operations. It means for example, when "insert" function is called - it will return error if there is not enough capacity in string. No implicit reservations will be made.

5. Attached usage: string should not necessarily use dynamically-allocated memory - which may be not available on your embedded system (or if malloc()/free() are forbidden to use in you safety-critical software). Just attach string/data to your statically-allocated memory and use all string functionality.

6. Cache-friendly: it is possible to keep controlling and data parts together in memory using "embedded part".

7. Unicode support: UTF-8, UTF-16, UTF-32 are supported.

8. wchar_t support: also wchar_t strings are supported, with 2 and 4 bytes-large wchar_t characters.

9. Endianness-aware strings: utf16 and utf32 strings are endiannes-aware thus may be used to produce and manipulate strings with pre-defined endianness even if endianness of host differs.

10. Unicode "surrogate-pairs" awareness: 2-byte Unicode strings correctly process/distinguish "surrogate-pairs" as 1 Unicode symbol.

11. Asynchronous execution: insert functions can be executed asynchronously.

Please refer to mdz_unicode Wiki for API details.

mdz_wchar Reference
mdz_utf8 Reference
mdz_utf16 Reference
mdz_utf32 Reference

Asynchronous execution

- struct mdz_asyncData* pAsyncData - pointer to shared async data for asynchronous call, or NULL if call should be synchronous

Fields of struct mdz_asyncData* are following:

mdz_unicode General Information and Functions

mdz_unicode initialization functions.

mdz_unicode_init

Initializes unicode library. This function should be called before any other function of the library.

mdz_bool mdz_unicode_init(const uint32_t* pFirstNameHash, const uint32_t* pLastNameHash, const uint32_t* pEmailHash, const uint32_t* pLicenseHash);

mdz_unicode_init_attached

Initializes unicode library. This function should be caled before any other function of the library.
Memory for license data starts at position pStart. Size of internal initialization structure is returned in pSize.

mdz_bool mdz_unicode_init_attached(const uint32_t* pFirstNameHash, const uint32_t* pLastNameHash, const uint32_t* pEmailHash, const uint32_t* pLicenseHash, const char* pStart, size_t nAreaSize, size_t* pOutSize);

mdz_unicode_uninit

Un-initializes unicode library and frees corresponding memory allocations.

void mdz_unicode_uninit(void);

mdz_wchar General Information and Functions

mdz_wchar is dynamically-sized contiguous string, containing C wchar_t characters.

Capacity - how many "wide"-characters memory is reserved for.
Size - how many "wide"-characters are actually used in a string, excluding terminating 0.
Length - actual length of string in symbols, excluding terminating 0. "surrogate pairs" count as 1 symbol.
"reserve/AndReserve" functions allocate/reallocate memory dynamically using malloc()/realloc().
"attach" functionality allows attaching contiguous block of memory to string, for using string functions on it.

UTF-16 "surrogate pairs" are supported and checked.
Unicode "combining characters" are not specially-distinguished and counted as distinct symbols.
Endianness of bytes in "wide"-character - is always endianness of platform.
Size of wchar_t in bytes - is compiler-dependend. At the moment, library supports only wchar_t with size of 2 or 4 bytes.

Init and destroy functions:

mdz_wchar_create
mdz_wchar_create_attached
mdz_wchar_destroy
mdz_wchar_clear
mdz_wchar_attachData

Reserve capacity functions:

mdz_wchar_reserve
mdz_wchar_capacity
mdz_wchar_size
mdz_wchar_length
mdz_wchar_sizeof
mdz_wchar_offsetFromStart
mdz_wchar_isAttachedData
mdz_wchar_embedSize

Insert/remove functions:

mdz_wchar_insertWchar_async
mdz_wchar_insertWchar_string_async
mdz_wchar_insertAnsi_async
mdz_wchar_insertAnsi_string_async
mdz_wchar_insertUtf8_async
mdz_wchar_insertUtf8_string_async
mdz_wchar_insertUtf16_async
mdz_wchar_insertUtf16_string_async
mdz_wchar_insertUtf32_async
mdz_wchar_insertUtf32_string_async

mdz_wchar_create

Create empty "wide"-character string with Capacity == 1 (for 0-terminator), Size == 0 and Length == 0.

struct mdz_Wchar* mdz_wchar_create(size_t nEmbedSize);

mdz_wchar_create_attached

Create empty "wide"-character string with Capacity == 1 (for 0-terminator), Size == 0 and Length == 0.
Memory for wchar structure starts at position pStart. Size of internal wchar structure (it is usually bigger than mdz_Wchar!) is returned in pSize.

struct mdz_Wchar* mdz_wchar_create_attached(const void* pStart, size_t nAreaSizeBytes, size_t* pOutSize);

mdz_wchar_destroy

Destroy "wide"-character string including underlying data. After destroying, pointer to string is set to NULL.
If wchar is attached using mdz_wchar_createAttached(), free() will not be called.
If wchar data is attached using mdz_wchar_attachData(), m_pData will not be destroyed.

void mdz_wchar_destroy(struct mdz_Wchar** ppWchar);

mdz_wchar_clear

Clear m_pData of "wide"-character string with setting Size in 0.

void mdz_wchar_clear(struct mdz_Wchar* pWchar);

mdz_wchar_attachData

Attach pre-allocated data to string, assigning pData to m_pData. If attached, m_pData will not be destroyed in mdz_wchar_destroy()

mdz_bool mdz_wchar_attachData(struct mdz_Wchar* pWchar, wchar_t* pData, size_t nOffsetFromStart, size_t nCapacity, enum mdz_attach_type enAttachType);

mdz_wchar_reserve

Reserve nNewCapacity bytes for string. Size and Length do not change.

mdz_bool mdz_wchar_reserve(struct mdz_Wchar* pWchar, size_t nNewCapacity);

mdz_wchar_capacity

Return string Capacity in "wide"-characters.

size_t mdz_wchar_capacity(const struct mdz_Wchar* pWchar);

mdz_wchar_size

Return string Size in "wide"-characters.

size_t mdz_wchar_size(const struct mdz_Wchar* pWchar);

mdz_wchar_length

Return string Length in symbols.

size_t mdz_wchar_length(const struct mdz_Wchar* pWchar);

mdz_wchar_sizeof

Return sizeof(wchar_t) of string in bytes.

size_t mdz_wchar_sizeof(const struct mdz_Wchar* pWchar);

mdz_wchar_offsetFromStart

Return string OffsetFromStart in "wide"-characters.

size_t mdz_wchar_offsetFromStart(const struct mdz_Wchar* pWchar);

mdz_wchar_isAttachedData

Return if string data is attached.

mdz_bool mdz_wchar_isAttachedData(const struct mdz_Wchar* pWchar);

mdz_wchar_embedSize

Return string "embedded part" Size in "wide" characters.

size_t mdz_wchar_embedSize(const struct mdz_Wchar* pWchar);

mdz_wchar_insertWchar_async

Insert nCount "wide"-characters pwcItems in string. Size grows on nCount. Length grows on symbols count.
String reserved area and pwcItems should not overlap.

mdz_bool mdz_wchar_insertWchar_async(struct mdz_Wchar* pWchar, size_t nLeftPos, const wchar_t* pwcItems, size_t nCount, size_t nWcharSize, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_wchar_insertWchar(pWchar, nLeftPos, pwcItems, nCount, nWcharSize, bReserve);

mdz_wchar_insertWchar_string_async

Insert "wide"-characters string pWcharSource in string. Size grows on nCount. Length grows on symbols count.
String reserved area and pWcharSource->m_pData should not overlap.

mdz_bool mdz_wchar_insertWchar_string_async(struct mdz_Wchar* pWchar, size_t nLeftPos, const struct mdz_Wchar* pWcharSource, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_wchar_insertWchar_string(pWchar, nLeftPos, pWcharSource, bReserve);

mdz_wchar_insertAnsi_async

Insert nCount ASCII/ANSI bytes in string. Characters are converted to "wide"-characters before isertion. Size grows on added "wide"-characters count. Length grows on added symbols count.
String reserved area and pcItems should not overlap.

mdz_bool mdz_wchar_insertAnsi_async(struct mdz_Wchar* pWchar, size_t nLeftPos, const unsigned char* pcItems, size_t nCount, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_wchar_insertAnsi(pWchar, nLeftPos, pcItems, nCount, bReserve);

mdz_wchar_insertAnsi_string_async

Insert ASCII/ANSI string pAnsiSource in string. Characters are converted to "wide"-characters before isertion. Size grows on added "wide"-characters count. Length grows on added symbols count.
String reserved area and pAnsiSource->m_pData should not overlap.

mdz_bool mdz_wchar_insertAnsi_string_async(struct mdz_Wchar* pWchar, size_t nLeftPos, const void* pAnsiSource, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_wchar_insertAnsi_string(pWchar, nLeftPos, pAnsiSource, bReserve);

mdz_wchar_insertUtf8_async

Insert nCount UTF-8 characters in string. UTF-8 characters are converted to "wide"-characters before isertion. Size grows on added "wide"-characters count. Length grows on added symbols count.
String reserved area and pcItems should not overlap.

mdz_bool mdz_wchar_insertUtf8_async(struct mdz_Wchar* pWchar, size_t nLeftPos, const unsigned char* pcItems, size_t nCount, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_wchar_insertUtf8(pWchar, nLeftPos, pcItems, nCount, bReserve);

mdz_wchar_insertUtf8_string_async

Insert UTF-8 string pUtf8Source in string. UTF-8 characters are converted to "wide"-characters before isertion. Size grows on added "wide"-characters count. Length grows on added symbols count.
String reserved area and pUtf8Source->m_pData should not overlap.

mdz_bool mdz_wchar_insertUtf8_string_async(struct mdz_Wchar* pWchar, size_t nLeftPos, const void* pUtf8Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_wchar_insertUtf8_string(pWchar, nLeftPos, pUtf8Source, bReserve);

mdz_wchar_insertUtf16_async

Insert nCount UTF-16 characters in string. UTF-16 characters are converted to "wide"-characters before isertion. Size grows on added "wide"-characters count. Length grows on added symbols count.
String reserved area and pItems should not overlap.

mdz_bool mdz_wchar_insertUtf16_async(struct mdz_Wchar* pWchar, size_t nLeftPos, const uint16_t* pItems, size_t nCount, enum mdz_endianness enEndianness, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_wchar_insertUtf16(pWchar, nLeftPos, pItems, nCount, enEndianness, bReserve);

mdz_wchar_insertUtf16_string_async

Insert UTF-16 string pUtf16Source in string. UTF-16 characters are converted to "wide"-characters before isertion. Size grows on added "wide"-characters count. Length grows on added symbols count.
String reserved area and pUtf16Source->m_pData should not overlap.

mdz_bool mdz_wchar_insertUtf16_string_async(struct mdz_Wchar* pWchar, size_t nLeftPos, const void* pUtf16Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_wchar_insertUtf16_string(pWchar, nLeftPos, pUtf16Source, bReserve);

mdz_wchar_insertUtf32_async

Insert nCount UTF-32 characters in string. UTF-32 characters are converted to "wide"-characters before isertion. Size grows on added "wide"-characters count. Length grows on added symbols count.
String reserved area and pItems should not overlap.

mdz_bool mdz_wchar_insertUtf32_async(struct mdz_Wchar* pWchar, size_t nLeftPos, const uint32_t* pItems, size_t nCount, enum mdz_endianness enEndianness, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_wchar_insertUtf32(pWchar, nLeftPos, pItems, nCount, enEndianness, bReserve);

mdz_wchar_insertUtf32_string_async

Insert UTF-32 string pUtf32Source in string. UTF-32 characters are converted to "wide"-characters before isertion. Size grows on added "wide"-characters count. Length grows on added symbols count.
String reserved area and pUtf32Source->m_pData should not overlap.

mdz_bool mdz_wchar_insertUtf32_string_async(struct mdz_Wchar* pWchar, size_t nLeftPos, const void* pUtf32Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_wchar_insertUtf32_string(pWchar, nLeftPos, pUtf32Source, bReserve);

mdz_utf8 General Information and Functions

mdz_utf8 is dynamically-sized contiguous string, containing UTF-8 characters

If only ASCII/ANSI characters should be stored/processed - use mdz_ansi instead. It is much more speedy.

Capacity - how many bytes of memory is reserved.
Size - how many bytes are actually used in a string, excluding terminating 0.
Length - actual length of string in UTF-8 symbols, excluding terminating 0.

"reserve/AndReserve" functions allocate/reallocate memory dynamically using malloc()/realloc().
"attach" functionality allows attaching contiguous block of memory to string, for using string functions on it.

Unicode "combining characters" are not specially-distinguished and counted as a distinct symbols.

See additional info on mdz_unicode library like version, portability, etc in mdz_unicode.h

Init and destroy functions:

mdz_utf8_create
mdz_utf8_create_attached
mdz_utf8_destroy
mdz_utf8_clear
mdz_utf8_attachData

Reserve capacity functions:

mdz_utf8_reserve
mdz_utf8_capacity
mdz_utf8_size
mdz_utf8_length
mdz_utf8_offsetFromStart
mdz_utf8_isAttachedData
mdz_utf8_embedSize

Insert/remove functions:

mdz_utf8_insertUtf8_async
mdz_utf8_insertUtf8_string_async
mdz_utf8_insertAnsi_async
mdz_utf8_insertAnsi_string_async
mdz_utf8_insertWchar_async
mdz_utf8_insertWchar_string_async
mdz_utf8_insertUtf16_async
mdz_utf8_insertUtf16_string_async
mdz_utf8_insertUtf32_async
mdz_utf8_insertUtf32_string_async

mdz_utf8_create

Create empty UTF-8 string with Capacity == 1 (for 0-terminator), Size == 0 and Length == 0.

struct mdz_Utf8* mdz_utf8_create(size_t nEmbedSize);

mdz_utf8_create_attached

Create empty UTF-8 string with Capacity == 1 (for 0-terminator), Size == 0 and Length == 0.
Memory for utf8 structure starts at position pStart. Size of internal utf8 structure (it is usually bigger than mdz_Utf8!) is returned in pSize.

struct mdz_Utf8* mdz_utf8_create_attached(const void* pStart, size_t nAreaSizeBytes, size_t* pOutSize);

mdz_utf8_destroy

Destroy UTF-8 string including underlying data. After destroying, pointer to string is set to NULL.
If utf8 is attached using mdz_utf8_createAttached(), free() will not be called.
If utf8 data is attached using mdz_utf8_attachData(), m_pData will not be destroyed.

void mdz_utf8_destroy(struct mdz_Utf8** ppUtf8);

mdz_utf8_clear

Clear m_pData of UTF-8 string with setting Size in 0.

void mdz_utf8_clear(struct mdz_Utf8* pUtf8);

mdz_utf8_attachData

Attach pre-allocated data to UTF-8 string, assigning pData to m_pData. If attached, m_pData will not be destroyed in mdz_utf8_destroy()

mdz_bool mdz_utf8_attachData(struct mdz_Utf8* pUtf8, unsigned char* pData, size_t nOffsetFromStart, size_t nCapacity, enum mdz_attach_type enAttachType);

mdz_utf8_reserve

Reserve nNewCapacity bytes for UTF-8 string. String Size and Length do not change.

mdz_bool mdz_utf8_reserve(struct mdz_Utf8* pUtf8, size_t nNewCapacity);

mdz_utf8_capacity

Return string Capacity in bytes.

size_t mdz_utf8_capacity(const struct mdz_Utf8* pUtf8);

mdz_utf8_size

Return string Size in bytes.

size_t mdz_utf8_size(const struct mdz_Utf8* pUtf8);

mdz_utf8_length

Return string Length in symbols.

size_t mdz_utf8_length(const struct mdz_Utf8* pUtf8);

mdz_utf8_offsetFromStart

Return string OffsetFromStart in bytes.

size_t mdz_utf8_offsetFromStart(const struct mdz_Utf8* pUtf8);

mdz_utf8_isAttachedData

Return if string data is attached.

mdz_bool mdz_utf8_isAttachedData(const struct mdz_Utf8* pUtf8);

mdz_utf8_embedSize

Return string "embedded part" Size in bytes.

size_t mdz_utf8_embedSize(const struct mdz_Utf8* pUtf8);

mdz_utf8_insertUtf8_async

Insert nCount UTF-8 bytes in string. Size grows on nCount. Length grows on symbols count.
String reserved area and pcItems should not overlap.

mdz_bool mdz_utf8_insertUtf8_async(struct mdz_Utf8* pUtf8, size_t nLeftPos, const unsigned char* pcItems, size_t nCount, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf8_insertUtf8(pUtf8, nLeftPos, pcItems, nCount, bReserve);

mdz_utf8_insertUtf8_string_async

Insert UTF-8 string pUtf8Source in string. Size grows on nCount. Length grows on symbols count.
String reserved area and pUtf8Source->m_pData should not overlap.
This function performs significantly better than mdz_utf8_insertUtf8_async() - because there is no additional validation of inserted string.

mdz_bool mdz_utf8_insertUtf8_string_async(struct mdz_Utf8* pUtf8, size_t nLeftPos, const struct mdz_Utf8* pUtf8Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf8_insertUtf8_string(pUtf8, nLeftPos, pUtf8Source, bReserve);

mdz_utf8_insertAnsi_async

Insert nCount ASCII/ANSI bytes in string. Characters are converted to UTF-8 characters before isertion. Size grows on added bytes. Length grows on added symbols count.
String reserved area and pcItems should not overlap.

mdz_bool mdz_utf8_insertAnsi_async(struct mdz_Utf8* pUtf8, size_t nLeftPos, const unsigned char* pcItems, size_t nCount, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf8_insertAnsi(pUtf8, nLeftPos, pcItems, nCount, bReserve);

mdz_utf8_insertAnsi_string_async

Insert ASCII/ANSI string pAnsiSource in string. Characters are converted to UTF-8 characters before isertion. Size grows on added bytes. Length grows on added symbols count.
String reserved area and pAnsiSource->m_pData should not overlap.

mdz_bool mdz_utf8_insertAnsi_string_async(struct mdz_Utf8* pUtf8, size_t nLeftPos, const void* pAnsiSource, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf8_insertAnsi_string(pUtf8, nLeftPos, pAnsiSource, bReserve);

mdz_utf8_insertWchar_async

Insert nCount "wide"-characters in string. Characters are converted to UTF-8 characters before isertion. Size grows on added bytes. Length grows on symbols count.
String reserved area and pwcItems should not overlap.

mdz_bool mdz_utf8_insertWchar_async(struct mdz_Utf8* pUtf8, size_t nLeftPos, const wchar_t* pwcItems, size_t nCount, size_t nWcharSize, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf8_insertWchar(pUtf8, nLeftPos, pwcItems, nCount, nWcharSize, bReserve);

mdz_utf8_insertWchar_string_async

Insert "wide"-characters string pWcharSource in string. Characters are converted to UTF-8 characters before isertion. Size grows on added bytes. Length grows on symbols count.
String reserved area and pWcharSource->m_pData should not overlap.
For 2-bytes "wide"-characters - see information concerning Unicode "surrogate pairs"/"combining characters" in description of mdz_utf8_insertUtf16_async().
For 4-bytes "wide"-characters - see information concerning Unicode "combining characters" in description of mdz_utf8_insertUtf32_async().

mdz_bool mdz_utf8_insertWchar_string_async(struct mdz_Utf8* pUtf8, size_t nLeftPos, const void* pWcharSource, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf8_insertWchar_string(pUtf8, nLeftPos, pWcharSource, bReserve);

mdz_utf8_insertUtf16_async

Insert nCount UTF-16 characters in string. UTF-16 characters are converted to UTF-8 before isertion. Size grows on added bytes. Length grows on symbols count.
String reserved area and pItems should not overlap.
UTF-16 "surrogate pairs" are supported and counted as 1 symbol (4 bytes size).
UTF-16 "combining characters" are not specially-distinguished and counted as a separate UTF-8 symbol.

mdz_bool mdz_utf8_insertUtf16_async(struct mdz_Utf8* pUtf8, size_t nLeftPos, const uint16_t* pItems, size_t nCount, enum mdz_endianness enEndianness, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf8_insertUtf16(pUtf8, nLeftPos, pItems, nCount, enEndianness, bReserve);

mdz_utf8_insertUtf16_string_async

Insert UTF-16 string pUtf16Source in string. UTF-16 characters are converted to UTF-8 before isertion. Size grows on added bytes. Length grows on symbols count.
String reserved area and pUtf16Source->m_pData should not overlap.
UTF-16 "surrogate pairs" are supported and counted as 1 symbol (4 bytes size).
UTF-16 "combining characters" are not specially-distinguished and counted as a separate UTF-8 symbol.

mdz_bool mdz_utf8_insertUtf16_string_async(struct mdz_Utf8* pUtf8, size_t nLeftPos, const void* pUtf16Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf8_insertUtf16_string(pUtf8, nLeftPos, pUtf16Source, bReserve);

mdz_utf8_insertUtf32_async

Insert nCount UTF-32 characters in string. UTF-32 characters are converted to UTF-8 before isertion. Size grows on added bytes. Length grows on symbols count.
String reserved area and pItems should not overlap.
Unicode "combining characters" are not specially-distinguished and counted as a separate UTF-8 symbol.

mdz_bool mdz_utf8_insertUtf32_async(struct mdz_Utf8* pUtf8, size_t nLeftPos, const uint32_t* pItems, size_t nCount, enum mdz_endianness enEndianness, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf8_insertUtf32(pUtf8, nLeftPos, pItems, nCount, enEndianness, bReserve);

mdz_utf8_insertUtf32_string_async

Insert UTF-32 string pUtf32Source in string. UTF-32 characters are converted to UTF-8 before isertion. Size grows on added bytes. Length grows on symbols count.
String reserved area and pUtf32Source->m_pData should not overlap.
Unicode "combining characters" are not specially-distinguished and counted as a separate UTF-8 symbol.

mdz_bool mdz_utf8_insertUtf32_string_async(struct mdz_Utf8* pUtf8, size_t nLeftPos, const void* pUtf32Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf8_insertUtf32_string(pUtf8, nLeftPos, pUtf32Source, bReserve);

mdz_utf16 General Information and Functions

mdz_utf16 is dynamically-sized contiguous string, containing UTF-16 characters.

Capacity - how many UTF-16 characters memory is reserved for.
Size - how many UTF-16 characters are actually used in a string, excluding terminating 0.
Length - actual length of string in symbols, excluding terminating 0. "surrogate pairs" count as 1 symbol.

"reserve/AndReserve" functions allocate/reallocate memory dynamically using malloc()/realloc().
"attach" functionality allows attaching contiguous block of memory to string, for using string functions on it.

UTF-16 "surrogate pairs" are supported and checked
Unicode "combining characters" are not specially-distinguished and counted as distinct symbols
Endianness of bytes in UTF-16 character - is defined by m_enEndianness and set using enEndianness during string-creation in mdz_utf16_create()

Init and destroy functions:

mdz_utf16_create
mdz_utf16_create_attached
mdz_utf16_destroy
mdz_utf16_clear
mdz_utf16_attachData

Reserve capacity functions:

mdz_utf16_reserve
mdz_utf16_capacity
mdz_utf16_size
mdz_utf16_length
mdz_utf16_endianness
mdz_utf16_offsetFromStart
mdz_utf16_isAttachedData
mdz_utf16_embedSize

Insert/remove functions:

mdz_utf16_insertUtf16_async
mdz_utf16_insertUtf16_string_async
mdz_utf16_insertAnsi_async
mdz_utf16_insertAnsi_string_async
mdz_utf16_insertWchar_async
mdz_utf16_insertWchar_string_async
mdz_utf16_insertUtf8_async
mdz_utf16_insertUtf8_string_async
mdz_utf16_insertUtf32_async
mdz_utf16_insertUtf32_string_async

mdz_utf16_create

Create empty UTF-16 string with Capacity == 1 (for 0-terminator), Size == 0 and Length == 0.

struct mdz_Utf16* mdz_utf16_create(size_t nEmbedSize, enum mdz_endianness enEndianness);

mdz_utf16_create_attached

Create empty UTF-16 string with Capacity == 1 (for 0-terminator), Size == 0 and Length == 0.
Memory for utf16 structure starts at position pStart. Size of internal utf16 structure (it is usually bigger than mdz_Utf16!) is returned in pSize.

struct mdz_Utf16* mdz_utf16_create_attached(const void* pStart, enum mdz_endianness enEndianness, size_t nAreaSizeBytes, size_t* pOutSize);

mdz_utf16_destroy

Destroy UTF-16 string including underlying data. After destroying, pointer to string is set to NULL.
If utf16 is attached using mdz_utf16_createAttached(), free() will not be called.
If utf16 data is attached using mdz_utf16_attachData(), m_pData will not be destroyed.

void mdz_utf16_destroy(struct mdz_Utf16** ppUtf16);

mdz_utf16_clear

Clear m_pData of UTF-16 string with setting Size in 0.

void mdz_utf16_clear(struct mdz_Utf16* pUtf16);

mdz_utf16_attachData

Attach pre-allocated data to UTF-16 string, assigning pData to m_pData. If attached, m_pData will not be destroyed in mdz_utf16_destroy()

mdz_bool mdz_utf16_attachData(struct mdz_Utf16* pUtf16, uint16_t* pData, size_t nOffsetFromStart, size_t nCapacity, enum mdz_attach_type enAttachType, enum mdz_endianness enEndianness);

mdz_utf16_reserve

Reserve nNewCapacity bytes for UTF-16 string. Size and Length do not change.

mdz_bool mdz_utf16_reserve(struct mdz_Utf16* pUtf16, size_t nNewCapacity);

mdz_utf16_capacity

Return string Capacity in UTF-16 characters.

size_t mdz_utf16_capacity(const struct mdz_Utf16* pUtf16);

mdz_utf16_size

Return string Size in UTF-16 characters.

size_t mdz_utf16_size(const struct mdz_Utf16* pUtf16);

mdz_utf16_length

Return string Length in symbols.

size_t mdz_utf16_length(const struct mdz_Utf16* pUtf16);

mdz_utf16_endianness

Return string endianness.

enum mdz_endianness mdz_utf16_endianness(const struct mdz_Utf16* pUtf16);

mdz_utf16_offsetFromStart

Return string OffsetFromStart in UTF-16 characters.

size_t mdz_utf16_offsetFromStart(const struct mdz_Utf16* pUtf16);

mdz_utf16_isAttachedData

Return if string data is attached.

mdz_bool mdz_utf16_isAttachedData(const struct mdz_Utf16* pUtf16);

mdz_utf16_embedSize

Return string "embedded part" Size in UTF-16 characters.

size_t mdz_utf16_embedSize(const struct mdz_Utf16* pUtf16);

mdz_utf16_insertUtf16_async

Insert nCount UTF-16 characters in string. Size grows on nCount. Length grows on symbols count.
String reserved area and pItems should not overlap.

mdz_bool mdz_utf16_insertUtf16_async(struct mdz_Utf16* pUtf16, size_t nLeftPos, const uint16_t* pItems, size_t nCount, enum mdz_endianness enEndianness, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf16_insertUtf16(pUtf16, nLeftPos, pItems, nCount, enEndianness, bReserve);

mdz_utf16_insertUtf16_string_async

Insert UTF-16 string pUtf16Source in string. Size grows on nCount. Length grows on symbols count.
String reserved area and pUtf16Source->m_pData should not overlap.
This function performs significantly better than mdz_utf16_insertUtf16_async() - because there is no additional validation of inserted string.

mdz_bool mdz_utf16_insertUtf16_string_async(struct mdz_Utf16* pUtf16, size_t nLeftPos, const struct mdz_Utf16* pUtf16Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf16_insertUtf16_string(pUtf16, nLeftPos, pUtf16Source, bReserve);

mdz_utf16_insertAnsi_async

Insert nCount ASCII/ANSI bytes in string. Characters are converted to UTF-16 characters before isertion. Size grows on added UTF-16 characters count. Length grows on added symbols count.
String reserved area and pcItems should not overlap.

mdz_bool mdz_utf16_insertAnsi_async(struct mdz_Utf16* pUtf16, size_t nLeftPos, const unsigned char* pcItems, size_t nCount, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf16_insertAnsi(pUtf16, nLeftPos, pcItems, nCount, bReserve);

mdz_utf16_insertAnsi_string_async

Insert ASCII/ANSI string pAnsiSource in string. Characters are converted to UTF-16 characters before isertion. Size grows on added UTF-16 characters count. Length grows on added symbols count.
String reserved area and pAnsiSource->m_pData should not overlap.

mdz_bool mdz_utf16_insertAnsi_string_async(struct mdz_Utf16* pUtf16, size_t nLeftPos, const void* pAnsiSource, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf16_insertAnsi_string(pUtf16, nLeftPos, pAnsiSource, bReserve);

mdz_utf16_insertWchar_async

Insert nCount "wide"-characters in string. Characters are converted to UTF-16 characters before isertion. Size grows on added UTF-16 characters count. Length grows on added symbols count.
String reserved area and pwcItems should not overlap.

mdz_bool mdz_utf16_insertWchar_async(struct mdz_Utf16* pUtf16, size_t nLeftPos, const wchar_t* pwcItems, size_t nCount, size_t nWcharSize, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf16_insertWchar(pUtf16, nLeftPos, pwcItems, nCount, nWcharSize, bReserve);

mdz_utf16_insertWchar_string_async

Insert "wide"-characters string pWcharSource in string. Characters are converted to UTF-16 characters before isertion. Size grows on added UTF-16 characters count. Length grows on added symbols count.
String reserved area and pWcharSource->m_pData should not overlap.
This function performs significantly better than mdz_utf16_insertWchar_async() - because there is no additional validation of inserted string.

mdz_bool mdz_utf16_insertWchar_string_async(struct mdz_Utf16* pUtf16, size_t nLeftPos, const void* pWcharSource, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf16_insertWchar_string(pUtf16, nLeftPos, pWcharSource, bReserve);

mdz_utf16_insertUtf8_async

Insert nCount UTF-8 bytes in string. UTF-8 bytes are converted to UTF-16 characters before isertion. Size grows on added UTF-16 characters count. Length grows on added symbols count.
String reserved area and pcItems should not overlap.

mdz_bool mdz_utf16_insertUtf8_async(struct mdz_Utf16* pUtf16, size_t nLeftPos, const unsigned char* pcItems, size_t nCount, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf16_insertUtf8(pUtf16, nLeftPos, pcItems, nCount, bReserve);

mdz_utf16_insertUtf8_string_async

Insert UTF-8 string pUtf8Source in string. UTF-8 bytes are converted to UTF-16 characters before isertion. Size grows on added UTF-16 characters count. Length grows on added symbols count.
String reserved area and pUtf8Source->m_pData should not overlap.

mdz_bool mdz_utf16_insertUtf8_string_async(struct mdz_Utf16* pUtf16, size_t nLeftPos, const void* pUtf8Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf16_insertUtf8_string(pUtf16, nLeftPos, pUtf8Source, bReserve);

mdz_utf16_insertUtf32_async

Insert nCount UTF-32 characters in string. UTF-32 characters are converted to UTF-16 characters before isertion. Size grows on added UTF-16 characters count. Length grows on added symbols count.
String reserved area and pItems should not overlap.

mdz_bool mdz_utf16_insertUtf32_async(struct mdz_Utf16* pUtf16, size_t nLeftPos, const uint32_t* pItems, size_t nCount, enum mdz_endianness enEndianness, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf16_insertUtf32(pUtf16, nLeftPos, pItems, nCount, enEndianness, bReserve);

mdz_utf16_insertUtf32_string_async

Insert UTF-32 string pUtf32Source in string. UTF-32 characters are converted to UTF-16 characters before isertion. Size grows on added UTF-16 characters count. Length grows on added symbols count.
String reserved area and pUtf32Source->m_pData should not overlap.

mdz_bool mdz_utf16_insertUtf32_string_async(struct mdz_Utf16* pUtf16, size_t nLeftPos, const void* pUtf32Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf16_insertUtf32_string(pUtf16, nLeftPos, pUtf32Source, bReserve);

mdz_utf32 General Information and Functions

mdz_utf32 is dynamically-sized contiguous string, containing UTF-32 characters.

Capacity - how many UTF-32 characters memory is reserved for.
Size - how many UTF-32 characters are actually used in a string, excluding terminating 0.
Length - actual length of string in symbols, excluding terminating 0.

"reserve/AndReserve" functions allocate/reallocate memory dynamically using malloc()/realloc().
"attach" functionality allows attaching contiguous block of memory to string, for using string functions on it.

Unicode "combining characters" are not specially-distinguished and counted as distinct symbols
Endianness of bytes in UTF-32 character - is defined by m_enEndianness and set using enEndianness during string-creation in mdz_utf32_create()

Init and destroy functions:

mdz_utf32_create
mdz_utf32_create_attached
mdz_utf32_destroy
mdz_utf32_clear
mdz_utf32_attachData

Reserve capacity functions:

mdz_utf32_reserve
mdz_utf32_capacity
mdz_utf32_size
mdz_utf32_length
mdz_utf32_endianness
mdz_utf32_offsetFromStart
mdz_utf32_isAttachedData
mdz_utf32_embedSize

Insert/remove functions:

mdz_utf32_insertUtf32_async
mdz_utf32_insertUtf32_string_async
mdz_utf32_insertAnsi_async
mdz_utf32_insertAnsi_string_async
mdz_utf32_insertWchar_async
mdz_utf32_insertWchar_string_async
mdz_utf32_insertUtf8_async
mdz_utf32_insertUtf8_string_async
mdz_utf32_insertUtf16_async
mdz_utf32_insertUtf16_string_async

mdz_utf32_create

Create empty UTF-32 string with Capacity == 1 (for 0-terminator), Size == 0 and Length == 0.

struct mdz_Utf32* mdz_utf32_create(size_t nEmbedSize, enum mdz_endianness enEndianness);

mdz_utf32_create_attached

Create empty UTF-32 string with Capacity == 1 (for 0-terminator), Size == 0 and Length == 0.
Memory for utf32 structure starts at position pStart. Size of internal utf32 structure (it is usually bigger than mdz_Utf32!) is returned in pSize.

struct mdz_Utf32* mdz_utf32_create_attached(const void* pStart, enum mdz_endianness enEndianness, size_t nAreaSizeBytes, size_t* pOutSize);

mdz_utf32_destroy

Destroy UTF-32 string including underlying data. After destroying, pointer to string is set to NULL.
If utf32 is attached using mdz_utf32_createAttached(), free() will not be called.
If utf32 data is attached using mdz_utf32_attachData(), m_pData will not be destroyed.

void mdz_utf32_destroy(struct mdz_Utf32** ppUtf32);

mdz_utf32_clear

Clear m_pData of UTF-32 string with setting Size in 0.

void mdz_utf32_clear(struct mdz_Utf32* pUtf32);

mdz_utf32_attachData

Attach pre-allocated data to UTF-32 string, assigning pData to m_pData. If attached, m_pData will not be destroyed in mdz_utf32_destroy()

mdz_bool mdz_utf32_attachData(struct mdz_Utf32* pUtf32, uint32_t* pData, size_t nOffsetFromStart, size_t nCapacity, enum mdz_attach_type enAttachType, enum mdz_endianness enEndianness);

mdz_utf32_reserve

Reserve nNewCapacity bytes for UTF-32 string. Size and Length do not change.

mdz_bool mdz_utf32_reserve(struct mdz_Utf32* pUtf32, size_t nNewCapacity);

mdz_utf32_capacity

Return string Capacity in UTF-32 characters.

size_t mdz_utf32_capacity(const struct mdz_Utf32* pUtf32);

mdz_utf32_size

Return string Size in UTF-32 characters.

size_t mdz_utf32_size(const struct mdz_Utf32* pUtf32);

mdz_utf32_length

Return string length in symbols.

size_t mdz_utf32_length(const struct mdz_Utf32* pUtf32);

mdz_utf32_endianness

Return string endianness.

enum mdz_endianness mdz_utf32_endianness(const struct mdz_Utf32* pUtf32);

mdz_utf32_offsetFromStart

Return string OffsetFromStart in UTF-32 characters.

size_t mdz_utf32_offsetFromStart(const struct mdz_Utf32* pUtf32);

mdz_utf32_isAttachedData

Return if string data is attached.

mdz_bool mdz_utf32_isAttachedData(const struct mdz_Utf32* pUtf32);

mdz_utf32_embedSize

Return string "embedded part" Size in UTF-32 characters.

size_t mdz_utf32_embedSize(const struct mdz_Utf32* pUtf32);

mdz_utf32_insertUtf32_async

Insert nCount UTF-32 characters in string. Size grows on nCount. Length grows on symbols count.
String reserved area and pItems should not overlap.

mdz_bool mdz_utf32_insertUtf32_async(struct mdz_Utf32* pUtf32, size_t nLeftPos, const uint32_t* pItems, size_t nCount, enum mdz_endianness enEndianness, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf32_insertUtf32(pUtf32, nLeftPos, pItems, nCount, enEndianness, bReserve);

mdz_utf32_insertUtf32_string_async

Insert UTF-32 string pUtf32Source in string. Size grows on nCount. Length grows on symbols count.
String reserved area and pUtf32Source->m_pData should not overlap.
This function performs significantly better than mdz_utf32_insertUtf32_async() - because there is no additional validation of inserted string.

mdz_bool mdz_utf32_insertUtf32_string_async(struct mdz_Utf32* pUtf32, size_t nLeftPos, const struct mdz_Utf32* pUtf32Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf32_insertUtf32_string(pUtf32, nLeftPos, pUtf32Source, bReserve);

mdz_utf32_insertAnsi_async

Insert nCount ASCII/ANSI bytes in string. Characters are converted to UTF-32 characters before isertion. Size grows on added UTF-32 characters count. Length grows on added symbols count.
String reserved area and pcItems should not overlap.

mdz_bool mdz_utf32_insertAnsi_async(struct mdz_Utf32* pUtf32, size_t nLeftPos, const unsigned char* pcItems, size_t nCount, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf32_insertAnsi(pUtf32, nLeftPos, pcItems, nCount, bReserve);

mdz_utf32_insertAnsi_string_async

Insert ASCII/ANSI string pAnsiSource in string. Characters are converted to UTF-32 characters before isertion. Size grows on added UTF-32 characters count. Length grows on added symbols count.
String reserved area and pAnsiSource->m_pData should not overlap.

mdz_bool mdz_utf32_insertAnsi_string_async(struct mdz_Utf32* pUtf32, size_t nLeftPos, const void* pAnsiSource, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf32_insertAnsi_string(pUtf32, nLeftPos, pAnsiSource, bReserve);

mdz_utf32_insertWchar_async

Insert nCount "wide"-characters in string. Characters are converted to UTF-32 characters before isertion. Size grows on added UTF-32 characters count. Length grows on added symbols count.
String reserved area and pwcItems should not overlap.

mdz_bool mdz_utf32_insertWchar_async(struct mdz_Utf32* pUtf32, size_t nLeftPos, const wchar_t* pwcItems, size_t nCount, size_t nWcharSize, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf32_insertWchar(pUtf32, nLeftPos, pwcItems, nCount, nWcharSize, bReserve);

mdz_utf32_insertWchar_string_async

Insert "wide"-characters string pWcharSource in string. Characters are converted to UTF-32 characters before isertion. Size grows on added UTF-32 characters count. Length grows on added symbols count.
String reserved area and pWcharSource->m_pData should not overlap.

mdz_bool mdz_utf32_insertWchar_string_async(struct mdz_Utf32* pUtf32, size_t nLeftPos, const void* pWcharSource, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf32_insertWchar_string(pUtf32, nLeftPos, pWcharSource, bReserve);

mdz_utf32_insertUtf8_async

Insert nCount UTF-8 characters in string. UTF-8 characters are converted to UTF-32 characters before isertion. Size grows on added UTF-32 characters count. Length grows on added symbols count.
String reserved area and pcItems should not overlap.

mdz_bool mdz_utf32_insertUtf8_async(struct mdz_Utf32* pUtf32, size_t nLeftPos, const unsigned char* pcItems, size_t nCount, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf32_insertUtf8(pUtf32, nLeftPos, pcItems, nCount, bReserve);

mdz_utf32_insertUtf8_string_async

Insert UTF-8 string pUtf8Source in string. UTF-8 characters are converted to UTF-32 characters before isertion. Size grows on added UTF-32 characters count. Length grows on added symbols count.
String reserved area and pUtf8Source->m_pData should not overlap.

mdz_bool mdz_utf32_insertUtf8_string_async(struct mdz_Utf32* pUtf32, size_t nLeftPos, const void* pUtf8Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf32_insertUtf8_string(pUtf32, nLeftPos, pUtf8Source, bReserve);

mdz_utf32_insertUtf16_async

Insert nCount UTF-16 characters in string. Characters are converted to UTF-32 characters before isertion. Size grows on added UTF-32 characters count. Length grows on symbols count.
String reserved area and pItems should not overlap.

mdz_bool mdz_utf32_insertUtf16_async(struct mdz_Utf32* pUtf32, size_t nLeftPos, const uint16_t* pItems, size_t nCount, enum mdz_endianness enEndianness, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf32_insertUtf16(pUtf32, nLeftPos, pItems, nCount, enEndianness, bReserve);

mdz_utf32_insertUtf16_string_async

Insert UTF-16 string pUtf16Source in string. Characters are converted to UTF-32 characters before isertion. Size grows on added UTF-32 characters count. Length grows on symbols count.
String reserved area and pUtf16Source->m_pData should not overlap.

mdz_bool mdz_utf32_insertUtf16_string_async(struct mdz_Utf32* pUtf32, size_t nLeftPos, const void* pUtf16Source, mdz_bool bReserve, struct mdz_asyncData* pAsyncData);

Synchronous version:
mdz_utf32_insertUtf16_string(pUtf32, nLeftPos, pUtf16Source, bReserve);