std::codecvt

Defined in header <locale>
template< 
    class InternT, 
    class ExternT, 
    class State
> class codecvt;

Class std::codecvt encapsulates conversion of character strings, including wide and multibyte, from one encoding to another. All file I/O operations performed through std::basic_fstream<CharT> use the std::codecvt<CharT, char, std::mbstate_t> facet of the locale imbued in the stream.

std-codecvt-inheritance.svg

Inheritance diagram.

Four standalone (locale-independent) specializations are provided by the standard library:

Defined in header <locale>
std::codecvt<char, char, std::mbstate_t> identity conversion
std::codecvt<char16_t, char, std::mbstate_t> conversion between UTF-16 and UTF-8 (since C++11)
std::codecvt<char32_t, char, std::mbstate_t> conversion between UTF-32 and UTF-8 (since C++11)
std::codecvt<wchar_t, char, std::mbstate_t> conversion between the system's native wide and multibyte encodings

In addition, every locale object constructed in a C++ program implements its own (locale-specific) versions of these four specializations.

Member types

Member type Definition
intern_type InternT
extern_type ExternT
state_type State

Member functions

constructs a new codecvt facet
(public member function)
(destructor)
destructs a codecvt facet
(protected member function)
invokes do_out
(public member function)
invokes do_in
(public member function)
invokes do_unshift
(public member function)
invokes do_encoding
(public member function)
invokes do_always_noconv
(public member function)
invokes do_length
(public member function)
invokes do_max_length
(public member function)

Member objects

Member name Type
id (static) std::locale::id

Protected member functions

[virtual]
converts a string from internT to externT, such as when writing to file
(virtual protected member function)
[virtual]
converts a string from externT to internT, such as when reading from file
(virtual protected member function)
[virtual]
generates the termination character sequence of externT characters for incomplete conversion
(virtual protected member function)
[virtual]
returns the number of externT characters necessary to produce one internT character, if constant
(virtual protected member function)
tests if the facet encodes an identity conversion for all valid argument values
(virtual protected member function)
[virtual]
calculates the length of the externT string that would be consumed by conversion into given internT buffer
(virtual protected member function)
[virtual]
returns the maximum number of externT characters that could be converted into a single internT character
(virtual protected member function)

Inherited from std::codecvt_base

Member type Definition
enum result { ok, partial, error, noconv }; Unscoped enumeration type
Enumeration constant Definition
ok conversion was completed with no error
partial not all source characters were converted
error encountered an invalid character
noconv no conversion required, input and output types are the same

Example

The following examples reads a UTF-8 file using a locale which implements UTF-8 conversion in codecvt<wchar_t, char, mbstate_t> and converts a UTF-8 string to UTF-16 using one of the standard specializations of std::codecvt.

#include <iostream>
#include <fstream>
#include <string>
#include <locale>
#include <iomanip>
#include <string>
#include <codecvt>
 
// utility wrapper to adapt locale-bound facets for wstring/wbuffer convert
template<class Facet>
struct deletable_facet : Facet
{
    template<class ...Args>
    deletable_facet(Args&& ...args) : Facet(std::forward<Args>(args)...) {}
    ~deletable_facet() {}
};
 
int main()
{
    // UTF-8 narrow multibyte encoding
    std::string data = u8"z\u00df\u6c34\U0001f34c";
                       // or u8"zß水🍌"
                       // or "\x7a\xc3\x9f\xe6\xb0\xb4\xf0\x9f\x8d\x8c";
 
    std::ofstream("text.txt") << data;
 
    // using system-supplied locale's codecvt facet
    std::wifstream fin("text.txt");
    // reading from wifstream will use codecvt<wchar_t, char, mbstate_t>
    // this locale's codecvt converts UTF-8 to UCS4 (on systems such as Linux)
    fin.imbue(std::locale("en_US.UTF-8"));
    std::cout << "The UTF-8 file contains the following UCS4 code points: \n";
    for (wchar_t c; fin >> c; )
        std::cout << "U+" << std::hex << std::setw(4) << std::setfill('0') << c << '\n';
 
    // using standard (locale-independent) codecvt facet
    std::wstring_convert<
        deletable_facet<std::codecvt<char16_t, char, std::mbstate_t>>, char16_t> conv16;
    std::u16string str16 = conv16.from_bytes(data);
 
    std::cout << "The UTF-8 file contains the following UTF-16 code points: \n";
    for (char16_t c : str16)
        std::cout << "U+" << std::hex << std::setw(4) << std::setfill('0') << c << '\n';
}

Output:

The UTF-8 file contains the following UCS4 code points:
U+007a
U+00df
U+6c34
U+1f34c
The UTF-8 file contains the following UTF-16 code points:
U+007a
U+00df
U+6c34
U+d83c
U+df4c

See also

Character
conversions
narrow multibyte
(char)
UTF-8
(char)
UTF-16
(char16_t)
UTF-16 mbrtoc16(one way) codecvt<char16_t, char, mbstate_t>
codecvt_utf8_utf16<char16_t>
codecvt_utf8_utf16<char32_t>
codecvt_utf8_utf16<wchar_t>
N/A
UCS2 c16rtomb(one way) codecvt_utf8<char16_t> codecvt_utf16<char16_t>
UTF-32/UCS4
(char32_t)
mbrtoc32 / c32rtomb codecvt<char32_t, char, mbstate_t>
codecvt_utf8<char32_t>
codecvt_utf16<char32_t>
UCS2/UCS4
(wchar_t)
No codecvt_utf8<wchar_t> codecvt_utf16<wchar_t>
wide
(wchar_t)
codecvt<wchar_t, char, mbstate_t>
mbsrtowcs / wcsrtombs
No No
defines character conversion errors
(class template)
creates a codecvt facet for the named locale
(class template)
(C++11)
converts between UTF-8 and UCS2/UCS4
(class template)
converts between UTF-16 and UCS2/UCS4
(class template)
converts between UTF-8 and UTF-16
(class template)
doc_CPP
2016-10-11 10:01:55
Comments
Leave a Comment

Please login to continue.