Member Avatar for amerninja2

Can someone please help me to understand how to read and write unicode from/to a file? also how to set and print a variable to/with unicode (like a wchar_t variable or whatever.)?
Im at an intermeddiate lvl in C++, and when I search this on google I don't understand a lot of the code on websites and stuff. (By the way I use int main() way more than int void() or anything that have to do with void...). Thanks!

P.S The purpose is for making a chinese program. Im an american person, but im taking chinese classes in school and I know a lot of it.

visit
visit

  • 3 Contributors
  • 3 Replies
  • 138 Views
  • 8 Hours Discussion Span
  • Latest Post Latest Post by NekoGráfico

Dani AI

Generated

Several practical routes work for Chinese text in C++. The posts here point toward wide-character I/O, which can be fine, but it is not the only or always the best choice. wanted something approachable; ‘s example mixes a char-based file stream with std::wstring (that combination will not compile because the stream element type must match the string type). correctly noted wide-character streams exist, but there are portability and encoding caveats worth calling out.

A clear, low-friction workflow is to keep files encoded as UTF-8 and treat file I/O as byte I/O (std::ifstream / std::string). Convert to wchar_t only when interacting with APIs that require it. A common (legacy) conversion looks like:

#include <string>
#include <locale>
#include <codecvt>

// s is a UTF-8 std::string read from file
std::wstring wide = std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>>{}.from_bytes(s);
std::string back  = std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>>{}.to_bytes(wide);

Note: std::wstring_convert / std::codecvt are deprecated in modern C++; for robust, production-ready conversion prefer platform APIs (Windows: WideCharToMultiByte/MultiByteToWideChar) or libraries such as ICU or Boost.Locale.

Important platform notes and troubleshooting:

  • wchar_t size and encoding differ: Windows typically uses 16-bit UTF-16 code units; Linux/macOS typically use 32-bit UTF-32. Writing raw wchar_t bytes to disk is not portable.
  • Files intended for interchange are best UTF-8 (optionally with a BOM). Detect BOM by inspecting the first bytes (EF BB BF for UTF-8; FF FE / FE FF for UTF-16).
  • Console output depends on the terminal: Unix terminals usually expect UTF-8; Windows consoles often require setting the code page or using wide-API output (WriteConsoleW) to avoid garbled text.
  • If characters appear as question marks or boxes, the problem is almost always an encoding mismatch between the file, the C++ side, and the display device.

Practical next steps: confirm the file encoding in a hex/editor, pick UTF-8 for storage, use explicit conversion at API/console boundaries, and adopt ICU/Boost.Locale when full Unicode correctness (normalization, collation) is needed.

Recommended Answers

All 3 Replies

Member Avatar for NekoGráfico

Please don't use such vibrant colors... lol.
Haven't tested the code but I think this should work...

#include <iostream>
#include <fstream>
#include <string>
using namespace std;

int main()
{
	
	ifstream myfile("example.txt");
	wstring line;

	if (myfile.is_open())
	{
		while (! myfile.eof() )
		{
     			getline(myfile,line);
			wcout << line << endl;
 		}

    		myfile.close();
 	 }

	return 0;
}

Writing unicode files should be self-explanatory.

Member Avatar for ArkM

Use std::wifstream, std::wofstream and other w..streams - streams of wchar_t elements.

Member Avatar for NekoGráfico

Thanks for clearing that out! I didn't notice the use of typical streams.

Use std::wifstream, std::wofstream and other w..streams - streams of wchar_t elements.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.