C++17 In Detail

25 November 2019

How to Convert Numbers into Text with std::to_char in C++17

std::to_chars

In this post, I’ll show you how to use the newest, low-level, conversion routines form C++17. With the new functionality, you can quickly transform numbers into text and have super performance compared to previous techniques.

Intro

Until C++17, we had several ways of converting numbers into strings:

  • sprintf / snprintf
  • stringstream
  • to_string
  • itoa
  • and 3rd-party libraries like boost - lexical cast

And with C++17 we get another option: std::to_chars (along with the corresponding method from_chars) ! The functions both reside in the <charconv> header.

Why do we need new methods? Weren’t the old technique good enough?

In short: because to_chars/from_chars is low-level, and offers the best possible performance.

The new conversion routines are:

  • non-throwing
  • non-allocating
  • no locale support
  • memory safety
  • error reporting gives additional information about the conversion outcome
  • bound checked
  • explicit round-trip guarantees - you can use to_chars and from_charsto convert the number back and forth, and it will give you the exact binary representations. This is not guaranteed by other routines like printf/sscanf/itoa, etc.

A simple example:

std::string str { "xxxxxxxx" };
const int value = 1986;
std::to_chars(str.data(), str.data() + str.size(), value);

// str is "1986xxxx"

The new methods are available in the following compilers:

The Series

This article is part of my series about C++17 Library Utilities. Here’s the list of the articles:

Resources about C++17 STL:

Using to_chars

to_chars is a set of overloaded functions for integral and floating-point types.

For integral types there’s one declaration:

std::to_chars_result to_chars(char* first, char* last,
                              TYPE value, int base = 10);

Where TYPE expands to all available signed and unsigned integer types and char.

Since base might range from 2 to 36, the output digits that are greater than 9 are represented as lowercase letters: a...z.

For floating-point numbers, there are more options.

Firstly there’s a basic function:

std::to_chars_result to_chars(char* first, char* last, FLOAT_TYPE value);

FLOAT_TYPE expands to float, double or long double.

The conversion works the same as with printf and in default (“C”) locale. It uses %f or %e format specifier favouring the representation that is the shortest.

The next function overload adds std::chars_format fmt that let’s you specify the output format:

std::to_chars_result to_chars(char* first, char* last, 
                              FLOAT_TYPE value,
                              std::chars_format fmt);

chars_format is an enum with the following values: scientific, fixed, hex and general (which is a composition of fixed and scientific).

Then there’s the “full” version that allows also to specify precision:

std::to_chars_result to_chars(char* first, char* last, 
                              FLOAT_TYPE value,
                              std::chars_format fmt, 
                              int precision);

The Output

When the conversion is successful, the range [first, last) is filled with the converted string.

The returned value for all functions (for integer and floating-point support) is to_chars_result, it’s defined as follows:

struct to_chars_result {
    char* ptr;
    std::errc ec;
};

The type holds information about the conversion process:

Return Condition State of from_chars_result
Success ec equals value-initialized std::errc and ptr is the one-past-the-end pointer of the characters written. Note that the string is not NULL-terminated.
Out of range ec equals std::errc::value_too_large the range [first, last) in unspecified state.

As you can see, we have only two options: success or out of range - as there’s a chance your buffer doesn’t have enough size to hold the result.

An Example - Integer types

To sum up, here’s a basic demo of to_chars.

#include <iostream>
#include <charconv> // from_chars, to_chars
#include <string>

int main() {
    std::string str { "xxxxxxxx" };
    const int value = 1986;

    const auto res = std::to_chars(str.data(), 
                                   str.data() + str.size(), 
                                   value);

    if (res.ec == std::errc())    {
        std::cout << str << ", filled: "
            << res.ptr - str.data() << " characters\n";
    }
    else if (res.ec == std::errc::value_too_large) {
        std::cout << "value too large!\n";
    }
}

Below you can find a sample output for a set of numbers:

value value output
1986 1986xxxx, filled: 4 characters
-1986 -1986xxx, filled: 5 characters
19861986 19861986, filled: 8 characters
-19861986 value too large! (the buffer is only 8 characters)

An Example - Floating Point

On MSVC (starting from 15.9, full support in 16.0 + improvements later) we can also try the floating-point support.

std::string str{ "xxxxxxxxxxxxxxx" }; // 15 chars for float

const auto res = std::to_chars(str.data(), str.data() + str.size(),  value);

if (res.ec == std::errc())     {
    std::cout << str << ", filled: "
              << res.ptr - str.data() << " characters\n";
}
else if (res.ec == std::errc::value_too_large)     {
    std::cout << "value too large!\n";
}

Below you can find a sample output for a set of numbers:

value value format output
0.1f - 0.1xxxxxxxxxxxx, filled: 3 characters
1986.1f general 1986.1xxxxxxxxx, filled: 6 characters
1986.1f scientific 1.9861e+03xxxxx, filled: 10 characters

Benchmark & Some numbers

In my book, I did some perf experiments for integer conversions, and the new functionality is several times faster than to_string or sprintf and more than 10… or even 23x faster than stringstream versions!

I also have to check the floating-point support, but the results that I see from various places also claim order of magnitude speedup over the older techniques.

See the Stephan T. Lavavej’s talk about implementing charconv in MSVC where he shared some floating-point benchmark results.

C++20 on the Way

In C++20, we’ll have more methods that allow us to convert data into strings and format them.

The library is called std::format and is based on a popular framework {fmt}

Have a look: https://en.cppreference.com/w/cpp/utility/format

So far (Nov 2019) any of the standard library implementations don’t offer std::format, but we can try an example from the base library:

Taking some samples from https://www.zverovich.net/2019/07/23/std-format-cpp20.html:

std::vector<char> buf;
std::format_to(std::back_inserter(buf), "{}", 42);
 // buf contains "42"

You can also check out this blog post that nicely introduces you to the concepts of {fmt}:
An Extraterrestrial Guide to C++ Formatting - Fluent C++

As for the benchmarks you can read this one: http://www.zverovich.net/2013/09/07/integer-to-string-conversion-in-cplusplus.html, but unfortunately there’s no comparison with std::to_chars.

Summary

With C++17, we got new functionality that allows easy and low-level conversions between numbers and text. The new routines are potent and expose all the information you need to create advanced parsers or serialises. They won’t throw, won’t allocate, they bound check, and they offer super performance.

Read here about the corresponding from_chars method

Extra: since CppCon 2019 Microsoft opened their STL implementation, so you can even have a look at the code of charconv!

And maybe there’s hope that the code will be used in GCC and Clang library implementations, as they lack for floating post support now. Do you know any rumours about that?

I also highly suggest watching Stephan’s talk about the progress and the efforts for full charconv support. The feature looked very simple at first sight, but it appeared to be super complicated to support as the C library couldn’t be used, and everything had to be done from scratch.

Floating-Point <charconv>: Making Your Code 10x Faster With C++17’s Final Boss by Stephan T. Lavavej

Your Turn

What do you think about the new conversion routines? Have you tried them?
What other conversion utilities do you use?

C++17 In Detail
© 2017, Bartlomiej Filipek, Blogger platform
Disclaimer: Any opinions expressed herein are in no way representative of those of my employers. All data and information provided on this site is for informational purposes only. I try to write complete and accurate articles, but the web-site will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use.
This site contains ads or referral links, which provide me with a commission. Thank you for your understanding.