Why does golang strings.Builder implements String() like this?

Question

The implementation is

// String returns the accumulated string.
func (b *Builder) String() string {
    return *(*string)(unsafe.Pointer(&b.buf))
}

According to my test, converting []byte to string uses "copy on write", or the compiler generates the deep copy instructions if either one is changing the insided slice:

{
        a := []byte{'a'}
        s1 := string(a)
        a[0] = 'b'
        fmt.Println(s1) // a
    }

    {
        a := "a"
        b := []byte(a)
        b[0] = 'b'
        fmt.Println(a) // a
    }

So what happens if it's implemented as below?

// String returns the accumulated string.
func (b *Builder) String() string {
    return string(b.buf)
}

Paul Hankin · Accepted Answer · 2020-05-26 09:25:21Z

You can view the discussion on the changelist that introduced the strings.Builder api here: https://go-review.googlesource.com/c/go/+/74931/4/src/strings/builder.go#30

As you'd expect, it's a discussion of API mechanics, correctness, and efficiency.

If you replace the code with string(b.buf), you'll cause a copy of the built string. It may be that the compiler optimizes away the copy in simple cases of converting a byte slice to a string, but it's very unlikely that the compiler can do that here in general (because it would require a proof that the buffer inside the string builder is never used again).

Note that the (standard-library) code looks dangerous, because if you write this:

var b strings.Builder
b.WriteString("hello world")
c := b.String()
b.WriteString("a")
d := b.String()

then c and d will end up pointing to the same memory. But that's fine, because strings contain the length of their buffer. And there's no way to mutate a string, because even though in theory the memory backing the string is accessible via the buf in the strings.Builder, the only apis provided append to the backed memory.

Peter · Accepted Answer · 2020-05-26 09:16:43Z

9

Given large enough strings the type conversion requires memory allocations whereas the conversion using the unsafe package does not:

package main

import (
    "testing"
    "unsafe"
)

func BenchmarkConversion(b *testing.B) {
    buf := make([]byte, 16<<10)
    b.ResetTimer()

    for i := 0; i < b.N; i++ {
        var _ string = string(buf)
    }
}

func BenchmarkUnsafe(b *testing.B) {
    buf := make([]byte, 16<<10)
    b.ResetTimer()

    for i := 0; i < b.N; i++ {
        var _ string = *(*string)(unsafe.Pointer(&buf))
    }
}

$ go test -bench=. -benchmem
goos: linux
goarch: amd64
BenchmarkConversion-8            307087      3897 ns/op     16384 B/op     1 allocs/op
BenchmarkUnsafe-8            1000000000     0.299 ns/op         0 B/op     0 allocs/op
PASS
ok      _/tmp/tmp.KECLzZwkUn    1.579s

answered May 26, 2020 at 9:16

Peter

32.1k6 gold badges57 silver badges68 bronze badges

2 Comments

James Anderson Over a year ago

Go is developed at google so you can expect it to be optimised for a Google type environment. I.E. fast servers with lots of memory and high reliability requirements. So they will always default to safety in a "resources vs. safety" conflict. Although the nice people at golang do give you an unsafe but fast option should you feel the need.

user5728991 Over a year ago

@JamesAnderson Although the String() method uses the unsafe package, the method is safe to use. The method makes an assumption about the memory layout of strings and slices. The assumption is valid today and we can expect the Go team to update the String() method implementation should the Go team invalidate the assumption.

Collectives™ on Stack Overflow

Why does golang strings.Builder implements String() like this?

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related