98

Why does the slice a remain the same? Does append() generate a new slice?

package main

import (
    "fmt"
)

var a = make([]int, 7, 8)

func Test(slice []int) {
    slice = append(slice, 100)
    fmt.Println(slice)
}

func main() {
    for i := 0; i < 7; i++ {
        a[i] = i
    }

    Test(a)
    fmt.Println(a)
}

Output:

[0 1 2 3 4 5 6 100]
[0 1 2 3 4 5 6]
3
  • 3
    This code will explain what's happening: https://play.golang.org/p/eJYq65jeqwn. func Test(slice []int), receives a copy of slice value of a. And it is pointing to the same array as a pointing. Commented Feb 2, 2019 at 20:45
  • 1
    slice is a struct that passed by value, not by reference or pointer. The equal sign just breaks the chain of those slices in Test. Commented Dec 31, 2020 at 6:13
  • As in all languages in the C family, everything in Go is [golang.org/doc/faq#pass_by_value](passed by value). That means for slice type, passing a pointer value makes a copy of the pointer, but not the data it points to. it's especially important to return the slice, since when it reallocates the resulting slice describes a completely different array. Commented Jul 8, 2021 at 8:10

15 Answers 15

70

In your example the slice argument of the Test function receives a copy of the variable a in the caller's scope.

Since a slice variable holds a "slice descriptor" which merely references an underlying array, in your Test function you modify the slice descriptor held in the slice variable several times in a row, but this does not affect the caller and its a variable.

Inside the Test function, the first append reallocates the backing array under the slice variable, copies its original contents over, appends 100 to it, and that's what you're observing. Upon exiting from Test, the slice variable goes out of scope and so does the (new) underlying array that slice references. (Jeff Lee is correct about that it's not what really happens, so the updated version follows; as he correctly states, this answer is correct, if maybe a bit too terse.)

Outside the Test function, a slice of length 7 and capacity 8 is allocated, and its 7 elements filled.
Inside the Test function, the first append sees the that the slice's capacity is still one element larger than its length — in other words, there is room for one more element to add without reallocation. So it "eats up" that remaining element and places 100 to it, after which it adjusts the length in the copy of the slice descriptor to become equal to the slice's capaticy. This does not affect the slice descriptor's in the caller's scope.

And that's what you're observing. Upon exiting from Test, the slice variable goes out of scope and so does the (new) underlying array that slice references.

If you want to make Test behave like append, you have to return the new slice from it — just like append does — and require the callers of Test to use it in the same way they would use append:

func Test(slice []int) []int {
    slice = append(slice, 100)

    fmt.Println(slice)

    return slice
}

a = Test(a)

Please read this article thoroughly as it basically shows you how to implement append by hand, after explaining how slices are working internally. Then read this.

Sign up to request clarification or add additional context in comments.

14 Comments

I actually think that this description is incorrect in subtle ways. @doun's answer below is actually a more correct representation of what's going on internally: the append in Test does not reallocate any memory, because the original allocation of the array backing slice a can still fit a single additional item. In other words, as this program is written, the return value of Test(a) and a are different slice headers with different lengths, but they point to the exact same underlying array. Printing fmt.Println(a[:cap(a)] as the last line of the main function makes this clear.
This statement is WRONG; "In your example the slice argument of the Test function receives a copy of the variable a in the caller's scope". As it mentioned in Go slice usage, func receives a pointer. Try changing slice = append(slice, 100) -> slice[1] = 13. You'll get printed [0 13 2 3 4 5 6] twice. @kostix can you explain that?. Refer - play.golang.org/p/QKRnl5CTcM1
@gihanchanuka, in the case of func Test(slice []int), the function does not receive "a pointer". In Go, everything, ever, is passed by value; just some types happen to have pointer representation or contain pointers. Slices in Go are of that latter variety: any slice value is a struct of three fields, one of which is, indeed, a pointer to the memory block holding the slice's elements.
@gihanchanuka, now the built-in append function takes a slice value and returns a slice value. In both cases it's that three-field struct, which is copied on input and on output (into the append's stack frame, and then outside of it). Now if append had to reallocate the slice's storage to make room for the data to be appended, the slice value it returned contains a pointer different from the one in the input slice value. And this happens only if append had to reallocate, and does not happen otherwise (the underlying array had unused space). That's the gist of "the problem".
@kostix I got it "+1", Thanks! This code will explain what's happening: https://play.golang.org/p/zJT7CW-pfp8. func Test(slice []int), receives a copy of slice value of a. And it is pointing to the same array as a pointing. I can't edit my above comment and removing it will make confuse of this conversation.
|
43

Typical append usage is

a = append(a, x)

because append may either modify its argument in-place or return a copy of its argument with an additional entry, depending on the size and capacity of its input. Using a slice that was previously appended to may give unexpected results, e.g.

a := []int{1,2,3}
a = append(a, 4)
fmt.Println(a)
append(a[:3], 5)
fmt.Println(a)

may print

[1 2 3 4]
[1 2 3 5]

3 Comments

Thanks, larsmans, I modified some code. "make" would give a enough capacity. The result is the same and I'm sill confused.
This needs to be _ = append(a[:3], 5) to compile now
append(a[:3], 5) is same as a[3] = 5 in the following hack. I guess this is an example where unexpected surprise happens.
15

In order to make your code work without having to return the slice from Test, you can pass a pointer like this:

package main

import (
    "fmt"
)

var a = make([]int, 7, 8)

func Test(slice *[]int) {
    *slice = append(*slice, 100)

    fmt.Println(*slice)
}

func main() {

    for i := 0; i < 7; i++ {
        a[i] = i
    }

    Test(&a)

    fmt.Println(a)
}

Comments

6

NOTICE that append generates a new slice if cap is not sufficient. @kostix's answer is correct, or you can pass slice argument by pointer!

2 Comments

You're correct about pointers but I have decidedly not mentioned them because the slices were invented mostly to free the programmers from dealing with pointers to arrays. In a reference implementation (from Go), a slice variable holds a pointer and two integers, so copying it is cheap and that's why slice = append(slice, a, b, c) is idiomatic, not passing a slice variable by pointer and modifying it "in place" so that the caller sees the change.
@kostix You're right, the purpose of the codes should be explicit. But I think the whole story is just about passing a value storing a pointer and passing a pointer pointing to a pointer. If we modify the reference, both can work, but if we replace the reference, the first one loses effects. The programmer should know what he is doing.
5

Try this, which I think makes it clear. the underlying array is changed but our slice is not, print just prints len() chars, by another slice to the cap(), you can see the changed array:

func main() {

  for i := 0; i < 7; i++ {
      a[i] = i
  }

  Test(a)

  fmt.Println(a) // prints [0..6]
  fmt.Println(a[:cap(a)] // prints [0..6,100]
}

2 Comments

so 'a' and 'a[:cap(a)]' are different slice?
Yes,if you run the code you will find out that. because cap(a) is changed during the test(a) call
3

Explanation (read inline comments):


package main

import (
    "fmt"
)

var a = make([]int, 7, 8)
// A slice is a descriptor of an array segment. 
// It consists of a pointer to the array, the length of the segment, and its capacity (the maximum length of the segment).
// The length is the number of elements referred to by the slice.
// The capacity is the number of elements in the underlying array (beginning at the element referred to by the slice pointer).
// |-> Refer to: https://blog.golang.org/go-slices-usage-and-internals -> "Slice internals" section

func Test(slice []int) {
    // slice receives a copy of slice `a` which point to the same array as slice `a`
    slice[6] = 10
    slice = append(slice, 100)
    // since `slice` capacity is 8 & length is 7, it can add 100 and make the length 8
    fmt.Println(slice, len(slice), cap(slice), " << Test 1")
    slice = append(slice, 200)
    // since `slice` capacity is 8 & length also 8, slice has to make a new slice 
    // - with double of size with point to new array (see Reference 1 below).
    // (I'm also confused, why not (n+1)*2=20). But make a new slice of 16 capacity).
    slice[6] = 13 // make sure, it's a new slice :)
    fmt.Println(slice, len(slice), cap(slice), " << Test 2")
}

func main() {
    for i := 0; i < 7; i++ {
        a[i] = i
    }

    fmt.Println(a, len(a), cap(a))
    Test(a)
    fmt.Println(a, len(a), cap(a))
    fmt.Println(a[:cap(a)], len(a), cap(a))
    // fmt.Println(a[:cap(a)+1], len(a), cap(a)) -> this'll not work
}

Output:

[0 1 2 3 4 5 6] 7 8
[0 1 2 3 4 5 10 100] 8 8  << Test 1
[0 1 2 3 4 5 13 100 200] 9 16  << Test 2
[0 1 2 3 4 5 10] 7 8
[0 1 2 3 4 5 10 100] 7 8

Reference 1: https://blog.golang.org/go-slices-usage-and-internals

func AppendByte(slice []byte, data ...byte) []byte {
    m := len(slice)
    n := m + len(data)
    if n > cap(slice) { // if necessary, reallocate
        // allocate double what's needed, for future growth.
        newSlice := make([]byte, (n+1)*2)
        copy(newSlice, slice)
        slice = newSlice
    }
    slice = slice[0:n]
    copy(slice[m:n], data)
    return slice
}

Comments

3

Answer: append() will return new underlying array if has no sufficient capacity. In your example,

var a = make([]int, 7, 8)

You allocate a slice (length is 7)of an underlying array(capacity is 8) to a, then pass it to function as parameter slice. When append() is called, it found the 1 capacity, then just update the slice's len from 7 to 8 and put the value 100 into that position.

The slice a is different to the slice slice, having different len property. len and cap are the property of slice, not the underlying array. For more details: Are slices passed by value?.

Run the example below:

package main

import (
    "fmt"
)

var a = make([]int, 7, 8)

func Test(slice []int) {
    fmt.Printf("slice's address is %p\n", &slice)
    fmt.Println("slice: cap=",cap(slice),"len=",len(slice))
    slice = append(slice, 100)
    fmt.Println("slice: cap=",cap(slice),"len=",len(slice))
    fmt.Println(slice)
}

func main() {
    for i := 0; i < 7; i++ {
        a[i] = i
    }
    fmt.Printf("a's address is %p\n", &a)
    fmt.Println("a: cap=",cap(a),"len=",len(a))
    Test(a)
    fmt.Println("a: cap=",cap(a),"len=",len(a))
    fmt.Println(a)
    fmt.Println(a[:8]) // manully extend a's len to cap of 8
}

Result is:

❯❯  Temp  17:33  go run .\test.go
a's address is 0x2cbfc0
a: cap= 8 len= 7
slice's address is 0xc000098060
slice: cap= 8 len= 7
slice: cap= 8 len= 8
[0 1 2 3 4 5 6 100]
a: cap= 8 len= 7
[0 1 2 3 4 5 6]
[0 1 2 3 4 5 6 100]

Comments

2

Go takes a more lean and lazy approach in doing this. It keeps modifying the same underlying array until the capacity of a slice is reached.

Ref: http://criticalindirection.com/2016/02/17/slice-with-a-pinch-of-salt/

Output of the example from the link explains the behavior of slices in Go.

Creating slice a.

Slice a len=7 cap=7 [0 0 0 0 0 0 0]

Slice b refers to the 2, 3, 4 indices in slice a. Hence, the capacity is 5 (= 7-2).

b := a[2:5]
Slice b len=3 cap=5 [0 0 0]

Modifying slice b, also modifies a, since they are pointing to the same underlying array.

b[0] = 9
Slice a len=7 cap=7 [0 0 9 0 0 0 0]
Slice b len=3 cap=5 [9 0 0]

Appending 1 to slice b. Overwrites a.

Slice a len=7 cap=7 [0 0 9 0 0 1 0]
Slice b len=4 cap=5 [9 0 0 1]

Appending 2 to slice b. Overwrites a.

Slice a len=7 cap=7 [0 0 9 0 0 1 2]
Slice b len=5 cap=5 [9 0 0 1 2]

Appending 3 to slice b. Here, a new copy is made as the capacity is overloaded.

Slice a len=7 cap=7 [0 0 9 0 0 1 2]
Slice b len=6 cap=12 [9 0 0 1 2 3]

Verifying slices a and b point to different underlying arrays after the capacity-overload in the previous step.

b[1] = 8
Slice a len=7 cap=7 [0 0 9 0 0 1 2]
Slice b len=6 cap=12 [9 8 0 1 2 3]

Comments

2
package main

import (
    "fmt"
)

func a() {
    x := []int{}
    x = append(x, 0)
    x = append(x, 1)  // commonTags := labelsToTags(app.Labels)
    y := append(x, 2) // Tags: append(commonTags, labelsToTags(d.Labels)...)
    z := append(x, 3) // Tags: append(commonTags, labelsToTags(d.Labels)...)
    fmt.Println(y, z)
}

func b() {
    x := []int{}
    x = append(x, 0)
    x = append(x, 1)
    x = append(x, 2)  // commonTags := labelsToTags(app.Labels)
    y := append(x, 3) // Tags: append(commonTags, labelsToTags(d.Labels)...)
    z := append(x, 4) // Tags: append(commonTags, labelsToTags(d.Labels)...)
    fmt.Println(y, z)
}

func main() {
    a()
    b()
}

First guess could be

[0, 1, 2] [0, 1, 3]
[0, 1, 2, 3] [0, 1, 2, 4]

but in fact it results in

[0, 1, 2] [0, 1, 3]
[0, 1, 2, 4] [0, 1, 2, 4]

enter image description here

enter image description here

More details see https://allegro.tech/2017/07/golang-slices-gotcha.html

Comments

2

Very simple.

  1. If the capacity of the slice is enough to append, then just use the existing underlying array.

  2. If the capacity of the slice is not enough to append, then create a new underlying array.

Comments

1

I think the original answer is not exactly correct. append() changed both the slices and the underlying array even though the underlying array is changed but still shared by both of the slices.

As specified by the Go Doc:

A slice does not store any data, it just describes a section of an underlying array. (Link)

Slices are just wrapper values around arrays, meaning that they contain information about how they slice an underlying array which they use to store a set of data. Therefore, by default, a slice, when passed to another method, is actually passed by value, instead of reference/pointer even though they will still be using the same underlying array. Normally, arrays are also passed by value too, so I assume a slice points at an underlying array instead of store it as a value. Regarding your question, when you run passed your slice to the following function:

func Test(slice []int) {
    slice = append(slice, 100)
    fmt.Println(slice)
}

you actually passed a copy of your slice along with a pointer to the same underlying array.That means, the changes you did to the slice didn't affect the one in the main function. It is the slice itself which stores the information regarding how much of an array it slices and exposes to the public. Therefore, when you ran append(slice, 1000), while expanding the underlying array, you also changed slicing information of slice too, which was kept private in your Test() function.

However, if you have changed your code as follows, it might have worked:

func main() {
    for i := 0; i < 7; i++ {
        a[i] = i
    }

    Test(a)
    fmt.Println(a[:cap(a)])
}

The reason is that you expanded a by saying a[:cap(a)] over its changed underlying array, changed by Test() function. As specified here:

You can extend a slice's length by re-slicing it, provided it has sufficient capacity. (Link)

Comments

1

Yes, when you use the append(), to add values to a Slice, it usually creates a new Slice and doesn't overwrite the original Slice.

Check the below code snippet

package main

import "fmt"

func main() {

    var ages = []int{3,6,8,1,9}

    fmt.Println(append(ages, 13))

    fmt.Println("Original Slice is still: ", ages)
}

If you need to overwrite the original Slice, you need to set the append() to the Slice name as below.

ages = append(ages, 12)
fmt.Println("Now original Slice is: ", ages)

Comments

0

Here is a nice implementation of append for slices. I guess its similar to what is going on under the hood:

package main

import "fmt"

func main() {
    slice1 := []int{0, 1, 2, 3, 4}
    slice2 := []int{55, 66, 77}
    fmt.Println(slice1)
    slice1 = Append(slice1, slice2...) // The '...' is essential!
    fmt.Println(slice1)
}

// Append ...
func Append(slice []int, items ...int) []int {
    for _, item := range items {
        slice = Extend(slice, item)
    }
    return slice
}

// Extend ...
func Extend(slice []int, element int) []int {
    n := len(slice)
    if n == cap(slice) {
        // Slice is full; must grow.
        // We double its size and add 1, so if the size is zero we still grow.
        newSlice := make([]int, len(slice), 2*len(slice)+1)
        copy(newSlice, slice)
        slice = newSlice
    }
    slice = slice[0 : n+1]
    slice[n] = element
    return slice
}

Comments

0

Append to end of a slice, or create a new entry if slice is empty

// in := [][]int{{}}, in := [][]int{{1,3},{2,3}}
// addtoEndofSliceArray(in,10)
// out=[[10]], out=[[1,3],[2,3,10]]

func addtoEndofSliceArray(in [][]int,element int)(out [][]int){
    if len(in) >0 {
        k :=in[len(in)-1]
        k = append(k,element)
        in = in[:len(in)-1]
        in = append(in, k)
    }else{
        in = [][]int{{element}}
    }
    return in
}

Comments

0

In order to get the best out of "slices" and use them properly, we need to have a good grasp of the internals of them. They're composed of three machine words:

  1. A pointer to the backing array,
  2. Current length of slice,
  3. Capacity of the backing array.

slice header

These three items together form what's called Slice Header. When you pass a slice as an argument to a function, you are actually passing a copy of this header. Therefore whatever change done on the copy of this header, is not visible to the original slice created at the main function and it remains intact.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.