Go: io.ReadAll vs io.Copy

When writing Go, we often encounter situations where we need to read files or responses from API calls. Generally, io.ReadAll and io.Copy are used for this. This post will explore the differences between them.

io.ReadAll

io.ReadAll is a function used to read all data from a Reader and return it as a byte slice ([]byte) of a fixed size containing all the data read into memory. Using io.ReadAll might be suitable for reading small amounts of data because you get a byte slice to work with directly. However, since it loads all the data into memory, it’s not suitable for reading large files.

Example

file, _ := os.OpenFile("small-file.json", os.O_RDONLY, 0664)
defer file.Close()
bodyBytes, err := io.ReadAll(file)

io.Copy

io.Copy is used for copying data from a Reader to a Writer without needing to store all the data in memory. When data is received from the Reader, it copies it to the Writer in chunks, allowing it to handle very large amounts of data without burdening memory.

Example

file, _ := os.OpenFile("small-file.json", os.O_RDONLY, 0664)
defer file.Close()
bytesBuff := new(bytes.Buffer)
_,err := io.Copy(bytesBuff, file)
bodyBytes := bytesBuff.Bytes()

Performance Comparison

Writing the test

We will test by benchmarking the reading of a JSON file.

import (
	"os"
	"bytes"
	"testing"
)

func BenchmarkReadAll(b *testing.B) {

	b.ReportAllocs()
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			file, _ := os.Open("file.json")
			defer file.Close()
			io.ReadAll(file)
		}
	})
}

func BenchmarkCopy(b *testing.B) {

	b.ReportAllocs()
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			file, _ := os.Open("file.json")
			defer file.Close()
			bytesBuff := new(bytes.Buffer)
			io.Copy(bytesBuff, file)
		}
	})
}

Test Results

Test results for reading small 10KB, medium 2.9MB, and large 26MB files.

We can see that io.Copy is, on average, about 40% more performant than io.ReadAll. However, this comes at the cost of writing more code to create a buffer to receive the data, which might not appeal to the lazy programmer 🤣🤣🤣