Optimizing Memory Management in Go with GOMEMLIMIT
Background of the Issue
At my company, we faced an issue where the Go-built Server Application I am in charge of, was crashing due to Out of Memory (OOM) errors. This issue was causing significant inconvenience in the service and had the potential to negatively impact user experience, making it a critical concern.
Cause Analysis
We observed that the OOM errors were occurring only during specific times when the server had a high volume of requests, while there were no issues when the request volume was relatively low. Through analysis with logs and monitoring tools, it was identified that the OOM errors were triggered whenever there were operations temporarily occupying a large amount of memory.
Research on GOMEMLIMIT
To overcome this issue, I researched the GOMEMLIMIT environment variable. This variable limits the maximum amount of memory a Go language program can use, in bytes. By setting this, it’s possible to control memory usage and prevent OOM situations.
GOMEMLIMIT was introduced in Go version 1.19, allowing the limitation of memory usage in the Go runtime. This feature can help enhance GC (Garbage Collection) related performance and prevent GC related Out of Memory (OOM) situations. The GOMEMLIMIT environment variable can be set, or the SetMemoryLimit function available in the runtime/debug package can be used to set the runtime memory limit.
GOMEMLIMIT restricts the amount of memory a Go process can use, enabling better control over the memory usage of Go applications, and preventing excessive memory usage that could lead to performance issues or crashes.
Setting GOMEMLIMIT provides the Go garbage collector with the maximum heap size, facilitating better regulation of the memory model. This setting defines the maximum total amount of memory the Go runtime can use, as expressed in runtime.MemStats with the expression Sys — HeapReleased or in the runtime/metrics package with the expression /memory/classes/total:bytes — /memory/classes/heap/released:bytes.
How to Set GOMEMLIMIT
The GOMEMLIMIT environment variable can be set before running a program written in Go. It is typically set in the shell as follows:
export GOMEMLIMIT=500000000 # 500MBOr it can be specified during program execution:
GOMEMLIMIT=500000000 go run main.goIn this case, if the program tries to use more than 500MB of memory, the Go runtime will limit the memory allocation.
Status After Setting
When applying it to the server, GOMEMLIMIT was set at about 80% of the available memory. After setting GOMEMLIMIT, the OOM issues were resolved. Now, the server operates stably even during times of high request volume, providing uninterrupted service to users.
