Memory Reservation Pools

From Solaris 11.4 SRU1 onwards, you can reserve memory for dedicated use by kernel zones. This mitigates the “not enough contiguous free memory” error when trying to “live migrate” or sometimes even boot a zone. Oracle says this failure is “rare”, but not in our experience, live migrations would invariably fail, with 512G of memory and only half the memory in use!

On the global zone, enable the “memory-reserve” facility. This will likely end up in “maintenance” mode after the refresh, and a reboot of the global zone is required:

# svccfg -s svc:/system/memory-reserve:zones setprop config/size=200g
# svccfg -s svc:/system/memory-reserve:zones setprop config/type=solaris-kz
# svccfg -s svc:/system/memory-reserve:zones setprop config/lgrps=""
# svccfg -s svc:/system/memory-reserve:zones refresh
# svcadm enable svc:/system/memory-reserve:zones
# init 6 

To move an existing zone into the MRP use the following command (if there is not enough MRP free it will fail when you reboot it):

# zonecfg -z zone1 'select capped-memory; set memory-reserve=zones; end'
# zoneadm -z zone1 shutdown -r

To determine the current size of the MRP:

# svccfg -s svc:/system/memory-reserve:zones listprop config/size
 config/size astring     200g

To check how much of Regular (non-MRP) and MRP memory is currently free, use:

# echo ::memstat|mdb -k|egrep 'Regular Free|MRP'
   Regular Free                       15278532   116.5g       24.4%/63.8%
   MRP reserved mem                    8650752      66g       13.8%/36.1%

MRP can be increased (or decreased) on the fly:

# svccfg -s svc:/system/memory-reserve:zones setprop config/size=300g
# svccfg -s svc:/system/memory-reserve:zones refresh
# svccfg -s svc:/system/memory-reserve:zones listprop config/size
 config/size astring     300g

To identify how much memory is in total use by kernel zones:

# echo ::memstat|mdb -k|grep 'Kernel Zones'
 Kernel Zones                         30418944   232.0g 48.6%

Leave a Reply

Your email address will not be published. Required fields are marked *