1fa9e406ahrens/* 2fa9e406ahrens * CDDL HEADER START 3fa9e406ahrens * 4fa9e406ahrens * The contents of this file are subject to the terms of the 5033f983ek * Common Development and Distribution License (the "License"). 6033f983ek * You may not use this file except in compliance with the License. 7fa9e406ahrens * 8fa9e406ahrens * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE 9fa9e406ahrens * or http://www.opensolaris.org/os/licensing. 10fa9e406ahrens * See the License for the specific language governing permissions 11fa9e406ahrens * and limitations under the License. 12fa9e406ahrens * 13fa9e406ahrens * When distributing Covered Code, include this CDDL HEADER in each 14fa9e406ahrens * file and include the License file at usr/src/OPENSOLARIS.LICENSE. 15fa9e406ahrens * If applicable, add the following below this CDDL HEADER, with the 16fa9e406ahrens * fields enclosed by brackets "[]" replaced with your own identifying 17fa9e406ahrens * information: Portions Copyright [yyyy] [name of copyright owner] 18fa9e406ahrens * 19fa9e406ahrens * CDDL HEADER END 20fa9e406ahrens */ 21fa9e406ahrens/* 223f9d6adLin Ling * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. 23af1d63aPaul Dagnelie * Copyright (c) 2019, Joyent, Inc. 24fa98e48Matthew Ahrens * Copyright (c) 2011, 2018 by Delphix. All rights reserved. 2571cb1b7Saso Kiselkov * Copyright (c) 2014 by Saso Kiselkov. All rights reserved. 2601a059eRoman Strashkin * Copyright 2017 Nexenta Systems, Inc. All rights reserved. 27fa9e406ahrens */ 28fa9e406ahrens 29fa9e406ahrens/* 3044cb6abbmc * DVA-based Adjustable Replacement Cache 31fa9e406ahrens * 32ea8dc4beschrock * While much of the theory of operation used here is 33ea8dc4beschrock * based on the self-tuning, low overhead replacement cache 34fa9e406ahrens * presented by Megiddo and Modha at FAST 2003, there are some 35fa9e406ahrens * significant differences: 36fa9e406ahrens * 37fa9e406ahrens * 1. The Megiddo and Modha model assumes any page is evictable. 38fa9e406ahrens * Pages in its cache cannot be "locked" into memory. This makes 39fa9e406ahrens * the eviction algorithm simple: evict the last page in the list. 40fa9e406ahrens * This also make the performance characteristics easy to reason 41fa9e406ahrens * about. Our cache is not so simple. At any given moment, some 42fa9e406ahrens * subset of the blocks in the cache are un-evictable because we 43fa9e406ahrens * have handed out a reference to them. Blocks are only evictable 44fa9e406ahrens * when there are no external references active. This makes 45fa9e406ahrens * eviction far more problematic: we choose to evict the evictable 46fa9e406ahrens * blocks that are the "lowest" in the list. 47fa9e406ahrens * 48fa9e406ahrens * There are times when it is not possible to evict the requested 49fa9e406ahrens * space. In these circumstances we are unable to adjust the cache 50fa9e406ahrens * size. To prevent the cache growing unbounded at these times we 51fa94a07brendan * implement a "cache throttle" that slows the flow of new data 52fa94a07brendan * into the cache until we can make space available. 53fa9e406ahrens * 54fa9e406ahrens * 2. The Megiddo and Modha model assumes a fixed cache size. 55fa9e406ahrens * Pages are evicted when the cache is full and there is a cache 56fa9e406ahrens * miss. Our model has a variable sized cache. It grows with 57fa94a07brendan * high use, but also tries to react to memory pressure from the 58fa9e406ahrens * operating system: decreasing its size when system memory is 59fa9e406ahrens * tight. 60fa9e406ahrens * 61fa9e406ahrens * 3. The Megiddo and Modha model assumes a fixed page size. All 62f717074Will Andrews * elements of the cache are therefore exactly the same size. So 63fa9e406ahrens * when adjusting the cache size following a cache miss, its simply 64fa9e406ahrens * a matter of choosing a single page to evict. In our model, we 65fa9e406ahrens * have variable sized cache blocks (rangeing from 512 bytes to 66f717074Will Andrews * 128K bytes). We therefore choose a set of blocks to evict to make 67fa9e406ahrens * space for a cache miss that approximates as closely as possible 68fa9e406ahrens * the space used by the new block. 69fa9e406ahrens * 70fa9e406ahrens * See also: "ARC: A Self-Tuning, Low Overhead Replacement Cache" 71fa9e406ahrens * by N. Megiddo & D. Modha, FAST 2003 72fa9e406ahrens */ 73fa9e406ahrens 74fa9e406ahrens/* 75fa9e406ahrens * The locking model: 76fa9e406ahrens * 77fa9e406ahrens * A new reference to a cache buffer can be obtained in two 78fa9e406ahrens * ways: 1) via a hash table lookup using the DVA as a key, 79fa94a07brendan * or 2) via one of the ARC lists. The arc_read() interface 805602294Dan Kimmel * uses method 1, while the internal ARC algorithms for 81f717074Will Andrews * adjusting the cache use method 2. We therefore provide two 82fa9e406ahrens * types of locks: 1) the hash table lock array, and 2) the 835602294Dan Kimmel * ARC list locks. 84fa9e406ahrens * 85fc98feaBart Coddens * Buffers do not have their own mutexes, rather they rely on the 86fc98feaBart Coddens * hash table mutexes for the bulk of their protection (i.e. most 87fc98feaBart Coddens * fields in the arc_buf_hdr_t are protected by these mutexes). 88fa9e406ahrens * 89fa9e406ahrens * buf_hash_find() returns the appropriate mutex (held) when it 90fa9e406ahrens * locates the requested buffer in the hash table. It returns 91fa9e406ahrens * NULL for the mutex if the buffer was not in the table. 92fa9e406ahrens * 93fa9e406ahrens * buf_hash_remove() expects the appropriate hash mutex to be 94fa9e406ahrens * already held before it is invoked. 95fa9e406ahrens * 965602294Dan Kimmel * Each ARC state also has a mutex which is used to protect the 97fa9e406ahrens * buffer list associated with the state. When attempting to 985602294Dan Kimmel * obtain a hash table lock while holding an ARC list lock you 99fa9e406ahrens * must use: mutex_tryenter() to avoid deadlock. Also note that 10044eda4dmaybee * the active state mutex must be held before the ghost state mutex. 101fa9e406ahrens * 102fa9e406ahrens * Note that the majority of the performance stats are manipulated 103fa9e406ahrens * with atomic operations. 104fa94a07brendan * 10589c86e3Chris Williamson * The L2ARC uses the l2ad_mtx on each vdev for the following: 106fa94a07brendan * 107fa94a07brendan * - L2ARC buflist creation 108fa94a07brendan * - L2ARC buflist eviction 109fa94a07brendan * - L2ARC write completion, which walks L2ARC buflists 110fa94a07brendan * - ARC header destruction, as it removes from L2ARC buflists 111fa94a07brendan * - ARC header release, as it removes from L2ARC buflists 112fa9e406ahrens */ 113fa9e406ahrens 114dcbf3bdGeorge Wilson/* 115dcbf3bdGeorge Wilson * ARC operation: 116dcbf3bdGeorge Wilson * 117dcbf3bdGeorge Wilson * Every block that is in the ARC is tracked by an arc_buf_hdr_t structure. 118dcbf3bdGeorge Wilson * This structure can point either to a block that is still in the cache or to 119dcbf3bdGeorge Wilson * one that is only accessible in an L2 ARC device, or it can provide 120dcbf3bdGeorge Wilson * information about a block that was recently evicted. If a block is 121dcbf3bdGeorge Wilson * only accessible in the L2ARC, then the arc_buf_hdr_t only has enough 122dcbf3bdGeorge Wilson * information to retrieve it from the L2ARC device. This information is 123dcbf3bdGeorge Wilson * stored in the l2arc_buf_hdr_t sub-structure of the arc_buf_hdr_t. A block 124dcbf3bdGeorge Wilson * that is in this state cannot access the data directly. 125dcbf3bdGeorge Wilson * 126dcbf3bdGeorge Wilson * Blocks that are actively being referenced or have not been evicted 127dcbf3bdGeorge Wilson * are cached in the L1ARC. The L1ARC (l1arc_buf_hdr_t) is a structure within 128dcbf3bdGeorge Wilson * the arc_buf_hdr_t that will point to the data block in memory. A block can 129dcbf3bdGeorge Wilson * only be read by a consumer if it has an l1arc_buf_hdr_t. The L1ARC 1305602294Dan Kimmel * caches data in two ways -- in a list of ARC buffers (arc_buf_t) and 131770499eDan Kimmel * also in the arc_buf_hdr_t's private physical data block pointer (b_pabd). 1325602294Dan Kimmel * 1335602294Dan Kimmel * The L1ARC's data pointer may or may not be uncompressed. The ARC has the 134770499eDan Kimmel * ability to store the physical data (b_pabd) associated with the DVA of the 135770499eDan Kimmel * arc_buf_hdr_t. Since the b_pabd is a copy of the on-disk physical block, 1365602294Dan Kimmel * it will match its on-disk compression characteristics. This behavior can be 1375602294Dan Kimmel * disabled by setting 'zfs_compressed_arc_enabled' to B_FALSE. When the 138770499eDan Kimmel * compressed ARC functionality is disabled, the b_pabd will point to an 1395602294Dan Kimmel * uncompressed version of the on-disk data. 1405602294Dan Kimmel * 1415602294Dan Kimmel * Data in the L1ARC is not accessed by consumers of the ARC directly. Each 1425602294Dan Kimmel * arc_buf_hdr_t can have multiple ARC buffers (arc_buf_t) which reference it. 1435602294Dan Kimmel * Each ARC buffer (arc_buf_t) is being actively accessed by a specific ARC 1445602294Dan Kimmel * consumer. The ARC will provide references to this data and will keep it 1455602294Dan Kimmel * cached until it is no longer in use. The ARC caches only the L1ARC's physical 1465602294Dan Kimmel * data block and will evict any arc_buf_t that is no longer referenced. The 1475602294Dan Kimmel * amount of memory consumed by the arc_buf_ts' data buffers can be seen via the 148dcbf3bdGeorge Wilson * "overhead_size" kstat. 149dcbf3bdGeorge Wilson * 1505602294Dan Kimmel * Depending on the consumer, an arc_buf_t can be requested in uncompressed or 1515602294Dan Kimmel * compressed form. The typical case is that consumers will want uncompressed 1525602294Dan Kimmel * data, and when that happens a new data buffer is allocated where the data is 1535602294Dan Kimmel * decompressed for them to use. Currently the only consumer who wants 1545602294Dan Kimmel * compressed arc_buf_t's is "zfs send", when it streams data exactly as it 1555602294Dan Kimmel * exists on disk. When this happens, the arc_buf_t's data buffer is shared 1565602294Dan Kimmel * with the arc_buf_hdr_t. 157dcbf3bdGeorge Wilson * 1585602294Dan Kimmel * Here is a diagram showing an arc_buf_hdr_t referenced by two arc_buf_t's. The 1595602294Dan Kimmel * first one is owned by a compressed send consumer (and therefore references 1605602294Dan Kimmel * the same compressed data buffer as the arc_buf_hdr_t) and the second could be 1615602294Dan Kimmel * used by any other consumer (and has its own uncompressed copy of the data 1625602294Dan Kimmel * buffer). 163dcbf3bdGeorge Wilson * 1645602294Dan Kimmel * arc_buf_hdr_t 1655602294Dan Kimmel * +-----------+ 1665602294Dan Kimmel * | fields | 1675602294Dan Kimmel * | common to | 1685602294Dan Kimmel * | L1- and | 1695602294Dan Kimmel * | L2ARC | 1705602294Dan Kimmel * +-----------+ 1715602294Dan Kimmel * | l2arc_buf_hdr_t 1725602294Dan Kimmel * | | 1735602294Dan Kimmel * +-----------+ 1745602294Dan Kimmel * | l1arc_buf_hdr_t 1755602294Dan Kimmel * | | arc_buf_t 1765602294Dan Kimmel * | b_buf +------------>+-----------+ arc_buf_t 177770499eDan Kimmel * | b_pabd +-+ |b_next +---->+-----------+ 1785602294Dan Kimmel * +-----------+ | |-----------| |b_next +-->NULL 1795602294Dan Kimmel * | |b_comp = T | +-----------+ 1805602294Dan Kimmel * | |b_data +-+ |b_comp = F | 1815602294Dan Kimmel * | +-----------+ | |b_data +-+ 1825602294Dan Kimmel * +->+------+ | +-----------+ | 1835602294Dan Kimmel * compressed | | | | 1845602294Dan Kimmel * data | |<--------------+ | uncompressed 1855602294Dan Kimmel * +------+ compressed, | data 1865602294Dan Kimmel * shared +-->+------+ 1875602294Dan Kimmel * data | | 1885602294Dan Kimmel * | | 1895602294Dan Kimmel * +------+ 190dcbf3bdGeorge Wilson * 191dcbf3bdGeorge Wilson * When a consumer reads a block, the ARC must first look to see if the 1925602294Dan Kimmel * arc_buf_hdr_t is cached. If the hdr is cached then the ARC allocates a new 1935602294Dan Kimmel * arc_buf_t and either copies uncompressed data into a new data buffer from an 194770499eDan Kimmel * existing uncompressed arc_buf_t, decompresses the hdr's b_pabd buffer into a 195770499eDan Kimmel * new data buffer, or shares the hdr's b_pabd buffer, depending on whether the 1965602294Dan Kimmel * hdr is compressed and the desired compression characteristics of the 1975602294Dan Kimmel * arc_buf_t consumer. If the arc_buf_t ends up sharing data with the 1985602294Dan Kimmel * arc_buf_hdr_t and both of them are uncompressed then the arc_buf_t must be 1995602294Dan Kimmel * the last buffer in the hdr's b_buf list, however a shared compressed buf can 2005602294Dan Kimmel * be anywhere in the hdr's list. 201dcbf3bdGeorge Wilson * 202dcbf3bdGeorge Wilson * The diagram below shows an example of an uncompressed ARC hdr that is 2035602294Dan Kimmel * sharing its data with an arc_buf_t (note that the shared uncompressed buf is 2045602294Dan Kimmel * the last element in the buf list): 205dcbf3bdGeorge Wilson * 206dcbf3bdGeorge Wilson * arc_buf_hdr_t 207dcbf3bdGeorge Wilson * +-----------+ 208dcbf3bdGeorge Wilson * | | 209dcbf3bdGeorge Wilson * | | 210dcbf3bdGeorge Wilson * | | 211dcbf3bdGeorge Wilson * +-----------+ 212dcbf3bdGeorge Wilson * l2arc_buf_hdr_t| | 213dcbf3bdGeorge Wilson * | | 214dcbf3bdGeorge Wilson * +-----------+ 215dcbf3bdGeorge Wilson * l1arc_buf_hdr_t| | 216dcbf3bdGeorge Wilson * | | arc_buf_t (shared) 217dcbf3bdGeorge Wilson * | b_buf +------------>+---------+ arc_buf_t 218dcbf3bdGeorge Wilson * | | |b_next +---->+---------+ 219770499eDan Kimmel * | b_pabd +-+ |---------| |b_next +-->NULL 220dcbf3bdGeorge Wilson * +-----------+ | | | +---------+ 221dcbf3bdGeorge Wilson * | |b_data +-+ | | 222dcbf3bdGeorge Wilson * | +---------+ | |b_data +-+ 223dcbf3bdGeorge Wilson * +->+------+ | +---------+ | 224dcbf3bdGeorge Wilson * | | | | 225dcbf3bdGeorge Wilson * uncompressed | | | | 226dcbf3bdGeorge Wilson * data +------+ | | 227dcbf3bdGeorge Wilson * ^ +->+------+ | 228dcbf3bdGeorge Wilson * | uncompressed | | | 229dcbf3bdGeorge Wilson * | data | | | 230dcbf3bdGeorge Wilson * | +------+ | 231dcbf3bdGeorge Wilson * +---------------------------------+ 232dcbf3bdGeorge Wilson * 233770499eDan Kimmel * Writing to the ARC requires that the ARC first discard the hdr's b_pabd 234dcbf3bdGeorge Wilson * since the physical block is about to be rewritten. The new data contents 2355602294Dan Kimmel * will be contained in the arc_buf_t. As the I/O pipeline performs the write, 2365602294Dan Kimmel * it may compress the data before writing it to disk. The ARC will be called 2375602294Dan Kimmel * with the transformed data and will bcopy the transformed on-disk block into 238770499eDan Kimmel * a newly allocated b_pabd. Writes are always done into buffers which have 2395602294Dan Kimmel * either been loaned (and hence are new and don't have other readers) or 2405602294Dan Kimmel * buffers which have been released (and hence have their own hdr, if there 2415602294Dan Kimmel * were originally other readers of the buf's original hdr). This ensures that 2425602294Dan Kimmel * the ARC only needs to update a single buf and its hdr after a write occurs. 243dcbf3bdGeorge Wilson * 244770499eDan Kimmel * When the L2ARC is in use, it will also take advantage of the b_pabd. The 245770499eDan Kimmel * L2ARC will always write the contents of b_pabd to the L2ARC. This means 2465602294Dan Kimmel * that when compressed ARC is enabled that the L2ARC blocks are identical 247dcbf3bdGeorge Wilson * to the on-disk block in the main data pool. This provides a significant 248dcbf3bdGeorge Wilson * advantage since the ARC can leverage the bp's checksum when reading from the 249dcbf3bdGeorge Wilson * L2ARC to determine if the contents are valid. However, if the compressed 2505602294Dan Kimmel * ARC is disabled, then the L2ARC's block must be transformed to look 251dcbf3bdGeorge Wilson * like the physical block in the main data pool before comparing the 252dcbf3bdGeorge Wilson * checksum and determining its validity. 253eb63303Tom Caputi * 254eb63303Tom Caputi * The L1ARC has a slightly different system for storing encrypted data. 255eb63303Tom Caputi * Raw (encrypted + possibly compressed) data has a few subtle differences from 256eb63303Tom Caputi * data that is just compressed. The biggest difference is that it is not 257eb63303Tom Caputi * possible to decrypt encrypted data (or visa versa) if the keys aren't loaded. 258eb63303Tom Caputi * The other difference is that encryption cannot be treated as a suggestion. 259eb63303Tom Caputi * If a caller would prefer compressed data, but they actually wind up with 260eb63303Tom Caputi * uncompressed data the worst thing that could happen is there might be a 261eb63303Tom Caputi * performance hit. If the caller requests encrypted data, however, we must be 262eb63303Tom Caputi * sure they actually get it or else secret information could be leaked. Raw 263eb63303Tom Caputi * data is stored in hdr->b_crypt_hdr.b_rabd. An encrypted header, therefore, 264eb63303Tom Caputi * may have both an encrypted version and a decrypted version of its data at 265eb63303Tom Caputi * once. When a caller needs a raw arc_buf_t, it is allocated and the data is 266eb63303Tom Caputi * copied out of this header. To avoid complications with b_pabd, raw buffers 267eb63303Tom Caputi * cannot be shared. 268dcbf3bdGeorge Wilson */ 269dcbf3bdGeorge Wilson 270fa9e406ahrens#include <sys/spa.h> 271fa9e406ahrens#include <sys/zio.h> 272dcbf3bdGeorge Wilson#include <sys/spa_impl.h> 273aad0257Saso Kiselkov#include <sys/zio_compress.h> 274dcbf3bdGeorge Wilson#include <sys/zio_checksum.h> 275fa9e406ahrens#include <sys/zfs_context.h> 276fa9e406ahrens#include <sys/arc.h> 277fa9e406ahrens#include <sys/refcount.h> 278c5904d1eschrock#include <sys/vdev.h> 279573ca77George Wilson#include <sys/vdev_impl.h> 28069962b5Matthew Ahrens#include <sys/dsl_pool.h> 281770499eDan Kimmel#include <sys/zio_checksum.h> 282244781fPrakash Surya#include <sys/multilist.h> 283770499eDan Kimmel#include <sys/abd.h> 284eb63303Tom Caputi#include <sys/zil.h> 285eb63303Tom Caputi#include <sys/fm/fs/zfs.h> 286fa9e406ahrens#ifdef _KERNEL 287fa9e406ahrens#include <sys/vmsystm.h> 288fa9e406ahrens#include <vm/anon.h> 289fa9e406ahrens#include <sys/fs/swapnode.h> 290033f983ek#include <sys/dnlc.h> 291fa9e406ahrens#endif 292fa9e406ahrens#include <sys/callb.h> 29344cb6abbmc#include <sys/kstat.h> 294de753e3Brad Lewis#include <sys/zthr.h> 295b24ab67Jeff Bonwick#include <zfs_fletcher.h> 2963a2d8a1Paul Dagnelie#include <sys/aggsum.h> 2973a2d8a1Paul Dagnelie#include <sys/cityhash.h> 298af1d63aPaul Dagnelie#include <sys/param.h> 299fa9e406ahrens 300cd1c8b8Matthew Ahrens#ifndef _KERNEL 301cd1c8b8Matthew Ahrens/* set with ZFS_DEBUG=watch, to enable watchpoints on frozen buffers */ 302cd1c8b8Matthew Ahrensboolean_t arc_watch = B_FALSE; 303cd1c8b8Matthew Ahrensint arc_procfd; 304cd1c8b8Matthew Ahrens#endif 305cd1c8b8Matthew Ahrens 306de753e3Brad Lewis/* 307de753e3Brad Lewis * This thread's job is to keep enough free memory in the system, by 308de753e3Brad Lewis * calling arc_kmem_reap_now() plus arc_shrink(), which improves 309de753e3Brad Lewis * arc_available_memory(). 310de753e3Brad Lewis */ 311de753e3Brad Lewisstatic zthr_t *arc_reap_zthr; 312de753e3Brad Lewis 313de753e3Brad Lewis/* 314de753e3Brad Lewis * This thread's job is to keep arc_size under arc_c, by calling 315de753e3Brad Lewis * arc_adjust(), which improves arc_is_overflowing(). 316de753e3Brad Lewis */ 317de753e3Brad Lewisstatic zthr_t *arc_adjust_zthr; 318de753e3Brad Lewis 319de753e3Brad Lewisstatic kmutex_t arc_adjust_lock; 320de753e3Brad Lewisstatic kcondvar_t arc_adjust_waiters_cv; 321de753e3Brad Lewisstatic boolean_t arc_adjust_needed = B_FALSE; 322244781fPrakash Surya 3232ec99e3Matthew Ahrensuint_t arc_reduce_dnlc_percent = 3; 324fa9e406ahrens 32569962b5Matthew Ahrens/* 326244781fPrakash Surya * The number of headers to evict in arc_evict_state_impl() before 327244781fPrakash Surya * dropping the sublist lock and evicting from another sublist. A lower 328244781fPrakash Surya * value means we're more likely to evict the "correct" header (i.e. the 329244781fPrakash Surya * oldest header in the arc state), but comes with higher overhead 330244781fPrakash Surya * (i.e. more invocations of arc_evict_state_impl()). 331244781fPrakash Surya */ 332244781fPrakash Suryaint zfs_arc_evict_batch_limit = 10; 333244781fPrakash Surya 334fa9e406ahrens/* number of seconds before growing cache again */ 335de753e3Brad Lewisint arc_grow_retry = 60; 336fa9e406ahrens 337de753e3Brad Lewis/* 338de753e3Brad Lewis * Minimum time between calls to arc_kmem_reap_soon(). Note that this will 339de753e3Brad Lewis * be converted to ticks, so with the default hz=100, a setting of 15 ms 340de753e3Brad Lewis * will actually wait 2 ticks, or 20ms. 341de753e3Brad Lewis */ 342de753e3Brad Lewisint arc_kmem_cache_reap_retry_ms = 1000; 34336a64e6Tim Kordas 344770499eDan Kimmel/* shift of arc_c for calculating overflow limit in arc_get_data_impl */ 345de753e3Brad Lewisint zfs_arc_overflow_shift = 8; 346244781fPrakash Surya 3475a98e54Brendan Gregg - Sun Microsystems/* shift of arc_c for calculating both min and max arc_p */ 348de753e3Brad Lewisint arc_p_min_shift = 4; 3495a98e54Brendan Gregg - Sun Microsystems 3505a98e54Brendan Gregg - Sun Microsystems/* log2(fraction of arc to reclaim) */ 351de753e3Brad Lewisint arc_shrink_shift = 7; 3522ec99e3Matthew Ahrens 3532ec99e3Matthew Ahrens/* 3542ec99e3Matthew Ahrens * log2(fraction of ARC which must be free to allow growing). 3552ec99e3Matthew Ahrens * I.e. If there is less than arc_c >> arc_no_grow_shift free memory, 3562ec99e3Matthew Ahrens * when reading a new block into the ARC, we will evict an equal-sized block 3572ec99e3Matthew Ahrens * from the ARC. 3582ec99e3Matthew Ahrens * 3592ec99e3Matthew Ahrens * This must be less than arc_shrink_shift, so that when we shrink the ARC, 3602ec99e3Matthew Ahrens * we will still not allow it to grow. 3612ec99e3Matthew Ahrens */ 3622ec99e3Matthew Ahrensint arc_no_grow_shift = 5; 3632ec99e3Matthew Ahrens 3645a98e54Brendan Gregg - Sun Microsystems 36513506d1maybee/* 366b19a79eperrin * minimum lifespan of a prefetch block in clock ticks 367b19a79eperrin * (initialized in arc_init()) 36813506d1maybee */ 369a3874b8Toomas Soomestatic int zfs_arc_min_prefetch_ms = 1; 370a3874b8Toomas Soomestatic int zfs_arc_min_prescient_prefetch_ms = 6; 37113506d1maybee 37269962b5Matthew Ahrens/* 37369962b5Matthew Ahrens * If this percent of memory is free, don't throttle. 37469962b5Matthew Ahrens */ 37569962b5Matthew Ahrensint arc_lotsfree_percent = 10; 37669962b5Matthew Ahrens 377de753e3Brad Lewisstatic boolean_t arc_initialized; 378fa9e406ahrens 379fa9e406ahrens/* 3803a737e0brendan * The arc has filled available memory and has now warmed up. 3813a737e0brendan */ 3823a737e0brendanstatic boolean_t arc_warm; 3833a737e0brendan 3843a737e0brendan/* 3850dd053dPrakash Surya * log2 fraction of the zio arena to keep free. 3860dd053dPrakash Surya */ 3870dd053dPrakash Suryaint arc_zio_arena_free_shift = 2; 3880dd053dPrakash Surya 3890dd053dPrakash Surya/* 390a2eea2eahrens * These tunables are for performance analysis. 391a2eea2eahrens */ 392a2eea2eahrensuint64_t zfs_arc_max; 393a2eea2eahrensuint64_t zfs_arc_min; 3941116048ekuint64_t zfs_arc_meta_limit = 0; 3953a5286aMatthew Ahrensuint64_t zfs_arc_meta_min = 0; 3965a98e54Brendan Gregg - Sun Microsystemsint zfs_arc_grow_retry = 0; 3975a98e54Brendan Gregg - Sun Microsystemsint zfs_arc_shrink_shift = 0; 3985a98e54Brendan Gregg - Sun Microsystemsint zfs_arc_p_min_shift = 0; 39963e911bMatthew Ahrensint zfs_arc_average_blocksize = 8 * 1024; /* 8KB */ 400a2eea2eahrens 401abe1fd0Don Brady/* 402abe1fd0Don Brady * ARC dirty data constraints for arc_tempreserve_space() throttle 403abe1fd0Don Brady */ 404abe1fd0Don Bradyuint_t zfs_arc_dirty_limit_percent = 50; /* total dirty data limit */ 405abe1fd0Don Bradyuint_t zfs_arc_anon_limit_percent = 25; /* anon block dirty limit */ 406abe1fd0Don Bradyuint_t zfs_arc_pool_dirty_percent = 20; /* each pool's anon allowance */ 407abe1fd0Don Brady 408dcbf3bdGeorge Wilsonboolean_t zfs_compressed_arc_enabled = B_TRUE; 409dcbf3bdGeorge Wilson 410a2eea2eahrens/* 411fa94a07brendan * Note that buffers can be in one of 6 states: 412fa9e406ahrens * ARC_anon - anonymous (discussed below) 413ea8dc4beschrock * ARC_mru - recently used, currently cached 414ea8dc4beschrock * ARC_mru_ghost - recentely used, no longer in cache 415ea8dc4beschrock * ARC_mfu - frequently used, currently cached 416ea8dc4beschrock * ARC_mfu_ghost - frequently used, no longer in cache 417fa94a07brendan * ARC_l2c_only - exists in L2ARC but not other states 4180e8c615maybee * When there are no active references to the buffer, they are 4190e8c615maybee * are linked onto a list in one of these arc states. These are 4200e8c615maybee * the only buffers that can be evicted or deleted. Within each 4210e8c615maybee * state there are multiple lists, one for meta-data and one for 4220e8c615maybee * non-meta-data. Meta-data (indirect blocks, blocks of dnodes, 4230e8c615maybee * etc.) is tracked separately so that it can be managed more 424fa94a07brendan * explicitly: favored over data, limited explicitly. 425fa9e406ahrens * 426fa9e406ahrens * Anonymous buffers are buffers that are not associated with 427fa9e406ahrens * a DVA. These are buffers that hold dirty block copies 428fa9e406ahrens * before they are written to stable storage. By definition, 429ea8dc4beschrock * they are "ref'd" and are considered part of arc_mru 430fa9e406ahrens * that cannot be freed. Generally, they will aquire a DVA 431ea8dc4beschrock * as they are written and migrate onto the arc_mru list. 432fa94a07brendan * 433fa94a07brendan * The ARC_l2c_only state is for buffers that are in the second 434fa94a07brendan * level ARC but no longer in any of the ARC_m* lists. The second 435fa94a07brendan * level ARC itself may also contain buffers that are in any of 436fa94a07brendan * the ARC_m* states - meaning that a buffer can exist in two 437fa94a07brendan * places. The reason for the ARC_l2c_only state is to keep the 438fa94a07brendan * buffer header in the hash table, so that reads that hit the 439fa94a07brendan * second level ARC benefit from these fast lookups. 440fa9e406ahrens */ 441fa9e406ahrens 442fa9e406ahrenstypedef struct arc_state { 443244781fPrakash Surya /* 444244781fPrakash Surya * list of evictable buffers 445244781fPrakash Surya */ 44694c2d0eMatthew Ahrens multilist_t *arcs_list[ARC_BUFC_NUMTYPES]; 447244781fPrakash Surya /* 448244781fPrakash Surya * total amount of evictable data in this state 449244781fPrakash Surya */ 450e914aceTim Schumacher zfs_refcount_t arcs_esize[ARC_BUFC_NUMTYPES]; 451244781fPrakash Surya /* 452244781fPrakash Surya * total amount of data in this state; this includes: evictable, 453244781fPrakash Surya * non-evictable, ARC_BUFC_DATA, and ARC_BUFC_METADATA. 454244781fPrakash Surya */ 455e914aceTim Schumacher zfs_refcount_t arcs_size; 456fa9e406ahrens} arc_state_t; 457fa9e406ahrens 458fa94a07brendan/* The 6 states: */ 459fa9e406ahrensstatic arc_state_t ARC_anon; 460ea8dc4beschrockstatic arc_state_t ARC_mru; 461ea8dc4beschrockstatic arc_state_t ARC_mru_ghost; 462ea8dc4beschrockstatic arc_state_t ARC_mfu; 463ea8dc4beschrockstatic arc_state_t ARC_mfu_ghost; 464fa94a07brendanstatic arc_state_t ARC_l2c_only; 465fa9e406ahrens 46644cb6abbmctypedef struct arc_stats { 46744cb6abbmc kstat_named_t arcstat_hits; 46844cb6abbmc kstat_named_t arcstat_misses; 46944cb6abbmc kstat_named_t arcstat_demand_data_hits; 47044cb6abbmc kstat_named_t arcstat_demand_data_misses; 47144cb6abbmc kstat_named_t arcstat_demand_metadata_hits; 47244cb6abbmc kstat_named_t arcstat_demand_metadata_misses; 47344cb6abbmc kstat_named_t arcstat_prefetch_data_hits; 47444cb6abbmc kstat_named_t arcstat_prefetch_data_misses; 47544cb6abbmc kstat_named_t arcstat_prefetch_metadata_hits; 47644cb6abbmc kstat_named_t arcstat_prefetch_metadata_misses; 47744cb6abbmc kstat_named_t arcstat_mru_hits; 47844cb6abbmc kstat_named_t arcstat_mru_ghost_hits; 47944cb6abbmc kstat_named_t arcstat_mfu_hits; 48044cb6abbmc kstat_named_t arcstat_mfu_ghost_hits; 48144cb6abbmc kstat_named_t arcstat_deleted; 4823e30c24Will Andrews /* 4833e30c24Will Andrews * Number of buffers that could not be evicted because the hash lock 4843e30c24Will Andrews * was held by another thread. The lock may not necessarily be held 4853e30c24Will Andrews * by something using the same buffer, since hash locks are shared 4863e30c24Will Andrews * by multiple buffers. 4873e30c24Will Andrews */ 48844cb6abbmc kstat_named_t arcstat_mutex_miss; 4893e30c24Will Andrews /* 4907b38fabAlexander Motin * Number of buffers skipped when updating the access state due to the 4917b38fabAlexander Motin * header having already been released after acquiring the hash lock. 4927b38fabAlexander Motin */ 4937b38fabAlexander Motin kstat_named_t arcstat_access_skip; 4947b38fabAlexander Motin /* 4953e30c24Will Andrews * Number of buffers skipped because they have I/O in progress, are 4967b38fabAlexander Motin * indirect prefetch buffers that have not lived long enough, or are 4973e30c24Will Andrews * not from the spa we're trying to evict from. 4983e30c24Will Andrews */ 49944cb6abbmc kstat_named_t arcstat_evict_skip; 500244781fPrakash Surya /* 501244781fPrakash Surya * Number of times arc_evict_state() was unable to evict enough 502eb63303Tom Caputi * buffers to reach its target amount. 503244781fPrakash Surya */ 504244781fPrakash Surya kstat_named_t arcstat_evict_not_enough; 5055ea40c0Brendan Gregg - Sun Microsystems kstat_named_t arcstat_evict_l2_cached; 5065ea40c0Brendan Gregg - Sun Microsystems kstat_named_t arcstat_evict_l2_eligible; 5075ea40c0Brendan Gregg - Sun Microsystems kstat_named_t arcstat_evict_l2_ineligible; 508244781fPrakash Surya kstat_named_t arcstat_evict_l2_skip; 50944cb6abbmc kstat_named_t arcstat_hash_elements; 51044cb6abbmc kstat_named_t arcstat_hash_elements_max; 51144cb6abbmc kstat_named_t arcstat_hash_collisions; 51244cb6abbmc kstat_named_t arcstat_hash_chains; 51344cb6abbmc kstat_named_t arcstat_hash_chain_max; 51444cb6abbmc kstat_named_t arcstat_p; 51544cb6abbmc kstat_named_t arcstat_c; 51644cb6abbmc kstat_named_t arcstat_c_min; 51744cb6abbmc kstat_named_t arcstat_c_max; 5183a2d8a1Paul Dagnelie /* Not updated directly; only synced in arc_kstat_update. */ 51944cb6abbmc kstat_named_t arcstat_size; 5204076b1bPrakash Surya /* 521770499eDan Kimmel * Number of compressed bytes stored in the arc_buf_hdr_t's b_pabd. 522dcbf3bdGeorge Wilson * Note that the compressed bytes may match the uncompressed bytes 523dcbf3bdGeorge Wilson * if the block is either not compressed or compressed arc is disabled. 524dcbf3bdGeorge Wilson */ 525dcbf3bdGeorge Wilson kstat_named_t arcstat_compressed_size; 526dcbf3bdGeorge Wilson /* 527770499eDan Kimmel * Uncompressed size of the data stored in b_pabd. If compressed 528dcbf3bdGeorge Wilson * arc is disabled then this value will be identical to the stat 529dcbf3bdGeorge Wilson * above. 530dcbf3bdGeorge Wilson */ 531dcbf3bdGeorge Wilson kstat_named_t arcstat_uncompressed_size; 532dcbf3bdGeorge Wilson /* 533dcbf3bdGeorge Wilson * Number of bytes stored in all the arc_buf_t's. This is classified 534dcbf3bdGeorge Wilson * as "overhead" since this data is typically short-lived and will 535dcbf3bdGeorge Wilson * be evicted from the arc when it becomes unreferenced unless the 536dcbf3bdGeorge Wilson * zfs_keep_uncompressed_metadata or zfs_keep_uncompressed_level 537dcbf3bdGeorge Wilson * values have been set (see comment in dbuf.c for more information). 538dcbf3bdGeorge Wilson */ 539dcbf3bdGeorge Wilson kstat_named_t arcstat_overhead_size; 540dcbf3bdGeorge Wilson /* 5414076b1bPrakash Surya * Number of bytes consumed by internal ARC structures necessary 5424076b1bPrakash Surya * for tracking purposes; these structures are not actually 5434076b1bPrakash Surya * backed by ARC buffers. This includes arc_buf_hdr_t structures 5444076b1bPrakash Surya * (allocated via arc_buf_hdr_t_full and arc_buf_hdr_t_l2only 5454076b1bPrakash Surya * caches), and arc_buf_t structures (allocated via arc_buf_t 5464076b1bPrakash Surya * cache). 5473a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 5484076b1bPrakash Surya */ 549fa94a07brendan kstat_named_t arcstat_hdr_size; 5504076b1bPrakash Surya /* 5514076b1bPrakash Surya * Number of bytes consumed by ARC buffers of type equal to 5524076b1bPrakash Surya * ARC_BUFC_DATA. This is generally consumed by buffers backing 5534076b1bPrakash Surya * on disk user data (e.g. plain file contents). 5543a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 5554076b1bPrakash Surya */ 5565a98e54Brendan Gregg - Sun Microsystems kstat_named_t arcstat_data_size; 5574076b1bPrakash Surya /* 5584076b1bPrakash Surya * Number of bytes consumed by ARC buffers of type equal to 5594076b1bPrakash Surya * ARC_BUFC_METADATA. This is generally consumed by buffers 5604076b1bPrakash Surya * backing on disk data that is used for internal ZFS 5614076b1bPrakash Surya * structures (e.g. ZAP, dnode, indirect blocks, etc). 5623a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 5634076b1bPrakash Surya */ 5644076b1bPrakash Surya kstat_named_t arcstat_metadata_size; 5654076b1bPrakash Surya /* 5664076b1bPrakash Surya * Number of bytes consumed by various buffers and structures 5674076b1bPrakash Surya * not actually backed with ARC buffers. This includes bonus 5684076b1bPrakash Surya * buffers (allocated directly via zio_buf_* functions), 5694076b1bPrakash Surya * dmu_buf_impl_t structures (allocated via dmu_buf_impl_t 5704076b1bPrakash Surya * cache), and dnode_t structures (allocated via dnode_t cache). 5713a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 5724076b1bPrakash Surya */ 5735a98e54Brendan Gregg - Sun Microsystems kstat_named_t arcstat_other_size; 5744076b1bPrakash Surya /* 5754076b1bPrakash Surya * Total number of bytes consumed by ARC buffers residing in the 5764076b1bPrakash Surya * arc_anon state. This includes *all* buffers in the arc_anon 5774076b1bPrakash Surya * state; e.g. data, metadata, evictable, and unevictable buffers 5784076b1bPrakash Surya * are all included in this value. 5793a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 5804076b1bPrakash Surya */ 5814076b1bPrakash Surya kstat_named_t arcstat_anon_size; 5824076b1bPrakash Surya /* 5834076b1bPrakash Surya * Number of bytes consumed by ARC buffers that meet the 5844076b1bPrakash Surya * following criteria: backing buffers of type ARC_BUFC_DATA, 5854076b1bPrakash Surya * residing in the arc_anon state, and are eligible for eviction 5864076b1bPrakash Surya * (e.g. have no outstanding holds on the buffer). 5873a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 5884076b1bPrakash Surya */ 5894076b1bPrakash Surya kstat_named_t arcstat_anon_evictable_data; 5904076b1bPrakash Surya /* 5914076b1bPrakash Surya * Number of bytes consumed by ARC buffers that meet the 5924076b1bPrakash Surya * following criteria: backing buffers of type ARC_BUFC_METADATA, 5934076b1bPrakash Surya * residing in the arc_anon state, and are eligible for eviction 5944076b1bPrakash Surya * (e.g. have no outstanding holds on the buffer). 5953a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 5964076b1bPrakash Surya */ 5974076b1bPrakash Surya kstat_named_t arcstat_anon_evictable_metadata; 5984076b1bPrakash Surya /* 5994076b1bPrakash Surya * Total number of bytes consumed by ARC buffers residing in the 6004076b1bPrakash Surya * arc_mru state. This includes *all* buffers in the arc_mru 6014076b1bPrakash Surya * state; e.g. data, metadata, evictable, and unevictable buffers 6024076b1bPrakash Surya * are all included in this value. 6033a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6044076b1bPrakash Surya */ 6054076b1bPrakash Surya kstat_named_t arcstat_mru_size; 6064076b1bPrakash Surya /* 6074076b1bPrakash Surya * Number of bytes consumed by ARC buffers that meet the 6084076b1bPrakash Surya * following criteria: backing buffers of type ARC_BUFC_DATA, 6094076b1bPrakash Surya * residing in the arc_mru state, and are eligible for eviction 6104076b1bPrakash Surya * (e.g. have no outstanding holds on the buffer). 6113a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6124076b1bPrakash Surya */ 6134076b1bPrakash Surya kstat_named_t arcstat_mru_evictable_data; 6144076b1bPrakash Surya /* 6154076b1bPrakash Surya * Number of bytes consumed by ARC buffers that meet the 6164076b1bPrakash Surya * following criteria: backing buffers of type ARC_BUFC_METADATA, 6174076b1bPrakash Surya * residing in the arc_mru state, and are eligible for eviction 6184076b1bPrakash Surya * (e.g. have no outstanding holds on the buffer). 6193a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6204076b1bPrakash Surya */ 6214076b1bPrakash Surya kstat_named_t arcstat_mru_evictable_metadata; 6224076b1bPrakash Surya /* 6234076b1bPrakash Surya * Total number of bytes that *would have been* consumed by ARC 6244076b1bPrakash Surya * buffers in the arc_mru_ghost state. The key thing to note 6254076b1bPrakash Surya * here, is the fact that this size doesn't actually indicate 6264076b1bPrakash Surya * RAM consumption. The ghost lists only consist of headers and 6274076b1bPrakash Surya * don't actually have ARC buffers linked off of these headers. 6284076b1bPrakash Surya * Thus, *if* the headers had associated ARC buffers, these 6294076b1bPrakash Surya * buffers *would have* consumed this number of bytes. 6303a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6314076b1bPrakash Surya */ 6324076b1bPrakash Surya kstat_named_t arcstat_mru_ghost_size; 6334076b1bPrakash Surya /* 6344076b1bPrakash Surya * Number of bytes that *would have been* consumed by ARC 6354076b1bPrakash Surya * buffers that are eligible for eviction, of type 6364076b1bPrakash Surya * ARC_BUFC_DATA, and linked off the arc_mru_ghost state. 6373a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6384076b1bPrakash Surya */ 6394076b1bPrakash Surya kstat_named_t arcstat_mru_ghost_evictable_data; 6404076b1bPrakash Surya /* 6414076b1bPrakash Surya * Number of bytes that *would have been* consumed by ARC 6424076b1bPrakash Surya * buffers that are eligible for eviction, of type 6434076b1bPrakash Surya * ARC_BUFC_METADATA, and linked off the arc_mru_ghost state. 6443a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6454076b1bPrakash Surya */ 6464076b1bPrakash Surya kstat_named_t arcstat_mru_ghost_evictable_metadata; 6474076b1bPrakash Surya /* 6484076b1bPrakash Surya * Total number of bytes consumed by ARC buffers residing in the 6494076b1bPrakash Surya * arc_mfu state. This includes *all* buffers in the arc_mfu 6504076b1bPrakash Surya * state; e.g. data, metadata, evictable, and unevictable buffers 6514076b1bPrakash Surya * are all included in this value. 6523a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6534076b1bPrakash Surya */ 6544076b1bPrakash Surya kstat_named_t arcstat_mfu_size; 6554076b1bPrakash Surya /* 6564076b1bPrakash Surya * Number of bytes consumed by ARC buffers that are eligible for 6574076b1bPrakash Surya * eviction, of type ARC_BUFC_DATA, and reside in the arc_mfu 6584076b1bPrakash Surya * state. 6593a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6604076b1bPrakash Surya */ 6614076b1bPrakash Surya kstat_named_t arcstat_mfu_evictable_data; 6624076b1bPrakash Surya /* 6634076b1bPrakash Surya * Number of bytes consumed by ARC buffers that are eligible for 6644076b1bPrakash Surya * eviction, of type ARC_BUFC_METADATA, and reside in the 6654076b1bPrakash Surya * arc_mfu state. 6663a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6674076b1bPrakash Surya */ 6684076b1bPrakash Surya kstat_named_t arcstat_mfu_evictable_metadata; 6694076b1bPrakash Surya /* 6704076b1bPrakash Surya * Total number of bytes that *would have been* consumed by ARC 6714076b1bPrakash Surya * buffers in the arc_mfu_ghost state. See the comment above 6724076b1bPrakash Surya * arcstat_mru_ghost_size for more details. 6733a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6744076b1bPrakash Surya */ 6754076b1bPrakash Surya kstat_named_t arcstat_mfu_ghost_size; 6764076b1bPrakash Surya /* 6774076b1bPrakash Surya * Number of bytes that *would have been* consumed by ARC 6784076b1bPrakash Surya * buffers that are eligible for eviction, of type 6794076b1bPrakash Surya * ARC_BUFC_DATA, and linked off the arc_mfu_ghost state. 6803a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6814076b1bPrakash Surya */ 6824076b1bPrakash Surya kstat_named_t arcstat_mfu_ghost_evictable_data; 6834076b1bPrakash Surya /* 6844076b1bPrakash Surya * Number of bytes that *would have been* consumed by ARC 6854076b1bPrakash Surya * buffers that are eligible for eviction, of type 6864076b1bPrakash Surya * ARC_BUFC_METADATA, and linked off the arc_mru_ghost state. 6873a2d8a1Paul Dagnelie * Not updated directly; only synced in arc_kstat_update. 6884076b1bPrakash Surya */ 6894076b1bPrakash Surya kstat_named_t arcstat_mfu_ghost_evictable_metadata; 690fa94a07brendan kstat_named_t arcstat_l2_hits; 691fa94a07brendan kstat_named_t arcstat_l2_misses; 692fa94a07brendan kstat_named_t arcstat_l2_feeds; 693fa94a07brendan kstat_named_t arcstat_l2_rw_clash; 6945a98e54Brendan Gregg - Sun Microsystems kstat_named_t arcstat_l2_read_bytes; 6955a98e54Brendan Gregg - Sun Microsystems kstat_named_t arcstat_l2_write_bytes; 696fa94a07brendan kstat_named_t arcstat_l2_writes_sent; 697fa94a07brendan kstat_named_t arcstat_l2_writes_done; 698fa94a07brendan kstat_named_t arcstat_l2_writes_error; 699244781fPrakash Surya kstat_named_t arcstat_l2_writes_lock_retry; 700fa94a07brendan kstat_named_t arcstat_l2_evict_lock_retry; 701fa94a07brendan kstat_named_t arcstat_l2_evict_reading; 70289c86e3Chris Williamson kstat_named_t arcstat_l2_evict_l1cached; 703fa94a07brendan kstat_named_t arcstat_l2_free_on_write; 704fa94a07brendan kstat_named_t arcstat_l2_abort_lowmem; 705fa94a07brendan kstat_named_t arcstat_l2_cksum_bad; 706fa94a07brendan kstat_named_t arcstat_l2_io_error; 70716a7e5aAndriy Gapon kstat_named_t arcstat_l2_lsize; 70816a7e5aAndriy Gapon kstat_named_t arcstat_l2_psize; 7093a2d8a1Paul Dagnelie /* Not updated directly; only synced in arc_kstat_update. */ 710fa94a07brendan kstat_named_t arcstat_l2_hdr_size; 7111ab7f2dmaybee kstat_named_t arcstat_memory_throttle_count; 7123a2d8a1Paul Dagnelie /* Not updated directly; only synced in arc_kstat_update. */ 71320128a0George Wilson kstat_named_t arcstat_meta_used; 71420128a0George Wilson kstat_named_t arcstat_meta_limit; 71520128a0George Wilson kstat_named_t arcstat_meta_max; 7163a5286aMatthew Ahrens kstat_named_t arcstat_meta_min; 717a3874b8Toomas Soome kstat_named_t arcstat_async_upgrade_sync; 718cf6106cMatthew Ahrens kstat_named_t arcstat_demand_hit_predictive_prefetch; 719a3874b8Toomas Soome kstat_named_t arcstat_demand_hit_prescient_prefetch; 72044cb6abbmc} arc_stats_t; 72144cb6abbmc 72244cb6abbmcstatic arc_stats_t arc_stats = { 72344cb6abbmc { "hits", KSTAT_DATA_UINT64 }, 72444cb6abbmc { "misses", KSTAT_DATA_UINT64 }, 72544cb6abbmc { "demand_data_hits", KSTAT_DATA_UINT64 }, 72644cb6abbmc { "demand_data_misses", KSTAT_DATA_UINT64 }, 72744cb6abbmc { "demand_metadata_hits", KSTAT_DATA_UINT64 }, 72844cb6abbmc { "demand_metadata_misses", KSTAT_DATA_UINT64 }, 72944cb6abbmc { "prefetch_data_hits", KSTAT_DATA_UINT64 }, 73044cb6abbmc { "prefetch_data_misses", KSTAT_DATA_UINT64 }, 73144cb6abbmc { "prefetch_metadata_hits", KSTAT_DATA_UINT64 }, 73244cb6abbmc { "prefetch_metadata_misses", KSTAT_DATA_UINT64 }, 73344cb6abbmc { "mru_hits", KSTAT_DATA_UINT64 }, 73444cb6abbmc { "mru_ghost_hits", KSTAT_DATA_UINT64 }, 73544cb6abbmc { "mfu_hits", KSTAT_DATA_UINT64 }, 73644cb6abbmc { "mfu_ghost_hits", KSTAT_DATA_UINT64 }, 73744cb6abbmc { "deleted", KSTAT_DATA_UINT64 }, 73844cb6abbmc { "mutex_miss", KSTAT_DATA_UINT64 }, 7397b38fabAlexander Motin { "access_skip", KSTAT_DATA_UINT64 }, 74044cb6abbmc { "evict_skip", KSTAT_DATA_UINT64 }, 741244781fPrakash Surya { "evict_not_enough", KSTAT_DATA_UINT64 }, 7425ea40c0Brendan Gregg - Sun Microsystems { "evict_l2_cached", KSTAT_DATA_UINT64 }, 7435ea40c0Brendan Gregg - Sun Microsystems { "evict_l2_eligible", KSTAT_DATA_UINT64 }, 7445ea40c0Brendan Gregg - Sun Microsystems { "evict_l2_ineligible", KSTAT_DATA_UINT64 }, 745244781fPrakash Surya { "evict_l2_skip", KSTAT_DATA_UINT64 }, 74644cb6abbmc { "hash_elements", KSTAT_DATA_UINT64 }, 74744cb6abbmc { "hash_elements_max", KSTAT_DATA_UINT64 }, 74844cb6abbmc { "hash_collisions", KSTAT_DATA_UINT64 }, 74944cb6abbmc { "hash_chains", KSTAT_DATA_UINT64 }, 75044cb6abbmc { "hash_chain_max", KSTAT_DATA_UINT64 }, 75144cb6abbmc { "p", KSTAT_DATA_UINT64 }, 75244cb6abbmc { "c", KSTAT_DATA_UINT64 }, 75344cb6abbmc { "c_min", KSTAT_DATA_UINT64 }, 75444cb6abbmc { "c_max", KSTAT_DATA_UINT64 }, 755fa94a07brendan { "size", KSTAT_DATA_UINT64 }, 756dcbf3bdGeorge Wilson { "compressed_size", KSTAT_DATA_UINT64 }, 757dcbf3bdGeorge Wilson { "uncompressed_size", KSTAT_DATA_UINT64 }, 758dcbf3bdGeorge Wilson { "overhead_size", KSTAT_DATA_UINT64 }, 759fa94a07brendan { "hdr_size", KSTAT_DATA_UINT64 }, 7605a98e54Brendan Gregg - Sun Microsystems { "data_size", KSTAT_DATA_UINT64 }, 7614076b1bPrakash Surya { "metadata_size", KSTAT_DATA_UINT64 }, 7625a98e54Brendan Gregg - Sun Microsystems { "other_size", KSTAT_DATA_UINT64 }, 7634076b1bPrakash Surya { "anon_size", KSTAT_DATA_UINT64 }, 7644076b1bPrakash Surya { "anon_evictable_data", KSTAT_DATA_UINT64 }, 7654076b1bPrakash Surya { "anon_evictable_metadata", KSTAT_DATA_UINT64 }, 7664076b1bPrakash Surya { "mru_size", KSTAT_DATA_UINT64 }, 7674076b1bPrakash Surya { "mru_evictable_data", KSTAT_DATA_UINT64 }, 7684076b1bPrakash Surya { "mru_evictable_metadata", KSTAT_DATA_UINT64 }, 7694076b1bPrakash Surya { "mru_ghost_size", KSTAT_DATA_UINT64 }, 7704076b1bPrakash Surya { "mru_ghost_evictable_data", KSTAT_DATA_UINT64 }, 7714076b1bPrakash Surya { "mru_ghost_evictable_metadata", KSTAT_DATA_UINT64 }, 7724076b1bPrakash Surya { "mfu_size", KSTAT_DATA_UINT64 }, 773