19d26e4fcSRobert Mustacchi /* 29d26e4fcSRobert Mustacchi * This file and its contents are supplied under the terms of the 39d26e4fcSRobert Mustacchi * Common Development and Distribution License ("CDDL"), version 1.0. 49d26e4fcSRobert Mustacchi * You may only use this file in accordance with the terms of version 59d26e4fcSRobert Mustacchi * 1.0 of the CDDL. 69d26e4fcSRobert Mustacchi * 79d26e4fcSRobert Mustacchi * A full copy of the text of the CDDL should have accompanied this 89d26e4fcSRobert Mustacchi * source. A copy of the CDDL is also available via the Internet at 99d26e4fcSRobert Mustacchi * http://www.illumos.org/license/CDDL. 109d26e4fcSRobert Mustacchi */ 119d26e4fcSRobert Mustacchi 129d26e4fcSRobert Mustacchi /* 139d26e4fcSRobert Mustacchi * Copyright 2015 OmniTI Computer Consulting, Inc. All rights reserved. 1409aee612SRyan Zezeski * Copyright 2019 Joyent, Inc. 15396505afSPaul Winder * Copyright 2017 Tegile Systems, Inc. All rights reserved. 16234a3cfbSPaul Winder * Copyright 2020 RackTop Systems, Inc. 17*88628b1bSRyan Zezeski * Copyright 2020 Ryan Zezeski 189d26e4fcSRobert Mustacchi */ 199d26e4fcSRobert Mustacchi 209d26e4fcSRobert Mustacchi /* 219d26e4fcSRobert Mustacchi * i40e - Intel 10/40 Gb Ethernet driver 229d26e4fcSRobert Mustacchi * 239d26e4fcSRobert Mustacchi * The i40e driver is the main software device driver for the Intel 40 Gb family 249d26e4fcSRobert Mustacchi * of devices. Note that these devices come in many flavors with both 40 GbE 259d26e4fcSRobert Mustacchi * ports and 10 GbE ports. This device is the successor to the 82599 family of 269d26e4fcSRobert Mustacchi * devices (ixgbe). 279d26e4fcSRobert Mustacchi * 289d26e4fcSRobert Mustacchi * Unlike previous generations of Intel 1 GbE and 10 GbE devices, the 40 GbE 299d26e4fcSRobert Mustacchi * devices defined in the XL710 controller (previously known as Fortville) are a 309d26e4fcSRobert Mustacchi * rather different beast and have a small switch embedded inside of them. In 319d26e4fcSRobert Mustacchi * addition, the way that most of the programming is done has been overhauled. 329d26e4fcSRobert Mustacchi * As opposed to just using PCIe memory mapped registers, it also has an 339d26e4fcSRobert Mustacchi * administrative queue which is used to communicate with firmware running on 349d26e4fcSRobert Mustacchi * the chip. 359d26e4fcSRobert Mustacchi * 369d26e4fcSRobert Mustacchi * Each physical function in the hardware shows up as a device that this driver 379d26e4fcSRobert Mustacchi * will bind to. The hardware splits many resources evenly across all of the 389d26e4fcSRobert Mustacchi * physical functions present on the device, while other resources are instead 399d26e4fcSRobert Mustacchi * shared across the entire card and its up to the device driver to 409d26e4fcSRobert Mustacchi * intelligently partition them. 419d26e4fcSRobert Mustacchi * 429d26e4fcSRobert Mustacchi * ------------ 439d26e4fcSRobert Mustacchi * Organization 449d26e4fcSRobert Mustacchi * ------------ 459d26e4fcSRobert Mustacchi * 469d26e4fcSRobert Mustacchi * This driver is made up of several files which have their own theory 479d26e4fcSRobert Mustacchi * statements spread across them. We'll touch on the high level purpose of each 489d26e4fcSRobert Mustacchi * file here, and then we'll get into more discussion on how the device is 499d26e4fcSRobert Mustacchi * generally modelled with respect to the interfaces in illumos. 509d26e4fcSRobert Mustacchi * 519d26e4fcSRobert Mustacchi * i40e_gld.c: This file contains all of the bindings to MAC and the networking 529d26e4fcSRobert Mustacchi * stack. 539d26e4fcSRobert Mustacchi * 549d26e4fcSRobert Mustacchi * i40e_intr.c: This file contains all of the interrupt service routines and 559d26e4fcSRobert Mustacchi * contains logic to enable and disable interrupts on the hardware. 569d26e4fcSRobert Mustacchi * It also contains the logic to map hardware resources such as the 579d26e4fcSRobert Mustacchi * rings to and from interrupts and controls their ability to fire. 589d26e4fcSRobert Mustacchi * 599d26e4fcSRobert Mustacchi * There is a big theory statement on interrupts present there. 609d26e4fcSRobert Mustacchi * 619d26e4fcSRobert Mustacchi * i40e_main.c: The file that you're currently in. It interfaces with the 629d26e4fcSRobert Mustacchi * traditional OS DDI interfaces and is in charge of configuring 639d26e4fcSRobert Mustacchi * the device. 649d26e4fcSRobert Mustacchi * 659d26e4fcSRobert Mustacchi * i40e_osdep.[ch]: These files contain interfaces and definitions needed to 669d26e4fcSRobert Mustacchi * work with Intel's common code for the device. 679d26e4fcSRobert Mustacchi * 689d26e4fcSRobert Mustacchi * i40e_stats.c: This file contains the general work and logic around our 699d26e4fcSRobert Mustacchi * kstats. A theory statement on their organization and use of the 709d26e4fcSRobert Mustacchi * hardware exists there. 719d26e4fcSRobert Mustacchi * 729d26e4fcSRobert Mustacchi * i40e_sw.h: This header file contains all of the primary structure definitions 739d26e4fcSRobert Mustacchi * and constants that are used across the entire driver. 749d26e4fcSRobert Mustacchi * 759d26e4fcSRobert Mustacchi * i40e_transceiver.c: This file contains all of the logic for sending and 769d26e4fcSRobert Mustacchi * receiving data. It contains all of the ring and DMA 779d26e4fcSRobert Mustacchi * allocation logic, as well as, the actual interfaces to 789d26e4fcSRobert Mustacchi * send and receive data. 799d26e4fcSRobert Mustacchi * 809d26e4fcSRobert Mustacchi * A big theory statement on ring management, descriptors, 819d26e4fcSRobert Mustacchi * and how it ties into the OS is present there. 829d26e4fcSRobert Mustacchi * 839d26e4fcSRobert Mustacchi * -------------- 849d26e4fcSRobert Mustacchi * General Design 859d26e4fcSRobert Mustacchi * -------------- 869d26e4fcSRobert Mustacchi * 879d26e4fcSRobert Mustacchi * Before we go too far into the general way we've laid out data structures and 889d26e4fcSRobert Mustacchi * the like, it's worth taking some time to explain how the hardware is 899d26e4fcSRobert Mustacchi * organized. This organization informs a lot of how we do things at this time 909d26e4fcSRobert Mustacchi * in the driver. 919d26e4fcSRobert Mustacchi * 929d26e4fcSRobert Mustacchi * Each physical device consists of a number of one or more ports, which are 939d26e4fcSRobert Mustacchi * considered physical functions in the PCI sense and thus each get enumerated 949d26e4fcSRobert Mustacchi * by the system, resulting in an instance being created and attached to. While 959d26e4fcSRobert Mustacchi * there are many resources that are unique to each physical function eg. 969d26e4fcSRobert Mustacchi * instance of the device, there are many that are shared across all of them. 979d26e4fcSRobert Mustacchi * Several resources have an amount reserved for each Virtual Station Interface 989d26e4fcSRobert Mustacchi * (VSI) and then a static pool of resources, available for all functions on the 999d26e4fcSRobert Mustacchi * card. 1009d26e4fcSRobert Mustacchi * 1019d26e4fcSRobert Mustacchi * The most important resource in hardware are its transmit and receive queue 1029d26e4fcSRobert Mustacchi * pairs (i40e_trqpair_t). These should be thought of as rings in GLDv3 1039d26e4fcSRobert Mustacchi * parlance. There are a set number of these on each device; however, they are 1049d26e4fcSRobert Mustacchi * statically partitioned among all of the different physical functions. 1059d26e4fcSRobert Mustacchi * 1069d26e4fcSRobert Mustacchi * 'Fortville' (the code name for this device family) is basically a switch. To 1079d26e4fcSRobert Mustacchi * map MAC addresses and other things to queues, we end up having to create 1089d26e4fcSRobert Mustacchi * Virtual Station Interfaces (VSIs) and establish forwarding rules that direct 1099d26e4fcSRobert Mustacchi * traffic to a queue. A VSI owns a collection of queues and has a series of 1109d26e4fcSRobert Mustacchi * forwarding rules that point to it. One way to think of this is to treat it 1119d26e4fcSRobert Mustacchi * like MAC does a VNIC. When MAC refers to a group, a collection of rings and 1129d26e4fcSRobert Mustacchi * classification resources, that is a VSI in i40e. 1139d26e4fcSRobert Mustacchi * 1149d26e4fcSRobert Mustacchi * The sets of VSIs is shared across the entire device, though there may be some 1159d26e4fcSRobert Mustacchi * amount that are reserved to each PF. Because the GLDv3 does not let us change 1169d26e4fcSRobert Mustacchi * the number of groups dynamically, we instead statically divide this amount 1179d26e4fcSRobert Mustacchi * evenly between all the functions that exist. In addition, we have the same 1189d26e4fcSRobert Mustacchi * problem with the mac address forwarding rules. There are a static number that 1199d26e4fcSRobert Mustacchi * exist shared across all the functions. 1209d26e4fcSRobert Mustacchi * 1219d26e4fcSRobert Mustacchi * To handle both of these resources, what we end up doing is going through and 1229d26e4fcSRobert Mustacchi * determining which functions belong to the same device. Nominally one might do 1239d26e4fcSRobert Mustacchi * this by having a nexus driver; however, a prime requirement for a nexus 1249d26e4fcSRobert Mustacchi * driver is identifying the various children and activating them. While it is 1259d26e4fcSRobert Mustacchi * possible to get this information from NVRAM, we would end up duplicating a 1269d26e4fcSRobert Mustacchi * lot of the PCI enumeration logic. Really, at the end of the day, the device 1279d26e4fcSRobert Mustacchi * doesn't give us the traditional identification properties we want from a 1289d26e4fcSRobert Mustacchi * nexus driver. 1299d26e4fcSRobert Mustacchi * 1309d26e4fcSRobert Mustacchi * Instead, we rely on some properties that are guaranteed to be unique. While 1319d26e4fcSRobert Mustacchi * it might be tempting to leverage the PBA or serial number of the device from 1329d26e4fcSRobert Mustacchi * NVRAM, there is nothing that says that two devices can't be mis-programmed to 1339d26e4fcSRobert Mustacchi * have the same values in NVRAM. Instead, we uniquely identify a group of 1349d26e4fcSRobert Mustacchi * functions based on their parent in the /devices tree, their PCI bus and PCI 1359d26e4fcSRobert Mustacchi * function identifiers. Using either on their own may not be sufficient. 1369d26e4fcSRobert Mustacchi * 1379d26e4fcSRobert Mustacchi * For each unique PCI device that we encounter, we'll create a i40e_device_t. 1389d26e4fcSRobert Mustacchi * From there, because we don't have a good way to tell the GLDv3 about sharing 1399d26e4fcSRobert Mustacchi * resources between everything, we'll end up just dividing the resources 1409d26e4fcSRobert Mustacchi * evenly between all of the functions. Longer term, if we don't have to declare 1419d26e4fcSRobert Mustacchi * to the GLDv3 that these resources are shared, then we'll maintain a pool and 1427267b93fSMarcel Telka * have each PF allocate from the pool in the device, thus if only two of four 1439d26e4fcSRobert Mustacchi * ports are being used, for example, then all of the resources can still be 1449d26e4fcSRobert Mustacchi * used. 1459d26e4fcSRobert Mustacchi * 1469d26e4fcSRobert Mustacchi * ------------------------------------------- 1479d26e4fcSRobert Mustacchi * Transmit and Receive Queue Pair Allocations 1489d26e4fcSRobert Mustacchi * ------------------------------------------- 1499d26e4fcSRobert Mustacchi * 1509d26e4fcSRobert Mustacchi * NVRAM ends up assigning each PF its own share of the transmit and receive LAN 1519d26e4fcSRobert Mustacchi * queue pairs, we have no way of modifying it, only observing it. From there, 1529d26e4fcSRobert Mustacchi * it's up to us to map these queues to VSIs and VFs. Since we don't support any 1539d26e4fcSRobert Mustacchi * VFs at this time, we only focus on assignments to VSIs. 1549d26e4fcSRobert Mustacchi * 1559d26e4fcSRobert Mustacchi * At the moment, we used a static mapping of transmit/receive queue pairs to a 1569d26e4fcSRobert Mustacchi * given VSI (eg. rings to a group). Though in the fullness of time, we want to 1579d26e4fcSRobert Mustacchi * make this something which is fully dynamic and take advantage of documented, 1589d26e4fcSRobert Mustacchi * but not yet available functionality for adding filters based on VXLAN and 1599d26e4fcSRobert Mustacchi * other encapsulation technologies. 1609d26e4fcSRobert Mustacchi * 1619d26e4fcSRobert Mustacchi * ------------------------------------- 1629d26e4fcSRobert Mustacchi * Broadcast, Multicast, and Promiscuous 1639d26e4fcSRobert Mustacchi * ------------------------------------- 1649d26e4fcSRobert Mustacchi * 1659d26e4fcSRobert Mustacchi * As part of the GLDv3, we need to make sure that we can handle receiving 1669d26e4fcSRobert Mustacchi * broadcast and multicast traffic. As well as enabling promiscuous mode when 1679d26e4fcSRobert Mustacchi * requested. GLDv3 requires that all broadcast and multicast traffic be 1689d26e4fcSRobert Mustacchi * retrieved by the default group, eg. the first one. This is the same thing as 1699d26e4fcSRobert Mustacchi * the default VSI. 1709d26e4fcSRobert Mustacchi * 1719d26e4fcSRobert Mustacchi * To receieve broadcast traffic, we enable it through the admin queue, rather 1729d26e4fcSRobert Mustacchi * than use one of our filters for it. For multicast traffic, we reserve a 1739d26e4fcSRobert Mustacchi * certain number of the hash filters and assign them to a given PF. When we 1747267b93fSMarcel Telka * exceed those, we then switch to using promiscuous mode for multicast traffic. 1759d26e4fcSRobert Mustacchi * 1769d26e4fcSRobert Mustacchi * More specifically, once we exceed the number of filters (indicated because 1779d26e4fcSRobert Mustacchi * the i40e_t`i40e_resources.ifr_nmcastfilt == 1789d26e4fcSRobert Mustacchi * i40e_t`i40e_resources.ifr_nmcastfilt_used), we then instead need to toggle 1799d26e4fcSRobert Mustacchi * promiscuous mode. If promiscuous mode is toggled then we keep track of the 1809d26e4fcSRobert Mustacchi * number of MACs added to it by incrementing i40e_t`i40e_mcast_promisc_count. 1819d26e4fcSRobert Mustacchi * That will stay enabled until that count reaches zero indicating that we have 1829d26e4fcSRobert Mustacchi * only added multicast addresses that we have a corresponding entry for. 1839d26e4fcSRobert Mustacchi * 1849d26e4fcSRobert Mustacchi * Because MAC itself wants to toggle promiscuous mode, which includes both 1859d26e4fcSRobert Mustacchi * unicast and multicast traffic, we go through and keep track of that 1869d26e4fcSRobert Mustacchi * ourselves. That is maintained through the use of the i40e_t`i40e_promisc_on 1879d26e4fcSRobert Mustacchi * member. 1889d26e4fcSRobert Mustacchi * 1899d26e4fcSRobert Mustacchi * -------------- 1909d26e4fcSRobert Mustacchi * VSI Management 1919d26e4fcSRobert Mustacchi * -------------- 1929d26e4fcSRobert Mustacchi * 19309aee612SRyan Zezeski * The PFs share 384 VSIs. The firmware creates one VSI per PF by default. 19409aee612SRyan Zezeski * During chip start we retrieve the SEID of this VSI and assign it as the 19509aee612SRyan Zezeski * default VSI for our VEB (one VEB per PF). We then add additional VSIs to 19609aee612SRyan Zezeski * the VEB up to the determined number of rx groups: i40e_t`i40e_num_rx_groups. 19709aee612SRyan Zezeski * We currently cap this number to I40E_GROUP_MAX to a) make sure all PFs can 19809aee612SRyan Zezeski * allocate the same number of VSIs, and b) to keep the interrupt multiplexing 19909aee612SRyan Zezeski * under control. In the future, when we improve the interrupt allocation, we 20009aee612SRyan Zezeski * may want to revisit this cap to make better use of the available VSIs. The 20109aee612SRyan Zezeski * VSI allocation and configuration can be found in i40e_chip_start(). 2029d26e4fcSRobert Mustacchi * 2039d26e4fcSRobert Mustacchi * ---------------- 2049d26e4fcSRobert Mustacchi * Structure Layout 2059d26e4fcSRobert Mustacchi * ---------------- 2069d26e4fcSRobert Mustacchi * 2079d26e4fcSRobert Mustacchi * The following images relates the core data structures together. The primary 2089d26e4fcSRobert Mustacchi * structure in the system is the i40e_t. It itself contains multiple rings, 2099d26e4fcSRobert Mustacchi * i40e_trqpair_t's which contain the various transmit and receive data. The 2109d26e4fcSRobert Mustacchi * receive data is stored outside of the i40e_trqpair_t and instead in the 2119d26e4fcSRobert Mustacchi * i40e_rx_data_t. The i40e_t has a corresponding i40e_device_t which keeps 2129d26e4fcSRobert Mustacchi * track of per-physical device state. Finally, for every active descriptor, 2139d26e4fcSRobert Mustacchi * there is a corresponding control block, which is where the 2149d26e4fcSRobert Mustacchi * i40e_rx_control_block_t and the i40e_tx_control_block_t come from. 2159d26e4fcSRobert Mustacchi * 2169d26e4fcSRobert Mustacchi * +-----------------------+ +-----------------------+ 2179d26e4fcSRobert Mustacchi * | Global i40e_t list | | Global Device list | 2189d26e4fcSRobert Mustacchi * | | +--| | 2199d26e4fcSRobert Mustacchi * | i40e_glist | | | i40e_dlist | 2209d26e4fcSRobert Mustacchi * +-----------------------+ | +-----------------------+ 2219d26e4fcSRobert Mustacchi * | v 2229d26e4fcSRobert Mustacchi * | +------------------------+ +-----------------------+ 2239d26e4fcSRobert Mustacchi * | | Device-wide Structure |----->| Device-wide Structure |--> ... 2249d26e4fcSRobert Mustacchi * | | i40e_device_t | | i40e_device_t | 2259d26e4fcSRobert Mustacchi * | | | +-----------------------+ 2269d26e4fcSRobert Mustacchi * | | dev_info_t * ------+--> Parent in devices tree. 2279d26e4fcSRobert Mustacchi * | | uint_t ------+--> PCI bus number 2289d26e4fcSRobert Mustacchi * | | uint_t ------+--> PCI device number 2299d26e4fcSRobert Mustacchi * | | uint_t ------+--> Number of functions 2309d26e4fcSRobert Mustacchi * | | i40e_switch_rsrcs_t ---+--> Captured total switch resources 2319d26e4fcSRobert Mustacchi * | | list_t ------+-------------+ 2329d26e4fcSRobert Mustacchi * | +------------------------+ | 2339d26e4fcSRobert Mustacchi * | ^ | 2349d26e4fcSRobert Mustacchi * | +--------+ | 2359d26e4fcSRobert Mustacchi * | | v 2369d26e4fcSRobert Mustacchi * | +---------------------------+ | +-------------------+ 2379d26e4fcSRobert Mustacchi * +->| GLDv3 Device, per PF |-----|-->| GLDv3 Device (PF) |--> ... 2389d26e4fcSRobert Mustacchi * | i40e_t | | | i40e_t | 2399d26e4fcSRobert Mustacchi * | **Primary Structure** | | +-------------------+ 2409d26e4fcSRobert Mustacchi * | | | 2419d26e4fcSRobert Mustacchi * | i40e_device_t * --+-----+ 2429d26e4fcSRobert Mustacchi * | i40e_state_t --+---> Device State 2439d26e4fcSRobert Mustacchi * | i40e_hw_t --+---> Intel common code structure 2449d26e4fcSRobert Mustacchi * | mac_handle_t --+---> GLDv3 handle to MAC 2459d26e4fcSRobert Mustacchi * | ddi_periodic_t --+---> Link activity timer 24609aee612SRyan Zezeski * | i40e_vsi_t * --+---> Array of VSIs 2479d26e4fcSRobert Mustacchi * | i40e_func_rsrc_t --+---> Available hardware resources 2489d26e4fcSRobert Mustacchi * | i40e_switch_rsrc_t * --+---> Switch resource snapshot 2499d26e4fcSRobert Mustacchi * | i40e_sdu --+---> Current MTU 2509d26e4fcSRobert Mustacchi * | i40e_frame_max --+---> Current HW frame size 2519d26e4fcSRobert Mustacchi * | i40e_uaddr_t * --+---> Array of assigned unicast MACs 2529d26e4fcSRobert Mustacchi * | i40e_maddr_t * --+---> Array of assigned multicast MACs 2539d26e4fcSRobert Mustacchi * | i40e_mcast_promisccount --+---> Active multicast state 2549d26e4fcSRobert Mustacchi * | i40e_promisc_on --+---> Current promiscuous mode state 25509aee612SRyan Zezeski * | uint_t --+---> Number of transmit/receive pairs 25609aee612SRyan Zezeski * | i40e_rx_group_t * --+---> Array of Rx groups 2579d26e4fcSRobert Mustacchi * | kstat_t * --+---> PF kstats 2589d26e4fcSRobert Mustacchi * | i40e_pf_stats_t --+---> PF kstat backing data 2599d26e4fcSRobert Mustacchi * | i40e_trqpair_t * --+---------+ 2609d26e4fcSRobert Mustacchi * +---------------------------+ | 2619d26e4fcSRobert Mustacchi * | 2629d26e4fcSRobert Mustacchi * v 2639d26e4fcSRobert Mustacchi * +-------------------------------+ +-----------------------------+ 2649d26e4fcSRobert Mustacchi * | Transmit/Receive Queue Pair |-------| Transmit/Receive Queue Pair |->... 2659d26e4fcSRobert Mustacchi * | i40e_trqpair_t | | i40e_trqpair_t | 2669d26e4fcSRobert Mustacchi * + Ring Data Structure | +-----------------------------+ 2679d26e4fcSRobert Mustacchi * | | 2689d26e4fcSRobert Mustacchi * | mac_ring_handle_t +--> MAC RX ring handle 2699d26e4fcSRobert Mustacchi * | mac_ring_handle_t +--> MAC TX ring handle 2709d26e4fcSRobert Mustacchi * | i40e_rxq_stat_t --+--> RX Queue stats 2719d26e4fcSRobert Mustacchi * | i40e_txq_stat_t --+--> TX Queue stats 2729d26e4fcSRobert Mustacchi * | uint32_t (tx ring size) +--> TX Ring Size 2739d26e4fcSRobert Mustacchi * | uint32_t (tx free list size) +--> TX Free List Size 2749d26e4fcSRobert Mustacchi * | i40e_dma_buffer_t --------+--> TX Descriptor ring DMA 2759d26e4fcSRobert Mustacchi * | i40e_tx_desc_t * --------+--> TX descriptor ring 2769d26e4fcSRobert Mustacchi * | volatile unt32_t * +--> TX Write back head 2779d26e4fcSRobert Mustacchi * | uint32_t -------+--> TX ring head 2789d26e4fcSRobert Mustacchi * | uint32_t -------+--> TX ring tail 2799d26e4fcSRobert Mustacchi * | uint32_t -------+--> Num TX desc free 2809d26e4fcSRobert Mustacchi * | i40e_tx_control_block_t * --+--> TX control block array ---+ 2819d26e4fcSRobert Mustacchi * | i40e_tx_control_block_t ** --+--> TCB work list ----+ 2829d26e4fcSRobert Mustacchi * | i40e_tx_control_block_t ** --+--> TCB free list ---+ 2839d26e4fcSRobert Mustacchi * | uint32_t -------+--> Free TCB count | 2849d26e4fcSRobert Mustacchi * | i40e_rx_data_t * -------+--+ v 2859d26e4fcSRobert Mustacchi * +-------------------------------+ | +---------------------------+ 2869d26e4fcSRobert Mustacchi * | | Per-TX Frame Metadata | 2879d26e4fcSRobert Mustacchi * | | i40e_tx_control_block_t | 2889d26e4fcSRobert Mustacchi * +--------------------+ | | 2899d26e4fcSRobert Mustacchi * | mblk to transmit <--+--- mblk_t * | 2909d26e4fcSRobert Mustacchi * | type of transmit <--+--- i40e_tx_type_t | 2919d26e4fcSRobert Mustacchi * | TX DMA handle <--+--- ddi_dma_handle_t | 2929d26e4fcSRobert Mustacchi * v TX DMA buffer <--+--- i40e_dma_buffer_t | 2939d26e4fcSRobert Mustacchi * +------------------------------+ +---------------------------+ 2949d26e4fcSRobert Mustacchi * | Core Receive Data | 2959d26e4fcSRobert Mustacchi * | i40e_rx_data_t | 2969d26e4fcSRobert Mustacchi * | | 2979d26e4fcSRobert Mustacchi * | i40e_dma_buffer_t --+--> RX descriptor DMA Data 2989d26e4fcSRobert Mustacchi * | i40e_rx_desc_t --+--> RX descriptor ring 2999d26e4fcSRobert Mustacchi * | uint32_t --+--> Next free desc. 3009d26e4fcSRobert Mustacchi * | i40e_rx_control_block_t * --+--> RX Control Block Array ---+ 3019d26e4fcSRobert Mustacchi * | i40e_rx_control_block_t ** --+--> RCB work list ---+ 3029d26e4fcSRobert Mustacchi * | i40e_rx_control_block_t ** --+--> RCB free list ---+ 3039d26e4fcSRobert Mustacchi * +------------------------------+ | 3049d26e4fcSRobert Mustacchi * ^ | 3059d26e4fcSRobert Mustacchi * | +---------------------------+ | 3069d26e4fcSRobert Mustacchi * | | Per-RX Frame Metadata |<---------------+ 3079d26e4fcSRobert Mustacchi * | | i40e_rx_control_block_t | 3089d26e4fcSRobert Mustacchi * | | | 3099d26e4fcSRobert Mustacchi * | | mblk_t * ----+--> Received mblk_t data 3109d26e4fcSRobert Mustacchi * | | uint32_t ----+--> Reference count 3119d26e4fcSRobert Mustacchi * | | i40e_dma_buffer_t ----+--> Receive data DMA info 3129d26e4fcSRobert Mustacchi * | | frtn_t ----+--> mblk free function info 3139d26e4fcSRobert Mustacchi * +-----+-- i40e_rx_data_t * | 3149d26e4fcSRobert Mustacchi * +---------------------------+ 3159d26e4fcSRobert Mustacchi * 3169d26e4fcSRobert Mustacchi * ------------- 3179d26e4fcSRobert Mustacchi * Lock Ordering 3189d26e4fcSRobert Mustacchi * ------------- 3199d26e4fcSRobert Mustacchi * 3209d26e4fcSRobert Mustacchi * In order to ensure that we don't deadlock, the following represents the 3219d26e4fcSRobert Mustacchi * lock order being used. When grabbing locks, follow the following order. Lower 3229d26e4fcSRobert Mustacchi * numbers are more important. Thus, the i40e_glock which is number 0, must be 3239d26e4fcSRobert Mustacchi * taken before any other locks in the driver. On the other hand, the 3249d26e4fcSRobert Mustacchi * i40e_t`i40e_stat_lock, has the highest number because it's the least 3259d26e4fcSRobert Mustacchi * important lock. Note, that just because one lock is higher than another does 3269d26e4fcSRobert Mustacchi * not mean that all intermediary locks are required. 3279d26e4fcSRobert Mustacchi * 3289d26e4fcSRobert Mustacchi * 0) i40e_glock 3299d26e4fcSRobert Mustacchi * 1) i40e_t`i40e_general_lock 3309d26e4fcSRobert Mustacchi * 3319d26e4fcSRobert Mustacchi * 2) i40e_trqpair_t`itrq_rx_lock 3329d26e4fcSRobert Mustacchi * 3) i40e_trqpair_t`itrq_tx_lock 3339d26e4fcSRobert Mustacchi * 4) i40e_t`i40e_rx_pending_lock 3349d26e4fcSRobert Mustacchi * 5) i40e_trqpair_t`itrq_tcb_lock 3359d26e4fcSRobert Mustacchi * 3369d26e4fcSRobert Mustacchi * 6) i40e_t`i40e_stat_lock 3379d26e4fcSRobert Mustacchi * 3389d26e4fcSRobert Mustacchi * Rules and expectations: 3399d26e4fcSRobert Mustacchi * 3409d26e4fcSRobert Mustacchi * 1) A thread holding locks belong to one PF should not hold locks belonging to 3419d26e4fcSRobert Mustacchi * a second. If for some reason this becomes necessary, locks should be grabbed 3429d26e4fcSRobert Mustacchi * based on the list order in the i40e_device_t, which implies that the 3439d26e4fcSRobert Mustacchi * i40e_glock is held. 3449d26e4fcSRobert Mustacchi * 3459d26e4fcSRobert Mustacchi * 2) When grabbing locks between multiple transmit and receive queues, the 3469d26e4fcSRobert Mustacchi * locks for the lowest number transmit/receive queue should be grabbed first. 3479d26e4fcSRobert Mustacchi * 3489d26e4fcSRobert Mustacchi * 3) When grabbing both the transmit and receive lock for a given queue, always 3499d26e4fcSRobert Mustacchi * grab i40e_trqpair_t`itrq_rx_lock before the i40e_trqpair_t`itrq_tx_lock. 3509d26e4fcSRobert Mustacchi * 3519d26e4fcSRobert Mustacchi * 4) The following pairs of locks are not expected to be held at the same time: 3529d26e4fcSRobert Mustacchi * 3539d26e4fcSRobert Mustacchi * o i40e_t`i40e_rx_pending_lock and i40e_trqpair_t`itrq_tcb_lock 3549d26e4fcSRobert Mustacchi * 3559d26e4fcSRobert Mustacchi * ----------- 3569d26e4fcSRobert Mustacchi * Future Work 3579d26e4fcSRobert Mustacchi * ----------- 3589d26e4fcSRobert Mustacchi * 3599d26e4fcSRobert Mustacchi * At the moment the i40e_t driver is rather bare bones, allowing us to start 3609d26e4fcSRobert Mustacchi * getting data flowing and folks using it while we develop additional features. 3619d26e4fcSRobert Mustacchi * While bugs have been filed to cover this future work, the following gives an 3629d26e4fcSRobert Mustacchi * overview of expected work: 3639d26e4fcSRobert Mustacchi * 3649d26e4fcSRobert Mustacchi * o DMA binding and breaking up the locking in ring recycling. 3659d26e4fcSRobert Mustacchi * o Enhanced detection of device errors 3669d26e4fcSRobert Mustacchi * o Participation in IRM 3679d26e4fcSRobert Mustacchi * o FMA device reset 3689d26e4fcSRobert Mustacchi * o Stall detection, temperature error detection, etc. 3699d26e4fcSRobert Mustacchi * o More dynamic resource pools 3709d26e4fcSRobert Mustacchi */ 3719d26e4fcSRobert Mustacchi 3729d26e4fcSRobert Mustacchi #include "i40e_sw.h" 3739d26e4fcSRobert Mustacchi 37409aee612SRyan Zezeski static char i40e_ident[] = "Intel 10/40Gb Ethernet v1.0.3"; 3759d26e4fcSRobert Mustacchi 3769d26e4fcSRobert Mustacchi /* 3779d26e4fcSRobert Mustacchi * The i40e_glock primarily protects the lists below and the i40e_device_t 3789d26e4fcSRobert Mustacchi * structures. 3799d26e4fcSRobert Mustacchi */ 3809d26e4fcSRobert Mustacchi static kmutex_t i40e_glock; 3819d26e4fcSRobert Mustacchi static list_t i40e_glist; 3829d26e4fcSRobert Mustacchi static list_t i40e_dlist; 3839d26e4fcSRobert Mustacchi 3849d26e4fcSRobert Mustacchi /* 3859d26e4fcSRobert Mustacchi * Access attributes for register mapping. 3869d26e4fcSRobert Mustacchi */ 3879d26e4fcSRobert Mustacchi static ddi_device_acc_attr_t i40e_regs_acc_attr = { 3889d26e4fcSRobert Mustacchi DDI_DEVICE_ATTR_V1, 3899d26e4fcSRobert Mustacchi DDI_STRUCTURE_LE_ACC, 3909d26e4fcSRobert Mustacchi DDI_STRICTORDER_ACC, 3919d26e4fcSRobert Mustacchi DDI_FLAGERR_ACC 3929d26e4fcSRobert Mustacchi }; 3939d26e4fcSRobert Mustacchi 3949d26e4fcSRobert Mustacchi /* 3959d26e4fcSRobert Mustacchi * Logging function for this driver. 3969d26e4fcSRobert Mustacchi */ 3979d26e4fcSRobert Mustacchi static void 3989d26e4fcSRobert Mustacchi i40e_dev_err(i40e_t *i40e, int level, boolean_t console, const char *fmt, 3999d26e4fcSRobert Mustacchi va_list ap) 4009d26e4fcSRobert Mustacchi { 4019d26e4fcSRobert Mustacchi char buf[1024]; 4029d26e4fcSRobert Mustacchi 4039d26e4fcSRobert Mustacchi (void) vsnprintf(buf, sizeof (buf), fmt, ap); 4049d26e4fcSRobert Mustacchi 4059d26e4fcSRobert Mustacchi if (i40e == NULL) { 4069d26e4fcSRobert Mustacchi cmn_err(level, (console) ? "%s: %s" : "!%s: %s", 4079d26e4fcSRobert Mustacchi I40E_MODULE_NAME, buf); 4089d26e4fcSRobert Mustacchi } else { 4099d26e4fcSRobert Mustacchi dev_err(i40e->i40e_dip, level, (console) ? "%s" : "!%s", 4109d26e4fcSRobert Mustacchi buf); 4119d26e4fcSRobert Mustacchi } 4129d26e4fcSRobert Mustacchi } 4139d26e4fcSRobert Mustacchi 4149d26e4fcSRobert Mustacchi /* 4159d26e4fcSRobert Mustacchi * Because there's the stupid trailing-comma problem with the C preprocessor 4169d26e4fcSRobert Mustacchi * and variable arguments, I need to instantiate these. Pardon the redundant 4179d26e4fcSRobert Mustacchi * code. 4189d26e4fcSRobert Mustacchi */ 4199d26e4fcSRobert Mustacchi /*PRINTFLIKE2*/ 4209d26e4fcSRobert Mustacchi void 4219d26e4fcSRobert Mustacchi i40e_error(i40e_t *i40e, const char *fmt, ...) 4229d26e4fcSRobert Mustacchi { 4239d26e4fcSRobert Mustacchi va_list ap; 4249d26e4fcSRobert Mustacchi 4259d26e4fcSRobert Mustacchi va_start(ap, fmt); 4269d26e4fcSRobert Mustacchi i40e_dev_err(i40e, CE_WARN, B_FALSE, fmt, ap); 4279d26e4fcSRobert Mustacchi va_end(ap); 4289d26e4fcSRobert Mustacchi } 4299d26e4fcSRobert Mustacchi 4309d26e4fcSRobert Mustacchi /*PRINTFLIKE2*/ 4319d26e4fcSRobert Mustacchi void 4329d26e4fcSRobert Mustacchi i40e_log(i40e_t *i40e, const char *fmt, ...) 4339d26e4fcSRobert Mustacchi { 4349d26e4fcSRobert Mustacchi va_list ap; 4359d26e4fcSRobert Mustacchi 4369d26e4fcSRobert Mustacchi va_start(ap, fmt); 4379d26e4fcSRobert Mustacchi i40e_dev_err(i40e, CE_NOTE, B_FALSE, fmt, ap); 4389d26e4fcSRobert Mustacchi va_end(ap); 4399d26e4fcSRobert Mustacchi } 4409d26e4fcSRobert Mustacchi 4419d26e4fcSRobert Mustacchi /*PRINTFLIKE2*/ 4429d26e4fcSRobert Mustacchi void 4439d26e4fcSRobert Mustacchi i40e_notice(i40e_t *i40e, const char *fmt, ...) 4449d26e4fcSRobert Mustacchi { 4459d26e4fcSRobert Mustacchi va_list ap; 4469d26e4fcSRobert Mustacchi 4479d26e4fcSRobert Mustacchi va_start(ap, fmt); 4489d26e4fcSRobert Mustacchi i40e_dev_err(i40e, CE_NOTE, B_TRUE, fmt, ap); 4499d26e4fcSRobert Mustacchi va_end(ap); 4509d26e4fcSRobert Mustacchi } 4519d26e4fcSRobert Mustacchi 452b9d34b9dSRobert Mustacchi /* 453b9d34b9dSRobert Mustacchi * Various parts of the driver need to know if the controller is from the X722 454b9d34b9dSRobert Mustacchi * family, which has a few additional capabilities and different programming 455b9d34b9dSRobert Mustacchi * means. We don't consider virtual functions as part of this as they are quite 456b9d34b9dSRobert Mustacchi * different and will require substantially more work. 457b9d34b9dSRobert Mustacchi */ 458b9d34b9dSRobert Mustacchi static boolean_t 459b9d34b9dSRobert Mustacchi i40e_is_x722(i40e_t *i40e) 460b9d34b9dSRobert Mustacchi { 461b9d34b9dSRobert Mustacchi return (i40e->i40e_hw_space.mac.type == I40E_MAC_X722); 462b9d34b9dSRobert Mustacchi } 463b9d34b9dSRobert Mustacchi 4649d26e4fcSRobert Mustacchi static void 4659d26e4fcSRobert Mustacchi i40e_device_rele(i40e_t *i40e) 4669d26e4fcSRobert Mustacchi { 4679d26e4fcSRobert Mustacchi i40e_device_t *idp = i40e->i40e_device; 4689d26e4fcSRobert Mustacchi 4699d26e4fcSRobert Mustacchi if (idp == NULL) 4709d26e4fcSRobert Mustacchi return; 4719d26e4fcSRobert Mustacchi 4729d26e4fcSRobert Mustacchi mutex_enter(&i40e_glock); 4739d26e4fcSRobert Mustacchi VERIFY(idp->id_nreg > 0); 4749d26e4fcSRobert Mustacchi list_remove(&idp->id_i40e_list, i40e); 4759d26e4fcSRobert Mustacchi idp->id_nreg--; 4769d26e4fcSRobert Mustacchi if (idp->id_nreg == 0) { 4779d26e4fcSRobert Mustacchi list_remove(&i40e_dlist, idp); 4789d26e4fcSRobert Mustacchi list_destroy(&idp->id_i40e_list); 4799d26e4fcSRobert Mustacchi kmem_free(idp->id_rsrcs, sizeof (i40e_switch_rsrc_t) * 4809d26e4fcSRobert Mustacchi idp->id_rsrcs_alloc); 4819d26e4fcSRobert Mustacchi kmem_free(idp, sizeof (i40e_device_t)); 4829d26e4fcSRobert Mustacchi } 4839d26e4fcSRobert Mustacchi i40e->i40e_device = NULL; 4849d26e4fcSRobert Mustacchi mutex_exit(&i40e_glock); 4859d26e4fcSRobert Mustacchi } 4869d26e4fcSRobert Mustacchi 4879d26e4fcSRobert Mustacchi static i40e_device_t * 4889d26e4fcSRobert Mustacchi i40e_device_find(i40e_t *i40e, dev_info_t *parent, uint_t bus, uint_t device) 4899d26e4fcSRobert Mustacchi { 4909d26e4fcSRobert Mustacchi i40e_device_t *idp; 4919d26e4fcSRobert Mustacchi mutex_enter(&i40e_glock); 4929d26e4fcSRobert Mustacchi for (idp = list_head(&i40e_dlist); idp != NULL; 4939d26e4fcSRobert Mustacchi idp = list_next(&i40e_dlist, idp)) { 4949d26e4fcSRobert Mustacchi if (idp->id_parent == parent && idp->id_pci_bus == bus && 4959d26e4fcSRobert Mustacchi idp->id_pci_device == device) { 4969d26e4fcSRobert Mustacchi break; 4979d26e4fcSRobert Mustacchi } 4989d26e4fcSRobert Mustacchi } 4999d26e4fcSRobert Mustacchi 5009d26e4fcSRobert Mustacchi if (idp != NULL) { 5019d26e4fcSRobert Mustacchi VERIFY(idp->id_nreg < idp->id_nfuncs); 5029d26e4fcSRobert Mustacchi idp->id_nreg++; 5039d26e4fcSRobert Mustacchi } else { 5049d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 5059d26e4fcSRobert Mustacchi ASSERT(hw->num_ports > 0); 5069d26e4fcSRobert Mustacchi ASSERT(hw->num_partitions > 0); 5079d26e4fcSRobert Mustacchi 5089d26e4fcSRobert Mustacchi /* 5099d26e4fcSRobert Mustacchi * The Intel common code doesn't exactly keep the number of PCI 5109d26e4fcSRobert Mustacchi * functions. But it calculates it during discovery of 5119d26e4fcSRobert Mustacchi * partitions and ports. So what we do is undo the calculation 5129d26e4fcSRobert Mustacchi * that it does originally, as functions are evenly spread 5139d26e4fcSRobert Mustacchi * across ports in the rare case of partitions. 5149d26e4fcSRobert Mustacchi */ 5159d26e4fcSRobert Mustacchi idp = kmem_alloc(sizeof (i40e_device_t), KM_SLEEP); 5169d26e4fcSRobert Mustacchi idp->id_parent = parent; 5179d26e4fcSRobert Mustacchi idp->id_pci_bus = bus; 5189d26e4fcSRobert Mustacchi idp->id_pci_device = device; 5199d26e4fcSRobert Mustacchi idp->id_nfuncs = hw->num_ports * hw->num_partitions; 5209d26e4fcSRobert Mustacchi idp->id_nreg = 1; 5219d26e4fcSRobert Mustacchi idp->id_rsrcs_alloc = i40e->i40e_switch_rsrc_alloc; 5229d26e4fcSRobert Mustacchi idp->id_rsrcs_act = i40e->i40e_switch_rsrc_actual; 5239d26e4fcSRobert Mustacchi idp->id_rsrcs = kmem_alloc(sizeof (i40e_switch_rsrc_t) * 5249d26e4fcSRobert Mustacchi idp->id_rsrcs_alloc, KM_SLEEP); 5259d26e4fcSRobert Mustacchi bcopy(i40e->i40e_switch_rsrcs, idp->id_rsrcs, 5269d26e4fcSRobert Mustacchi sizeof (i40e_switch_rsrc_t) * idp->id_rsrcs_alloc); 5279d26e4fcSRobert Mustacchi list_create(&idp->id_i40e_list, sizeof (i40e_t), 5289d26e4fcSRobert Mustacchi offsetof(i40e_t, i40e_dlink)); 5299d26e4fcSRobert Mustacchi 5309d26e4fcSRobert Mustacchi list_insert_tail(&i40e_dlist, idp); 5319d26e4fcSRobert Mustacchi } 5329d26e4fcSRobert Mustacchi 5339d26e4fcSRobert Mustacchi list_insert_tail(&idp->id_i40e_list, i40e); 5349d26e4fcSRobert Mustacchi mutex_exit(&i40e_glock); 5359d26e4fcSRobert Mustacchi 5369d26e4fcSRobert Mustacchi return (idp); 5379d26e4fcSRobert Mustacchi } 5389d26e4fcSRobert Mustacchi 5399d26e4fcSRobert Mustacchi static void 5409d26e4fcSRobert Mustacchi i40e_link_state_set(i40e_t *i40e, link_state_t state) 5419d26e4fcSRobert Mustacchi { 5429d26e4fcSRobert Mustacchi if (i40e->i40e_link_state == state) 5439d26e4fcSRobert Mustacchi return; 5449d26e4fcSRobert Mustacchi 5459d26e4fcSRobert Mustacchi i40e->i40e_link_state = state; 5469d26e4fcSRobert Mustacchi mac_link_update(i40e->i40e_mac_hdl, i40e->i40e_link_state); 5479d26e4fcSRobert Mustacchi } 5489d26e4fcSRobert Mustacchi 5499d26e4fcSRobert Mustacchi /* 5509d26e4fcSRobert Mustacchi * This is a basic link check routine. Mostly we're using this just to see 5519d26e4fcSRobert Mustacchi * if we can get any accurate information about the state of the link being 5529d26e4fcSRobert Mustacchi * up or down, as well as updating the link state, speed, etc. information. 5539d26e4fcSRobert Mustacchi */ 5549d26e4fcSRobert Mustacchi void 5559d26e4fcSRobert Mustacchi i40e_link_check(i40e_t *i40e) 5569d26e4fcSRobert Mustacchi { 5579d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 5589d26e4fcSRobert Mustacchi boolean_t ls; 5599d26e4fcSRobert Mustacchi int ret; 5609d26e4fcSRobert Mustacchi 5619d26e4fcSRobert Mustacchi ASSERT(MUTEX_HELD(&i40e->i40e_general_lock)); 5629d26e4fcSRobert Mustacchi 5639d26e4fcSRobert Mustacchi hw->phy.get_link_info = B_TRUE; 5649d26e4fcSRobert Mustacchi if ((ret = i40e_get_link_status(hw, &ls)) != I40E_SUCCESS) { 5659d26e4fcSRobert Mustacchi i40e->i40e_s_link_status_errs++; 5669d26e4fcSRobert Mustacchi i40e->i40e_s_link_status_lasterr = ret; 5679d26e4fcSRobert Mustacchi return; 5689d26e4fcSRobert Mustacchi } 5699d26e4fcSRobert Mustacchi 5709d26e4fcSRobert Mustacchi /* 5719d26e4fcSRobert Mustacchi * Firmware abstracts all of the mac and phy information for us, so we 5729d26e4fcSRobert Mustacchi * can use i40e_get_link_status to determine the current state. 5739d26e4fcSRobert Mustacchi */ 5749d26e4fcSRobert Mustacchi if (ls == B_TRUE) { 5759d26e4fcSRobert Mustacchi enum i40e_aq_link_speed speed; 5769d26e4fcSRobert Mustacchi 5779d26e4fcSRobert Mustacchi speed = i40e_get_link_speed(hw); 5789d26e4fcSRobert Mustacchi 5799d26e4fcSRobert Mustacchi /* 5809d26e4fcSRobert Mustacchi * Translate from an i40e value to a value in Mbits/s. 5819d26e4fcSRobert Mustacchi */ 5829d26e4fcSRobert Mustacchi switch (speed) { 5839d26e4fcSRobert Mustacchi case I40E_LINK_SPEED_100MB: 5849d26e4fcSRobert Mustacchi i40e->i40e_link_speed = 100; 5859d26e4fcSRobert Mustacchi break; 5869d26e4fcSRobert Mustacchi case I40E_LINK_SPEED_1GB: 5879d26e4fcSRobert Mustacchi i40e->i40e_link_speed = 1000; 5889d26e4fcSRobert Mustacchi break; 5899d26e4fcSRobert Mustacchi case I40E_LINK_SPEED_10GB: 5909d26e4fcSRobert Mustacchi i40e->i40e_link_speed = 10000; 5919d26e4fcSRobert Mustacchi break; 5929d26e4fcSRobert Mustacchi case I40E_LINK_SPEED_20GB: 5939d26e4fcSRobert Mustacchi i40e->i40e_link_speed = 20000; 5949d26e4fcSRobert Mustacchi break; 5959d26e4fcSRobert Mustacchi case I40E_LINK_SPEED_40GB: 5969d26e4fcSRobert Mustacchi i40e->i40e_link_speed = 40000; 5979d26e4fcSRobert Mustacchi break; 5983d75a287SRobert Mustacchi case I40E_LINK_SPEED_25GB: 5993d75a287SRobert Mustacchi i40e->i40e_link_speed = 25000; 6003d75a287SRobert Mustacchi break; 6019d26e4fcSRobert Mustacchi default: 6029d26e4fcSRobert Mustacchi i40e->i40e_link_speed = 0; 6039d26e4fcSRobert Mustacchi break; 6049d26e4fcSRobert Mustacchi } 6059d26e4fcSRobert Mustacchi 6069d26e4fcSRobert Mustacchi /* 6079d26e4fcSRobert Mustacchi * At this time, hardware does not support half-duplex 6089d26e4fcSRobert Mustacchi * operation, hence why we don't ask the hardware about our 6099d26e4fcSRobert Mustacchi * current speed. 6109d26e4fcSRobert Mustacchi */ 6119d26e4fcSRobert Mustacchi i40e->i40e_link_duplex = LINK_DUPLEX_FULL; 6129d26e4fcSRobert Mustacchi i40e_link_state_set(i40e, LINK_STATE_UP); 6139d26e4fcSRobert Mustacchi } else { 6149d26e4fcSRobert Mustacchi i40e->i40e_link_speed = 0; 6159d26e4fcSRobert Mustacchi i40e->i40e_link_duplex = 0; 6169d26e4fcSRobert Mustacchi i40e_link_state_set(i40e, LINK_STATE_DOWN); 6179d26e4fcSRobert Mustacchi } 6189d26e4fcSRobert Mustacchi } 6199d26e4fcSRobert Mustacchi 6209d26e4fcSRobert Mustacchi static void 6219d26e4fcSRobert Mustacchi i40e_rem_intrs(i40e_t *i40e) 6229d26e4fcSRobert Mustacchi { 6239d26e4fcSRobert Mustacchi int i, rc; 6249d26e4fcSRobert Mustacchi 6259d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_intr_count; i++) { 6269d26e4fcSRobert Mustacchi rc = ddi_intr_free(i40e->i40e_intr_handles[i]); 6279d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 6289d26e4fcSRobert Mustacchi i40e_log(i40e, "failed to free interrupt %d: %d", 6299d26e4fcSRobert Mustacchi i, rc); 6309d26e4fcSRobert Mustacchi } 6319d26e4fcSRobert Mustacchi } 6329d26e4fcSRobert Mustacchi 6339d26e4fcSRobert Mustacchi kmem_free(i40e->i40e_intr_handles, i40e->i40e_intr_size); 6349d26e4fcSRobert Mustacchi i40e->i40e_intr_handles = NULL; 6359d26e4fcSRobert Mustacchi } 6369d26e4fcSRobert Mustacchi 6379d26e4fcSRobert Mustacchi static void 6389d26e4fcSRobert Mustacchi i40e_rem_intr_handlers(i40e_t *i40e) 6399d26e4fcSRobert Mustacchi { 6409d26e4fcSRobert Mustacchi int i, rc; 6419d26e4fcSRobert Mustacchi 6429d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_intr_count; i++) { 6439d26e4fcSRobert Mustacchi rc = ddi_intr_remove_handler(i40e->i40e_intr_handles[i]); 6449d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 6459d26e4fcSRobert Mustacchi i40e_log(i40e, "failed to remove interrupt %d: %d", 6469d26e4fcSRobert Mustacchi i, rc); 6479d26e4fcSRobert Mustacchi } 6489d26e4fcSRobert Mustacchi } 6499d26e4fcSRobert Mustacchi } 6509d26e4fcSRobert Mustacchi 6519d26e4fcSRobert Mustacchi /* 6529d26e4fcSRobert Mustacchi * illumos Fault Management Architecture (FMA) support. 6539d26e4fcSRobert Mustacchi */ 6549d26e4fcSRobert Mustacchi 6559d26e4fcSRobert Mustacchi int 6569d26e4fcSRobert Mustacchi i40e_check_acc_handle(ddi_acc_handle_t handle) 6579d26e4fcSRobert Mustacchi { 6589d26e4fcSRobert Mustacchi ddi_fm_error_t de; 6599d26e4fcSRobert Mustacchi 6609d26e4fcSRobert Mustacchi ddi_fm_acc_err_get(handle, &de, DDI_FME_VERSION); 6619d26e4fcSRobert Mustacchi ddi_fm_acc_err_clear(handle, DDI_FME_VERSION); 6629d26e4fcSRobert Mustacchi return (de.fme_status); 6639d26e4fcSRobert Mustacchi } 6649d26e4fcSRobert Mustacchi 6659d26e4fcSRobert Mustacchi int 6669d26e4fcSRobert Mustacchi i40e_check_dma_handle(ddi_dma_handle_t handle) 6679d26e4fcSRobert Mustacchi { 6689d26e4fcSRobert Mustacchi ddi_fm_error_t de; 6699d26e4fcSRobert Mustacchi 6709d26e4fcSRobert Mustacchi ddi_fm_dma_err_get(handle, &de, DDI_FME_VERSION); 6719d26e4fcSRobert Mustacchi return (de.fme_status); 6729d26e4fcSRobert Mustacchi } 6739d26e4fcSRobert Mustacchi 6749d26e4fcSRobert Mustacchi /* 6759d26e4fcSRobert Mustacchi * Fault service error handling callback function. 6769d26e4fcSRobert Mustacchi */ 6779d26e4fcSRobert Mustacchi /* ARGSUSED */ 6789d26e4fcSRobert Mustacchi static int 6799d26e4fcSRobert Mustacchi i40e_fm_error_cb(dev_info_t *dip, ddi_fm_error_t *err, const void *impl_data) 6809d26e4fcSRobert Mustacchi { 6819d26e4fcSRobert Mustacchi pci_ereport_post(dip, err, NULL); 6829d26e4fcSRobert Mustacchi return (err->fme_status); 6839d26e4fcSRobert Mustacchi } 6849d26e4fcSRobert Mustacchi 6859d26e4fcSRobert Mustacchi static void 6869d26e4fcSRobert Mustacchi i40e_fm_init(i40e_t *i40e) 6879d26e4fcSRobert Mustacchi { 6889d26e4fcSRobert Mustacchi ddi_iblock_cookie_t iblk; 6899d26e4fcSRobert Mustacchi 6909d26e4fcSRobert Mustacchi i40e->i40e_fm_capabilities = ddi_prop_get_int(DDI_DEV_T_ANY, 6919d26e4fcSRobert Mustacchi i40e->i40e_dip, DDI_PROP_DONTPASS, "fm_capable", 6929d26e4fcSRobert Mustacchi DDI_FM_EREPORT_CAPABLE | DDI_FM_ACCCHK_CAPABLE | 6939d26e4fcSRobert Mustacchi DDI_FM_DMACHK_CAPABLE | DDI_FM_ERRCB_CAPABLE); 6949d26e4fcSRobert Mustacchi 6959d26e4fcSRobert Mustacchi if (i40e->i40e_fm_capabilities < 0) { 6969d26e4fcSRobert Mustacchi i40e->i40e_fm_capabilities = 0; 6979d26e4fcSRobert Mustacchi } else if (i40e->i40e_fm_capabilities > 0xf) { 6989d26e4fcSRobert Mustacchi i40e->i40e_fm_capabilities = DDI_FM_EREPORT_CAPABLE | 6999d26e4fcSRobert Mustacchi DDI_FM_ACCCHK_CAPABLE | DDI_FM_DMACHK_CAPABLE | 7009d26e4fcSRobert Mustacchi DDI_FM_ERRCB_CAPABLE; 7019d26e4fcSRobert Mustacchi } 7029d26e4fcSRobert Mustacchi 7039d26e4fcSRobert Mustacchi /* 7049d26e4fcSRobert Mustacchi * Only register with IO Fault Services if we have some capability 7059d26e4fcSRobert Mustacchi */ 7069d26e4fcSRobert Mustacchi if (i40e->i40e_fm_capabilities & DDI_FM_ACCCHK_CAPABLE) { 7079d26e4fcSRobert Mustacchi i40e_regs_acc_attr.devacc_attr_access = DDI_FLAGERR_ACC; 7089d26e4fcSRobert Mustacchi } else { 7099d26e4fcSRobert Mustacchi i40e_regs_acc_attr.devacc_attr_access = DDI_DEFAULT_ACC; 7109d26e4fcSRobert Mustacchi } 7119d26e4fcSRobert Mustacchi 7129d26e4fcSRobert Mustacchi if (i40e->i40e_fm_capabilities) { 7139d26e4fcSRobert Mustacchi ddi_fm_init(i40e->i40e_dip, &i40e->i40e_fm_capabilities, &iblk); 7149d26e4fcSRobert Mustacchi 7159d26e4fcSRobert Mustacchi if (DDI_FM_EREPORT_CAP(i40e->i40e_fm_capabilities) || 7169d26e4fcSRobert Mustacchi DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities)) { 7179d26e4fcSRobert Mustacchi pci_ereport_setup(i40e->i40e_dip); 7189d26e4fcSRobert Mustacchi } 7199d26e4fcSRobert Mustacchi 7209d26e4fcSRobert Mustacchi if (DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities)) { 7219d26e4fcSRobert Mustacchi ddi_fm_handler_register(i40e->i40e_dip, 7229d26e4fcSRobert Mustacchi i40e_fm_error_cb, (void*)i40e); 7239d26e4fcSRobert Mustacchi } 7249d26e4fcSRobert Mustacchi } 7259d26e4fcSRobert Mustacchi 7269d26e4fcSRobert Mustacchi if (i40e->i40e_fm_capabilities & DDI_FM_DMACHK_CAPABLE) { 7279d26e4fcSRobert Mustacchi i40e_init_dma_attrs(i40e, B_TRUE); 7289d26e4fcSRobert Mustacchi } else { 7299d26e4fcSRobert Mustacchi i40e_init_dma_attrs(i40e, B_FALSE); 7309d26e4fcSRobert Mustacchi } 7319d26e4fcSRobert Mustacchi } 7329d26e4fcSRobert Mustacchi 7339d26e4fcSRobert Mustacchi static void 7349d26e4fcSRobert Mustacchi i40e_fm_fini(i40e_t *i40e) 7359d26e4fcSRobert Mustacchi { 7369d26e4fcSRobert Mustacchi if (i40e->i40e_fm_capabilities) { 7379d26e4fcSRobert Mustacchi 7389d26e4fcSRobert Mustacchi if (DDI_FM_EREPORT_CAP(i40e->i40e_fm_capabilities) || 7399d26e4fcSRobert Mustacchi DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities)) 7409d26e4fcSRobert Mustacchi pci_ereport_teardown(i40e->i40e_dip); 7419d26e4fcSRobert Mustacchi 7429d26e4fcSRobert Mustacchi if (DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities)) 7439d26e4fcSRobert Mustacchi ddi_fm_handler_unregister(i40e->i40e_dip); 7449d26e4fcSRobert Mustacchi 7459d26e4fcSRobert Mustacchi ddi_fm_fini(i40e->i40e_dip); 7469d26e4fcSRobert Mustacchi } 7479d26e4fcSRobert Mustacchi } 7489d26e4fcSRobert Mustacchi 7499d26e4fcSRobert Mustacchi void 7509d26e4fcSRobert Mustacchi i40e_fm_ereport(i40e_t *i40e, char *detail) 7519d26e4fcSRobert Mustacchi { 7529d26e4fcSRobert Mustacchi uint64_t ena; 7539d26e4fcSRobert Mustacchi char buf[FM_MAX_CLASS]; 7549d26e4fcSRobert Mustacchi 7559d26e4fcSRobert Mustacchi (void) snprintf(buf, FM_MAX_CLASS, "%s.%s", DDI_FM_DEVICE, detail); 7569d26e4fcSRobert Mustacchi ena = fm_ena_generate(0, FM_ENA_FMT1); 7579d26e4fcSRobert Mustacchi if (DDI_FM_EREPORT_CAP(i40e->i40e_fm_capabilities)) { 7589d26e4fcSRobert Mustacchi ddi_fm_ereport_post(i40e->i40e_dip, buf, ena, DDI_NOSLEEP, 7599d26e4fcSRobert Mustacchi FM_VERSION, DATA_TYPE_UINT8, FM_EREPORT_VERS0, NULL); 7609d26e4fcSRobert Mustacchi } 7619d26e4fcSRobert Mustacchi } 7629d26e4fcSRobert Mustacchi 7639d26e4fcSRobert Mustacchi /* 76409aee612SRyan Zezeski * Here we're trying to set the SEID of the default VSI. In general, 76509aee612SRyan Zezeski * when we come through and look at this shortly after attach, we 76609aee612SRyan Zezeski * expect there to only be a single element present, which is the 76709aee612SRyan Zezeski * default VSI. Importantly, each PF seems to not see any other 76809aee612SRyan Zezeski * devices, in part because of the simple switch mode that we're 76909aee612SRyan Zezeski * using. If for some reason, we see more artifacts, we'll need to 77009aee612SRyan Zezeski * revisit what we're doing here. 7719d26e4fcSRobert Mustacchi */ 77209aee612SRyan Zezeski static boolean_t 77309aee612SRyan Zezeski i40e_set_def_vsi_seid(i40e_t *i40e) 7749d26e4fcSRobert Mustacchi { 7759d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 7769d26e4fcSRobert Mustacchi struct i40e_aqc_get_switch_config_resp *sw_config; 7779d26e4fcSRobert Mustacchi uint8_t aq_buf[I40E_AQ_LARGE_BUF]; 7789d26e4fcSRobert Mustacchi uint16_t next = 0; 7799d26e4fcSRobert Mustacchi int rc; 7809d26e4fcSRobert Mustacchi 7819d26e4fcSRobert Mustacchi /* LINTED: E_BAD_PTR_CAST_ALIGN */ 7829d26e4fcSRobert Mustacchi sw_config = (struct i40e_aqc_get_switch_config_resp *)aq_buf; 7839d26e4fcSRobert Mustacchi rc = i40e_aq_get_switch_config(hw, sw_config, sizeof (aq_buf), &next, 7849d26e4fcSRobert Mustacchi NULL); 7859d26e4fcSRobert Mustacchi if (rc != I40E_SUCCESS) { 7869d26e4fcSRobert Mustacchi i40e_error(i40e, "i40e_aq_get_switch_config() failed %d: %d", 7879d26e4fcSRobert Mustacchi rc, hw->aq.asq_last_status); 78809aee612SRyan Zezeski return (B_FALSE); 7899d26e4fcSRobert Mustacchi } 7909d26e4fcSRobert Mustacchi 7919d26e4fcSRobert Mustacchi if (LE_16(sw_config->header.num_reported) != 1) { 7929d26e4fcSRobert Mustacchi i40e_error(i40e, "encountered multiple (%d) switching units " 7939d26e4fcSRobert Mustacchi "during attach, not proceeding", 7949d26e4fcSRobert Mustacchi LE_16(sw_config->header.num_reported)); 79509aee612SRyan Zezeski return (B_FALSE); 79609aee612SRyan Zezeski } 79709aee612SRyan Zezeski 79809aee612SRyan Zezeski I40E_DEF_VSI_SEID(i40e) = sw_config->element[0].seid; 79909aee612SRyan Zezeski return (B_TRUE); 80009aee612SRyan Zezeski } 80109aee612SRyan Zezeski 80209aee612SRyan Zezeski /* 80309aee612SRyan Zezeski * Get the SEID of the uplink MAC. 80409aee612SRyan Zezeski */ 80509aee612SRyan Zezeski static int 80609aee612SRyan Zezeski i40e_get_mac_seid(i40e_t *i40e) 80709aee612SRyan Zezeski { 80809aee612SRyan Zezeski i40e_hw_t *hw = &i40e->i40e_hw_space; 80909aee612SRyan Zezeski struct i40e_aqc_get_switch_config_resp *sw_config; 81009aee612SRyan Zezeski uint8_t aq_buf[I40E_AQ_LARGE_BUF]; 81109aee612SRyan Zezeski uint16_t next = 0; 81209aee612SRyan Zezeski int rc; 81309aee612SRyan Zezeski 81409aee612SRyan Zezeski /* LINTED: E_BAD_PTR_CAST_ALIGN */ 81509aee612SRyan Zezeski sw_config = (struct i40e_aqc_get_switch_config_resp *)aq_buf; 81609aee612SRyan Zezeski rc = i40e_aq_get_switch_config(hw, sw_config, sizeof (aq_buf), &next, 81709aee612SRyan Zezeski NULL); 81809aee612SRyan Zezeski if (rc != I40E_SUCCESS) { 81909aee612SRyan Zezeski i40e_error(i40e, "i40e_aq_get_switch_config() failed %d: %d", 82009aee612SRyan Zezeski rc, hw->aq.asq_last_status); 8219d26e4fcSRobert Mustacchi return (-1); 8229d26e4fcSRobert Mustacchi } 8239d26e4fcSRobert Mustacchi 82409aee612SRyan Zezeski return (LE_16(sw_config->element[0].uplink_seid)); 8259d26e4fcSRobert Mustacchi } 8269d26e4fcSRobert Mustacchi 8279d26e4fcSRobert Mustacchi /* 8289d26e4fcSRobert Mustacchi * We need to fill the i40e_hw_t structure with the capabilities of this PF. We 8299d26e4fcSRobert Mustacchi * must also provide the memory for it; however, we don't need to keep it around 8309d26e4fcSRobert Mustacchi * to the call to the common code. It takes it and parses it into an internal 8319d26e4fcSRobert Mustacchi * structure. 8329d26e4fcSRobert Mustacchi */ 8339d26e4fcSRobert Mustacchi static boolean_t 8349d26e4fcSRobert Mustacchi i40e_get_hw_capabilities(i40e_t *i40e, i40e_hw_t *hw) 8359d26e4fcSRobert Mustacchi { 8369d26e4fcSRobert Mustacchi struct i40e_aqc_list_capabilities_element_resp *buf; 8379d26e4fcSRobert Mustacchi int rc; 8389d26e4fcSRobert Mustacchi size_t len; 8399d26e4fcSRobert Mustacchi uint16_t needed; 8409d26e4fcSRobert Mustacchi int nelems = I40E_HW_CAP_DEFAULT; 8419d26e4fcSRobert Mustacchi 8429d26e4fcSRobert Mustacchi len = nelems * sizeof (*buf); 8439d26e4fcSRobert Mustacchi 8449d26e4fcSRobert Mustacchi for (;;) { 8459d26e4fcSRobert Mustacchi ASSERT(len > 0); 8469d26e4fcSRobert Mustacchi buf = kmem_alloc(len, KM_SLEEP); 8479d26e4fcSRobert Mustacchi rc = i40e_aq_discover_capabilities(hw, buf, len, 8489d26e4fcSRobert Mustacchi &needed, i40e_aqc_opc_list_func_capabilities, NULL); 8499d26e4fcSRobert Mustacchi kmem_free(buf, len); 8509d26e4fcSRobert Mustacchi 8519d26e4fcSRobert Mustacchi if (hw->aq.asq_last_status == I40E_AQ_RC_ENOMEM && 8529d26e4fcSRobert Mustacchi nelems == I40E_HW_CAP_DEFAULT) { 8539d26e4fcSRobert Mustacchi if (nelems == needed) { 8549d26e4fcSRobert Mustacchi i40e_error(i40e, "Capability discovery failed " 8559d26e4fcSRobert Mustacchi "due to byzantine common code"); 8569d26e4fcSRobert Mustacchi return (B_FALSE); 8579d26e4fcSRobert Mustacchi } 8589d26e4fcSRobert Mustacchi len = needed; 8599d26e4fcSRobert Mustacchi continue; 8609d26e4fcSRobert Mustacchi } else if (rc != I40E_SUCCESS || 8619d26e4fcSRobert Mustacchi hw->aq.asq_last_status != I40E_AQ_RC_OK) { 8629d26e4fcSRobert Mustacchi i40e_error(i40e, "Capability discovery failed: %d", rc); 8639d26e4fcSRobert Mustacchi return (B_FALSE); 8649d26e4fcSRobert Mustacchi } 8659d26e4fcSRobert Mustacchi 8669d26e4fcSRobert Mustacchi break; 8679d26e4fcSRobert Mustacchi } 8689d26e4fcSRobert Mustacchi 8699d26e4fcSRobert Mustacchi return (B_TRUE); 8709d26e4fcSRobert Mustacchi } 8719d26e4fcSRobert Mustacchi 8729d26e4fcSRobert Mustacchi /* 8739d26e4fcSRobert Mustacchi * Obtain the switch's capabilities as seen by this PF and keep it around for 8749d26e4fcSRobert Mustacchi * our later use. 8759d26e4fcSRobert Mustacchi */ 8769d26e4fcSRobert Mustacchi static boolean_t 8779d26e4fcSRobert Mustacchi i40e_get_switch_resources(i40e_t *i40e) 8789d26e4fcSRobert Mustacchi { 8799d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 8809d26e4fcSRobert Mustacchi uint8_t cnt = 2; 8819d26e4fcSRobert Mustacchi uint8_t act; 8829d26e4fcSRobert Mustacchi size_t size; 8839d26e4fcSRobert Mustacchi i40e_switch_rsrc_t *buf; 8849d26e4fcSRobert Mustacchi 8859d26e4fcSRobert Mustacchi for (;;) { 8869d26e4fcSRobert Mustacchi enum i40e_status_code ret; 8879d26e4fcSRobert Mustacchi size = cnt * sizeof (i40e_switch_rsrc_t); 8889d26e4fcSRobert Mustacchi ASSERT(size > 0); 8899d26e4fcSRobert Mustacchi if (size > UINT16_MAX) 8909d26e4fcSRobert Mustacchi return (B_FALSE); 8919d26e4fcSRobert Mustacchi buf = kmem_alloc(size, KM_SLEEP); 8929d26e4fcSRobert Mustacchi 8939d26e4fcSRobert Mustacchi ret = i40e_aq_get_switch_resource_alloc(hw, &act, buf, 8949d26e4fcSRobert Mustacchi cnt, NULL); 8959d26e4fcSRobert Mustacchi if (ret == I40E_ERR_ADMIN_QUEUE_ERROR && 8969d26e4fcSRobert Mustacchi hw->aq.asq_last_status == I40E_AQ_RC_EINVAL) { 8979d26e4fcSRobert Mustacchi kmem_free(buf, size); 8989d26e4fcSRobert Mustacchi cnt += I40E_SWITCH_CAP_DEFAULT; 8999d26e4fcSRobert Mustacchi continue; 9009d26e4fcSRobert Mustacchi } else if (ret != I40E_SUCCESS) { 9019d26e4fcSRobert Mustacchi kmem_free(buf, size); 9029d26e4fcSRobert Mustacchi i40e_error(i40e, 9039d26e4fcSRobert Mustacchi "failed to retrieve switch statistics: %d", ret); 9049d26e4fcSRobert Mustacchi return (B_FALSE); 9059d26e4fcSRobert Mustacchi } 9069d26e4fcSRobert Mustacchi 9079d26e4fcSRobert Mustacchi break; 9089d26e4fcSRobert Mustacchi } 9099d26e4fcSRobert Mustacchi 9109d26e4fcSRobert Mustacchi i40e->i40e_switch_rsrc_alloc = cnt; 9119d26e4fcSRobert Mustacchi i40e->i40e_switch_rsrc_actual = act; 9129d26e4fcSRobert Mustacchi i40e->i40e_switch_rsrcs = buf; 9139d26e4fcSRobert Mustacchi 9149d26e4fcSRobert Mustacchi return (B_TRUE); 9159d26e4fcSRobert Mustacchi } 9169d26e4fcSRobert Mustacchi 9179d26e4fcSRobert Mustacchi static void 9189d26e4fcSRobert Mustacchi i40e_cleanup_resources(i40e_t *i40e) 9199d26e4fcSRobert Mustacchi { 9209d26e4fcSRobert Mustacchi if (i40e->i40e_uaddrs != NULL) { 9219d26e4fcSRobert Mustacchi kmem_free(i40e->i40e_uaddrs, sizeof (i40e_uaddr_t) * 9229d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmacfilt); 9239d26e4fcSRobert Mustacchi i40e->i40e_uaddrs = NULL; 9249d26e4fcSRobert Mustacchi } 9259d26e4fcSRobert Mustacchi 9269d26e4fcSRobert Mustacchi if (i40e->i40e_maddrs != NULL) { 9279d26e4fcSRobert Mustacchi kmem_free(i40e->i40e_maddrs, sizeof (i40e_maddr_t) * 9289d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmcastfilt); 9299d26e4fcSRobert Mustacchi i40e->i40e_maddrs = NULL; 9309d26e4fcSRobert Mustacchi } 9319d26e4fcSRobert Mustacchi 9329d26e4fcSRobert Mustacchi if (i40e->i40e_switch_rsrcs != NULL) { 9339d26e4fcSRobert Mustacchi size_t sz = sizeof (i40e_switch_rsrc_t) * 9349d26e4fcSRobert Mustacchi i40e->i40e_switch_rsrc_alloc; 9359d26e4fcSRobert Mustacchi ASSERT(sz > 0); 9369d26e4fcSRobert Mustacchi kmem_free(i40e->i40e_switch_rsrcs, sz); 9379d26e4fcSRobert Mustacchi i40e->i40e_switch_rsrcs = NULL; 9389d26e4fcSRobert Mustacchi } 9399d26e4fcSRobert Mustacchi 9409d26e4fcSRobert Mustacchi if (i40e->i40e_device != NULL) 9419d26e4fcSRobert Mustacchi i40e_device_rele(i40e); 9429d26e4fcSRobert Mustacchi } 9439d26e4fcSRobert Mustacchi 9449d26e4fcSRobert Mustacchi static boolean_t 9459d26e4fcSRobert Mustacchi i40e_get_available_resources(i40e_t *i40e) 9469d26e4fcSRobert Mustacchi { 9479d26e4fcSRobert Mustacchi dev_info_t *parent; 9489d26e4fcSRobert Mustacchi uint16_t bus, device, func; 9499d26e4fcSRobert Mustacchi uint_t nregs; 9509d26e4fcSRobert Mustacchi int *regs, i; 9519d26e4fcSRobert Mustacchi i40e_device_t *idp; 9529d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 9539d26e4fcSRobert Mustacchi 9549d26e4fcSRobert Mustacchi parent = ddi_get_parent(i40e->i40e_dip); 9559d26e4fcSRobert Mustacchi 9569d26e4fcSRobert Mustacchi if (ddi_prop_lookup_int_array(DDI_DEV_T_ANY, i40e->i40e_dip, 0, "reg", 9579d26e4fcSRobert Mustacchi ®s, &nregs) != DDI_PROP_SUCCESS) { 9589d26e4fcSRobert Mustacchi return (B_FALSE); 9599d26e4fcSRobert Mustacchi } 9609d26e4fcSRobert Mustacchi 9619d26e4fcSRobert Mustacchi if (nregs < 1) { 9629d26e4fcSRobert Mustacchi ddi_prop_free(regs); 9639d26e4fcSRobert Mustacchi return (B_FALSE); 9649d26e4fcSRobert Mustacchi } 9659d26e4fcSRobert Mustacchi 9669d26e4fcSRobert Mustacchi bus = PCI_REG_BUS_G(regs[0]); 9679d26e4fcSRobert Mustacchi device = PCI_REG_DEV_G(regs[0]); 9689d26e4fcSRobert Mustacchi func = PCI_REG_FUNC_G(regs[0]); 9699d26e4fcSRobert Mustacchi ddi_prop_free(regs); 9709d26e4fcSRobert Mustacchi 9719d26e4fcSRobert Mustacchi i40e->i40e_hw_space.bus.func = func; 9729d26e4fcSRobert Mustacchi i40e->i40e_hw_space.bus.device = device; 9739d26e4fcSRobert Mustacchi 9749d26e4fcSRobert Mustacchi if (i40e_get_switch_resources(i40e) == B_FALSE) { 9759d26e4fcSRobert Mustacchi return (B_FALSE); 9769d26e4fcSRobert Mustacchi } 9779d26e4fcSRobert Mustacchi 9789d26e4fcSRobert Mustacchi /* 9799d26e4fcSRobert Mustacchi * To calculate the total amount of a resource we have available, we 9809d26e4fcSRobert Mustacchi * need to add how many our i40e_t thinks it has guaranteed, if any, and 9819d26e4fcSRobert Mustacchi * then we need to go through and divide the number of available on the 9829d26e4fcSRobert Mustacchi * device, which was snapshotted before anyone should have allocated 9839d26e4fcSRobert Mustacchi * anything, and use that to derive how many are available from the 9849d26e4fcSRobert Mustacchi * pool. Longer term, we may want to turn this into something that's 9859d26e4fcSRobert Mustacchi * more of a pool-like resource that everything can share (though that 9869d26e4fcSRobert Mustacchi * may require some more assistance from MAC). 9879d26e4fcSRobert Mustacchi * 9889d26e4fcSRobert Mustacchi * Though for transmit and receive queue pairs, we just have to ask 9899d26e4fcSRobert Mustacchi * firmware instead. 9909d26e4fcSRobert Mustacchi */ 9919d26e4fcSRobert Mustacchi idp = i40e_device_find(i40e, parent, bus, device); 9929d26e4fcSRobert Mustacchi i40e->i40e_device = idp; 9939d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nvsis = 0; 9949d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nvsis_used = 0; 9959d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmacfilt = 0; 9969d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmacfilt_used = 0; 9979d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmcastfilt = 0; 9989d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmcastfilt_used = 0; 9999d26e4fcSRobert Mustacchi 10009d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_switch_rsrc_actual; i++) { 10019d26e4fcSRobert Mustacchi i40e_switch_rsrc_t *srp = &i40e->i40e_switch_rsrcs[i]; 10029d26e4fcSRobert Mustacchi 10039d26e4fcSRobert Mustacchi switch (srp->resource_type) { 10049d26e4fcSRobert Mustacchi case I40E_AQ_RESOURCE_TYPE_VSI: 10059d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nvsis += 10069d26e4fcSRobert Mustacchi LE_16(srp->guaranteed); 10079d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nvsis_used = LE_16(srp->used); 10089d26e4fcSRobert Mustacchi break; 10099d26e4fcSRobert Mustacchi case I40E_AQ_RESOURCE_TYPE_MACADDR: 10109d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmacfilt += 10119d26e4fcSRobert Mustacchi LE_16(srp->guaranteed); 10129d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmacfilt_used = 10139d26e4fcSRobert Mustacchi LE_16(srp->used); 10149d26e4fcSRobert Mustacchi break; 10159d26e4fcSRobert Mustacchi case I40E_AQ_RESOURCE_TYPE_MULTICAST_HASH: 10169d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmcastfilt += 10179d26e4fcSRobert Mustacchi LE_16(srp->guaranteed); 10189d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmcastfilt_used = 10199d26e4fcSRobert Mustacchi LE_16(srp->used); 10209d26e4fcSRobert Mustacchi break; 10219d26e4fcSRobert Mustacchi default: 10229d26e4fcSRobert Mustacchi break; 10239d26e4fcSRobert Mustacchi } 10249d26e4fcSRobert Mustacchi } 10259d26e4fcSRobert Mustacchi 10269d26e4fcSRobert Mustacchi for (i = 0; i < idp->id_rsrcs_act; i++) { 10279d26e4fcSRobert Mustacchi i40e_switch_rsrc_t *srp = &i40e->i40e_switch_rsrcs[i]; 10289d26e4fcSRobert Mustacchi switch (srp->resource_type) { 10299d26e4fcSRobert Mustacchi case I40E_AQ_RESOURCE_TYPE_VSI: 10309d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nvsis += 10319d26e4fcSRobert Mustacchi LE_16(srp->total_unalloced) / idp->id_nfuncs; 10329d26e4fcSRobert Mustacchi break; 10339d26e4fcSRobert Mustacchi case I40E_AQ_RESOURCE_TYPE_MACADDR: 10349d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmacfilt += 10359d26e4fcSRobert Mustacchi LE_16(srp->total_unalloced) / idp->id_nfuncs; 10369d26e4fcSRobert Mustacchi break; 10379d26e4fcSRobert Mustacchi case I40E_AQ_RESOURCE_TYPE_MULTICAST_HASH: 10389d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmcastfilt += 10399d26e4fcSRobert Mustacchi LE_16(srp->total_unalloced) / idp->id_nfuncs; 10409d26e4fcSRobert Mustacchi default: 10419d26e4fcSRobert Mustacchi break; 10429d26e4fcSRobert Mustacchi } 10439d26e4fcSRobert Mustacchi } 10449d26e4fcSRobert Mustacchi 10459d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nrx_queue = hw->func_caps.num_rx_qp; 10469d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_ntx_queue = hw->func_caps.num_tx_qp; 10479d26e4fcSRobert Mustacchi 10489d26e4fcSRobert Mustacchi i40e->i40e_uaddrs = kmem_zalloc(sizeof (i40e_uaddr_t) * 10499d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmacfilt, KM_SLEEP); 10509d26e4fcSRobert Mustacchi i40e->i40e_maddrs = kmem_zalloc(sizeof (i40e_maddr_t) * 10519d26e4fcSRobert Mustacchi i40e->i40e_resources.ifr_nmcastfilt, KM_SLEEP); 10529d26e4fcSRobert Mustacchi 10539d26e4fcSRobert Mustacchi /* 10549d26e4fcSRobert Mustacchi * Initialize these as multicast addresses to indicate it's invalid for 10559d26e4fcSRobert Mustacchi * sanity purposes. Think of it like 0xdeadbeef. 10569d26e4fcSRobert Mustacchi */ 10579d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_resources.ifr_nmacfilt; i++) 10589d26e4fcSRobert Mustacchi i40e->i40e_uaddrs[i].iua_mac[0] = 0x01; 10599d26e4fcSRobert Mustacchi 10609d26e4fcSRobert Mustacchi return (B_TRUE); 10619d26e4fcSRobert Mustacchi } 10629d26e4fcSRobert Mustacchi 10639d26e4fcSRobert Mustacchi static boolean_t 10649d26e4fcSRobert Mustacchi i40e_enable_interrupts(i40e_t *i40e) 10659d26e4fcSRobert Mustacchi { 10669d26e4fcSRobert Mustacchi int i, rc; 10679d26e4fcSRobert Mustacchi 10689d26e4fcSRobert Mustacchi if (i40e->i40e_intr_cap & DDI_INTR_FLAG_BLOCK) { 10699d26e4fcSRobert Mustacchi rc = ddi_intr_block_enable(i40e->i40e_intr_handles, 10709d26e4fcSRobert Mustacchi i40e->i40e_intr_count); 10719d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 10729d26e4fcSRobert Mustacchi i40e_error(i40e, "Interrupt block-enable failed: %d", 10739d26e4fcSRobert Mustacchi rc); 10749d26e4fcSRobert Mustacchi return (B_FALSE); 10759d26e4fcSRobert Mustacchi } 10769d26e4fcSRobert Mustacchi } else { 10779d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_intr_count; i++) { 10789d26e4fcSRobert Mustacchi rc = ddi_intr_enable(i40e->i40e_intr_handles[i]); 10799d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 10809d26e4fcSRobert Mustacchi i40e_error(i40e, 10819d26e4fcSRobert Mustacchi "Failed to enable interrupt %d: %d", i, rc); 10829d26e4fcSRobert Mustacchi while (--i >= 0) { 10839d26e4fcSRobert Mustacchi (void) ddi_intr_disable( 10849d26e4fcSRobert Mustacchi i40e->i40e_intr_handles[i]); 10859d26e4fcSRobert Mustacchi } 10869d26e4fcSRobert Mustacchi return (B_FALSE); 10879d26e4fcSRobert Mustacchi } 10889d26e4fcSRobert Mustacchi } 10899d26e4fcSRobert Mustacchi } 10909d26e4fcSRobert Mustacchi 10919d26e4fcSRobert Mustacchi return (B_TRUE); 10929d26e4fcSRobert Mustacchi } 10939d26e4fcSRobert Mustacchi 10949d26e4fcSRobert Mustacchi static boolean_t 10959d26e4fcSRobert Mustacchi i40e_disable_interrupts(i40e_t *i40e) 10969d26e4fcSRobert Mustacchi { 10979d26e4fcSRobert Mustacchi int i, rc; 10989d26e4fcSRobert Mustacchi 10999d26e4fcSRobert Mustacchi if (i40e->i40e_intr_cap & DDI_INTR_FLAG_BLOCK) { 11009d26e4fcSRobert Mustacchi rc = ddi_intr_block_disable(i40e->i40e_intr_handles, 11019d26e4fcSRobert Mustacchi i40e->i40e_intr_count); 11029d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 11039d26e4fcSRobert Mustacchi i40e_error(i40e, 11049d26e4fcSRobert Mustacchi "Interrupt block-disabled failed: %d", rc); 11059d26e4fcSRobert Mustacchi return (B_FALSE); 11069d26e4fcSRobert Mustacchi } 11079d26e4fcSRobert Mustacchi } else { 11089d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_intr_count; i++) { 11099d26e4fcSRobert Mustacchi rc = ddi_intr_disable(i40e->i40e_intr_handles[i]); 11109d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 11119d26e4fcSRobert Mustacchi i40e_error(i40e, 11129d26e4fcSRobert Mustacchi "Failed to disable interrupt %d: %d", 11139d26e4fcSRobert Mustacchi i, rc); 11149d26e4fcSRobert Mustacchi return (B_FALSE); 11159d26e4fcSRobert Mustacchi } 11169d26e4fcSRobert Mustacchi } 11179d26e4fcSRobert Mustacchi } 11189d26e4fcSRobert Mustacchi 11199d26e4fcSRobert Mustacchi return (B_TRUE); 11209d26e4fcSRobert Mustacchi } 11219d26e4fcSRobert Mustacchi 11229d26e4fcSRobert Mustacchi /* 11239d26e4fcSRobert Mustacchi * Free receive & transmit rings. 11249d26e4fcSRobert Mustacchi */ 11259d26e4fcSRobert Mustacchi static void 11269d26e4fcSRobert Mustacchi i40e_free_trqpairs(i40e_t *i40e) 11279d26e4fcSRobert Mustacchi { 11289d26e4fcSRobert Mustacchi i40e_trqpair_t *itrq; 11299d26e4fcSRobert Mustacchi 113009aee612SRyan Zezeski if (i40e->i40e_rx_groups != NULL) { 113109aee612SRyan Zezeski kmem_free(i40e->i40e_rx_groups, 113209aee612SRyan Zezeski sizeof (i40e_rx_group_t) * i40e->i40e_num_rx_groups); 113309aee612SRyan Zezeski i40e->i40e_rx_groups = NULL; 113409aee612SRyan Zezeski } 113509aee612SRyan Zezeski 11369d26e4fcSRobert Mustacchi if (i40e->i40e_trqpairs != NULL) { 113709aee612SRyan Zezeski for (uint_t i = 0; i < i40e->i40e_num_trqpairs; i++) { 11389d26e4fcSRobert Mustacchi itrq = &i40e->i40e_trqpairs[i]; 11399d26e4fcSRobert Mustacchi mutex_destroy(&itrq->itrq_rx_lock); 11409d26e4fcSRobert Mustacchi mutex_destroy(&itrq->itrq_tx_lock); 11419d26e4fcSRobert Mustacchi mutex_destroy(&itrq->itrq_tcb_lock); 11429d26e4fcSRobert Mustacchi 11439d26e4fcSRobert Mustacchi /* 11449d26e4fcSRobert Mustacchi * Should have already been cleaned up by start/stop, 11459d26e4fcSRobert Mustacchi * etc. 11469d26e4fcSRobert Mustacchi */ 11479d26e4fcSRobert Mustacchi ASSERT(itrq->itrq_txkstat == NULL); 11489d26e4fcSRobert Mustacchi ASSERT(itrq->itrq_rxkstat == NULL); 11499d26e4fcSRobert Mustacchi } 11509d26e4fcSRobert Mustacchi 11519d26e4fcSRobert Mustacchi kmem_free(i40e->i40e_trqpairs, 11529d26e4fcSRobert Mustacchi sizeof (i40e_trqpair_t) * i40e->i40e_num_trqpairs); 11539d26e4fcSRobert Mustacchi i40e->i40e_trqpairs = NULL; 11549d26e4fcSRobert Mustacchi } 11559d26e4fcSRobert Mustacchi 11569d26e4fcSRobert Mustacchi cv_destroy(&i40e->i40e_rx_pending_cv); 11579d26e4fcSRobert Mustacchi mutex_destroy(&i40e->i40e_rx_pending_lock); 11589d26e4fcSRobert Mustacchi mutex_destroy(&i40e->i40e_general_lock); 11599d26e4fcSRobert Mustacchi } 11609d26e4fcSRobert Mustacchi 11619d26e4fcSRobert Mustacchi /* 11629d26e4fcSRobert Mustacchi * Allocate transmit and receive rings, as well as other data structures that we 11639d26e4fcSRobert Mustacchi * need. 11649d26e4fcSRobert Mustacchi */ 11659d26e4fcSRobert Mustacchi static boolean_t 11669d26e4fcSRobert Mustacchi i40e_alloc_trqpairs(i40e_t *i40e) 11679d26e4fcSRobert Mustacchi { 11689d26e4fcSRobert Mustacchi void *mutexpri = DDI_INTR_PRI(i40e->i40e_intr_pri); 11699d26e4fcSRobert Mustacchi 11709d26e4fcSRobert Mustacchi /* 11719d26e4fcSRobert Mustacchi * Now that we have the priority for the interrupts, initialize 11729d26e4fcSRobert Mustacchi * all relevant locks. 11739d26e4fcSRobert Mustacchi */ 11749d26e4fcSRobert Mustacchi mutex_init(&i40e->i40e_general_lock, NULL, MUTEX_DRIVER, mutexpri); 11759d26e4fcSRobert Mustacchi mutex_init(&i40e->i40e_rx_pending_lock, NULL, MUTEX_DRIVER, mutexpri); 11769d26e4fcSRobert Mustacchi cv_init(&i40e->i40e_rx_pending_cv, NULL, CV_DRIVER, NULL); 11779d26e4fcSRobert Mustacchi 11789d26e4fcSRobert Mustacchi i40e->i40e_trqpairs = kmem_zalloc(sizeof (i40e_trqpair_t) * 11799d26e4fcSRobert Mustacchi i40e->i40e_num_trqpairs, KM_SLEEP); 118009aee612SRyan Zezeski for (uint_t i = 0; i < i40e->i40e_num_trqpairs; i++) { 11819d26e4fcSRobert Mustacchi i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[i]; 11829d26e4fcSRobert Mustacchi 11839d26e4fcSRobert Mustacchi itrq->itrq_i40e = i40e; 11849d26e4fcSRobert Mustacchi mutex_init(&itrq->itrq_rx_lock, NULL, MUTEX_DRIVER, mutexpri); 11859d26e4fcSRobert Mustacchi mutex_init(&itrq->itrq_tx_lock, NULL, MUTEX_DRIVER, mutexpri); 11869d26e4fcSRobert Mustacchi mutex_init(&itrq->itrq_tcb_lock, NULL, MUTEX_DRIVER, mutexpri); 11879d26e4fcSRobert Mustacchi itrq->itrq_index = i; 11889d26e4fcSRobert Mustacchi } 11899d26e4fcSRobert Mustacchi 119009aee612SRyan Zezeski i40e->i40e_rx_groups = kmem_zalloc(sizeof (i40e_rx_group_t) * 119109aee612SRyan Zezeski i40e->i40e_num_rx_groups, KM_SLEEP); 119209aee612SRyan Zezeski 119309aee612SRyan Zezeski for (uint_t i = 0; i < i40e->i40e_num_rx_groups; i++) { 119409aee612SRyan Zezeski i40e_rx_group_t *rxg = &i40e->i40e_rx_groups[i]; 119509aee612SRyan Zezeski 119609aee612SRyan Zezeski rxg->irg_index = i; 119709aee612SRyan Zezeski rxg->irg_i40e = i40e; 119809aee612SRyan Zezeski } 119909aee612SRyan Zezeski 12009d26e4fcSRobert Mustacchi return (B_TRUE); 12019d26e4fcSRobert Mustacchi } 12029d26e4fcSRobert Mustacchi 12039d26e4fcSRobert Mustacchi 12049d26e4fcSRobert Mustacchi 12059d26e4fcSRobert Mustacchi /* 12069d26e4fcSRobert Mustacchi * Unless a .conf file already overrode i40e_t structure values, they will 12079d26e4fcSRobert Mustacchi * be 0, and need to be set in conjunction with the now-available HW report. 12089d26e4fcSRobert Mustacchi */ 12099d26e4fcSRobert Mustacchi /* ARGSUSED */ 12109d26e4fcSRobert Mustacchi static void 12119d26e4fcSRobert Mustacchi i40e_hw_to_instance(i40e_t *i40e, i40e_hw_t *hw) 12129d26e4fcSRobert Mustacchi { 121309aee612SRyan Zezeski if (i40e->i40e_num_trqpairs_per_vsi == 0) { 121409aee612SRyan Zezeski if (i40e_is_x722(i40e)) { 121509aee612SRyan Zezeski i40e->i40e_num_trqpairs_per_vsi = 121609aee612SRyan Zezeski I40E_722_MAX_TC_QUEUES; 121709aee612SRyan Zezeski } else { 121809aee612SRyan Zezeski i40e->i40e_num_trqpairs_per_vsi = 121909aee612SRyan Zezeski I40E_710_MAX_TC_QUEUES; 122009aee612SRyan Zezeski } 12219d26e4fcSRobert Mustacchi } 12229d26e4fcSRobert Mustacchi 12239d26e4fcSRobert Mustacchi if (i40e->i40e_num_rx_groups == 0) { 1224*88628b1bSRyan Zezeski i40e->i40e_num_rx_groups = I40E_DEF_NUM_RX_GROUPS; 12259d26e4fcSRobert Mustacchi } 12269d26e4fcSRobert Mustacchi } 12279d26e4fcSRobert Mustacchi 12289d26e4fcSRobert Mustacchi /* 12299d26e4fcSRobert Mustacchi * Free any resources required by, or setup by, the Intel common code. 12309d26e4fcSRobert Mustacchi */ 12319d26e4fcSRobert Mustacchi static void 12329d26e4fcSRobert Mustacchi i40e_common_code_fini(i40e_t *i40e) 12339d26e4fcSRobert Mustacchi { 12349d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 12359d26e4fcSRobert Mustacchi int rc; 12369d26e4fcSRobert Mustacchi 12379d26e4fcSRobert Mustacchi rc = i40e_shutdown_lan_hmc(hw); 12389d26e4fcSRobert Mustacchi if (rc != I40E_SUCCESS) 12399d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to shutdown LAN hmc: %d", rc); 12409d26e4fcSRobert Mustacchi 12419d26e4fcSRobert Mustacchi rc = i40e_shutdown_adminq(hw); 12429d26e4fcSRobert Mustacchi if (rc != I40E_SUCCESS) 12439d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to shutdown admin queue: %d", rc); 12449d26e4fcSRobert Mustacchi } 12459d26e4fcSRobert Mustacchi 12469d26e4fcSRobert Mustacchi /* 12479d26e4fcSRobert Mustacchi * Initialize and call Intel common-code routines, includes some setup 12489d26e4fcSRobert Mustacchi * the common code expects from the driver. Also prints on failure, so 12499d26e4fcSRobert Mustacchi * the caller doesn't have to. 12509d26e4fcSRobert Mustacchi */ 12519d26e4fcSRobert Mustacchi static boolean_t 12529d26e4fcSRobert Mustacchi i40e_common_code_init(i40e_t *i40e, i40e_hw_t *hw) 12539d26e4fcSRobert Mustacchi { 12549d26e4fcSRobert Mustacchi int rc; 12559d26e4fcSRobert Mustacchi 12569d26e4fcSRobert Mustacchi i40e_clear_hw(hw); 12579d26e4fcSRobert Mustacchi rc = i40e_pf_reset(hw); 12589d26e4fcSRobert Mustacchi if (rc != 0) { 12599d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to reset hardware: %d", rc); 12609d26e4fcSRobert Mustacchi i40e_fm_ereport(i40e, DDI_FM_DEVICE_NO_RESPONSE); 12619d26e4fcSRobert Mustacchi return (B_FALSE); 12629d26e4fcSRobert Mustacchi } 12639d26e4fcSRobert Mustacchi 12649d26e4fcSRobert Mustacchi rc = i40e_init_shared_code(hw); 12659d26e4fcSRobert Mustacchi if (rc != 0) { 12669d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to initialize i40e core: %d", rc); 12679d26e4fcSRobert Mustacchi return (B_FALSE); 12689d26e4fcSRobert Mustacchi } 12699d26e4fcSRobert Mustacchi 12709d26e4fcSRobert Mustacchi hw->aq.num_arq_entries = I40E_DEF_ADMINQ_SIZE; 12719d26e4fcSRobert Mustacchi hw->aq.num_asq_entries = I40E_DEF_ADMINQ_SIZE; 12729d26e4fcSRobert Mustacchi hw->aq.arq_buf_size = I40E_ADMINQ_BUFSZ; 12739d26e4fcSRobert Mustacchi hw->aq.asq_buf_size = I40E_ADMINQ_BUFSZ; 12749d26e4fcSRobert Mustacchi 12759d26e4fcSRobert Mustacchi rc = i40e_init_adminq(hw); 12769d26e4fcSRobert Mustacchi if (rc != 0) { 12779d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to initialize firmware admin queue: " 12789d26e4fcSRobert Mustacchi "%d, potential firmware version mismatch", rc); 12799d26e4fcSRobert Mustacchi i40e_fm_ereport(i40e, DDI_FM_DEVICE_INVAL_STATE); 12809d26e4fcSRobert Mustacchi return (B_FALSE); 12819d26e4fcSRobert Mustacchi } 12829d26e4fcSRobert Mustacchi 12839d26e4fcSRobert Mustacchi if (hw->aq.api_maj_ver == I40E_FW_API_VERSION_MAJOR && 12849d26e4fcSRobert Mustacchi hw->aq.api_min_ver > I40E_FW_API_VERSION_MINOR) { 12853d75a287SRobert Mustacchi i40e_log(i40e, "The driver for the device detected a newer " 12869d26e4fcSRobert Mustacchi "version of the NVM image (%d.%d) than expected (%d.%d).\n" 12879d26e4fcSRobert Mustacchi "Please install the most recent version of the network " 12889d26e4fcSRobert Mustacchi "driver.\n", hw->aq.api_maj_ver, hw->aq.api_min_ver, 12899d26e4fcSRobert Mustacchi I40E_FW_API_VERSION_MAJOR, I40E_FW_API_VERSION_MINOR); 12909d26e4fcSRobert Mustacchi } else if (hw->aq.api_maj_ver < I40E_FW_API_VERSION_MAJOR || 12919d26e4fcSRobert Mustacchi hw->aq.api_min_ver < (I40E_FW_API_VERSION_MINOR - 1)) { 12923d75a287SRobert Mustacchi i40e_log(i40e, "The driver for the device detected an older" 12939d26e4fcSRobert Mustacchi " version of the NVM image (%d.%d) than expected (%d.%d)." 12949d26e4fcSRobert Mustacchi "\nPlease update the NVM image.\n", 12959d26e4fcSRobert Mustacchi hw->aq.api_maj_ver, hw->aq.api_min_ver, 12969d26e4fcSRobert Mustacchi I40E_FW_API_VERSION_MAJOR, I40E_FW_API_VERSION_MINOR - 1); 12979d26e4fcSRobert Mustacchi } 12989d26e4fcSRobert Mustacchi 12999d26e4fcSRobert Mustacchi i40e_clear_pxe_mode(hw); 13009d26e4fcSRobert Mustacchi 13019d26e4fcSRobert Mustacchi /* 13029d26e4fcSRobert Mustacchi * We need to call this so that the common code can discover 13039d26e4fcSRobert Mustacchi * capabilities of the hardware, which it uses throughout the rest. 13049d26e4fcSRobert Mustacchi */ 13059d26e4fcSRobert Mustacchi if (!i40e_get_hw_capabilities(i40e, hw)) { 13069d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to obtain hardware capabilities"); 13079d26e4fcSRobert Mustacchi return (B_FALSE); 13089d26e4fcSRobert Mustacchi } 13099d26e4fcSRobert Mustacchi 13109d26e4fcSRobert Mustacchi if (i40e_get_available_resources(i40e) == B_FALSE) { 13119d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to obtain hardware resources"); 13129d26e4fcSRobert Mustacchi return (B_FALSE); 13139d26e4fcSRobert Mustacchi } 13149d26e4fcSRobert Mustacchi 13159d26e4fcSRobert Mustacchi i40e_hw_to_instance(i40e, hw); 13169d26e4fcSRobert Mustacchi 13179d26e4fcSRobert Mustacchi rc = i40e_init_lan_hmc(hw, hw->func_caps.num_tx_qp, 13189d26e4fcSRobert Mustacchi hw->func_caps.num_rx_qp, 0, 0); 13199d26e4fcSRobert Mustacchi if (rc != 0) { 13209d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to initialize hardware memory cache: " 13219d26e4fcSRobert Mustacchi "%d", rc); 13229d26e4fcSRobert Mustacchi return (B_FALSE); 13239d26e4fcSRobert Mustacchi } 13249d26e4fcSRobert Mustacchi 13259d26e4fcSRobert Mustacchi rc = i40e_configure_lan_hmc(hw, I40E_HMC_MODEL_DIRECT_ONLY); 13269d26e4fcSRobert Mustacchi if (rc != 0) { 13279d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to configure hardware memory cache: " 13289d26e4fcSRobert Mustacchi "%d", rc); 13299d26e4fcSRobert Mustacchi return (B_FALSE); 13309d26e4fcSRobert Mustacchi } 13319d26e4fcSRobert Mustacchi 13329d26e4fcSRobert Mustacchi (void) i40e_aq_stop_lldp(hw, TRUE, NULL); 13339d26e4fcSRobert Mustacchi 13349d26e4fcSRobert Mustacchi rc = i40e_get_mac_addr(hw, hw->mac.addr); 13359d26e4fcSRobert Mustacchi if (rc != I40E_SUCCESS) { 13369d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to retrieve hardware mac address: %d", 13379d26e4fcSRobert Mustacchi rc); 13389d26e4fcSRobert Mustacchi return (B_FALSE); 13399d26e4fcSRobert Mustacchi } 13409d26e4fcSRobert Mustacchi 13419d26e4fcSRobert Mustacchi rc = i40e_validate_mac_addr(hw->mac.addr); 13429d26e4fcSRobert Mustacchi if (rc != 0) { 13439d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to validate internal mac address: " 13449d26e4fcSRobert Mustacchi "%d", rc); 13459d26e4fcSRobert Mustacchi return (B_FALSE); 13469d26e4fcSRobert Mustacchi } 13479d26e4fcSRobert Mustacchi bcopy(hw->mac.addr, hw->mac.perm_addr, ETHERADDRL); 13489d26e4fcSRobert Mustacchi if ((rc = i40e_get_port_mac_addr(hw, hw->mac.port_addr)) != 13499d26e4fcSRobert Mustacchi I40E_SUCCESS) { 13509d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to retrieve port mac address: %d", 13519d26e4fcSRobert Mustacchi rc); 13529d26e4fcSRobert Mustacchi return (B_FALSE); 13539d26e4fcSRobert Mustacchi } 13549d26e4fcSRobert Mustacchi 13559d26e4fcSRobert Mustacchi /* 135609aee612SRyan Zezeski * We need to obtain the Default Virtual Station SEID (VSI) 135709aee612SRyan Zezeski * before we can perform other operations on the device. 13589d26e4fcSRobert Mustacchi */ 135909aee612SRyan Zezeski if (!i40e_set_def_vsi_seid(i40e)) { 136009aee612SRyan Zezeski i40e_error(i40e, "failed to obtain Default VSI SEID"); 13619d26e4fcSRobert Mustacchi return (B_FALSE); 13629d26e4fcSRobert Mustacchi } 13639d26e4fcSRobert Mustacchi 13649d26e4fcSRobert Mustacchi return (B_TRUE); 13659d26e4fcSRobert Mustacchi } 13669d26e4fcSRobert Mustacchi 13679d26e4fcSRobert Mustacchi static void 13689d26e4fcSRobert Mustacchi i40e_unconfigure(dev_info_t *devinfo, i40e_t *i40e) 13699d26e4fcSRobert Mustacchi { 13709d26e4fcSRobert Mustacchi int rc; 13719d26e4fcSRobert Mustacchi 13729d26e4fcSRobert Mustacchi if (i40e->i40e_attach_progress & I40E_ATTACH_ENABLE_INTR) 13739d26e4fcSRobert Mustacchi (void) i40e_disable_interrupts(i40e); 13749d26e4fcSRobert Mustacchi 13759d26e4fcSRobert Mustacchi if ((i40e->i40e_attach_progress & I40E_ATTACH_LINK_TIMER) && 13769d26e4fcSRobert Mustacchi i40e->i40e_periodic_id != 0) { 13779d26e4fcSRobert Mustacchi ddi_periodic_delete(i40e->i40e_periodic_id); 13789d26e4fcSRobert Mustacchi i40e->i40e_periodic_id = 0; 13799d26e4fcSRobert Mustacchi } 13809d26e4fcSRobert Mustacchi 138144b0ba91SRobert Mustacchi if (i40e->i40e_attach_progress & I40E_ATTACH_UFM_INIT) 138244b0ba91SRobert Mustacchi ddi_ufm_fini(i40e->i40e_ufmh); 138344b0ba91SRobert Mustacchi 13849d26e4fcSRobert Mustacchi if (i40e->i40e_attach_progress & I40E_ATTACH_MAC) { 13859d26e4fcSRobert Mustacchi rc = mac_unregister(i40e->i40e_mac_hdl); 13869d26e4fcSRobert Mustacchi if (rc != 0) { 13879d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to unregister from mac: %d", 13889d26e4fcSRobert Mustacchi rc); 13899d26e4fcSRobert Mustacchi } 13909d26e4fcSRobert Mustacchi } 13919d26e4fcSRobert Mustacchi 13929d26e4fcSRobert Mustacchi if (i40e->i40e_attach_progress & I40E_ATTACH_STATS) { 13939d26e4fcSRobert Mustacchi i40e_stats_fini(i40e); 13949d26e4fcSRobert Mustacchi } 13959d26e4fcSRobert Mustacchi 13969d26e4fcSRobert Mustacchi if (i40e->i40e_attach_progress & I40E_ATTACH_ADD_INTR) 13979d26e4fcSRobert Mustacchi i40e_rem_intr_handlers(i40e); 13989d26e4fcSRobert Mustacchi 13999d26e4fcSRobert Mustacchi if (i40e->i40e_attach_progress & I40E_ATTACH_ALLOC_RINGSLOCKS) 14009d26e4fcSRobert Mustacchi i40e_free_trqpairs(i40e); 14019d26e4fcSRobert Mustacchi 14029d26e4fcSRobert Mustacchi if (i40e->i40e_attach_progress & I40E_ATTACH_ALLOC_INTR) 14039d26e4fcSRobert Mustacchi i40e_rem_intrs(i40e); 14049d26e4fcSRobert Mustacchi 14059d26e4fcSRobert Mustacchi if (i40e->i40e_attach_progress & I40E_ATTACH_COMMON_CODE) 14069d26e4fcSRobert Mustacchi i40e_common_code_fini(i40e); 14079d26e4fcSRobert Mustacchi 14089d26e4fcSRobert Mustacchi i40e_cleanup_resources(i40e); 14099d26e4fcSRobert Mustacchi 14109d26e4fcSRobert Mustacchi if (i40e->i40e_attach_progress & I40E_ATTACH_PROPS) 14119d26e4fcSRobert Mustacchi (void) ddi_prop_remove_all(devinfo); 14129d26e4fcSRobert Mustacchi 14139d26e4fcSRobert Mustacchi if (i40e->i40e_attach_progress & I40E_ATTACH_REGS_MAP && 14149d26e4fcSRobert Mustacchi i40e->i40e_osdep_space.ios_reg_handle != NULL) { 14159d26e4fcSRobert Mustacchi ddi_regs_map_free(&i40e->i40e_osdep_space.ios_reg_handle); 14169d26e4fcSRobert Mustacchi i40e->i40e_osdep_space.ios_reg_handle = NULL; 14179d26e4fcSRobert Mustacchi } 14189d26e4fcSRobert Mustacchi 14199d26e4fcSRobert Mustacchi if ((i40e->i40e_attach_progress & I40E_ATTACH_PCI_CONFIG) && 14209d26e4fcSRobert Mustacchi i40e->i40e_osdep_space.ios_cfg_handle != NULL) { 14219d26e4fcSRobert Mustacchi pci_config_teardown(&i40e->i40e_osdep_space.ios_cfg_handle); 14229d26e4fcSRobert Mustacchi i40e->i40e_osdep_space.ios_cfg_handle = NULL; 14239d26e4fcSRobert Mustacchi } 14249d26e4fcSRobert Mustacchi 14259d26e4fcSRobert Mustacchi if (i40e->i40e_attach_progress & I40E_ATTACH_FM_INIT) 14269d26e4fcSRobert Mustacchi i40e_fm_fini(i40e); 14279d26e4fcSRobert Mustacchi 14289d26e4fcSRobert Mustacchi kmem_free(i40e->i40e_aqbuf, I40E_ADMINQ_BUFSZ); 14299d26e4fcSRobert Mustacchi kmem_free(i40e, sizeof (i40e_t)); 14309d26e4fcSRobert Mustacchi 14319d26e4fcSRobert Mustacchi ddi_set_driver_private(devinfo, NULL); 14329d26e4fcSRobert Mustacchi } 14339d26e4fcSRobert Mustacchi 14349d26e4fcSRobert Mustacchi static boolean_t 14359d26e4fcSRobert Mustacchi i40e_final_init(i40e_t *i40e) 14369d26e4fcSRobert Mustacchi { 14379d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 14389d26e4fcSRobert Mustacchi struct i40e_osdep *osdep = OS_DEP(hw); 14399d26e4fcSRobert Mustacchi uint8_t pbanum[I40E_PBANUM_STRLEN]; 14409d26e4fcSRobert Mustacchi enum i40e_status_code irc; 14419d26e4fcSRobert Mustacchi char buf[I40E_DDI_PROP_LEN]; 14429d26e4fcSRobert Mustacchi 14439d26e4fcSRobert Mustacchi pbanum[0] = '\0'; 14449d26e4fcSRobert Mustacchi irc = i40e_read_pba_string(hw, pbanum, sizeof (pbanum)); 14459d26e4fcSRobert Mustacchi if (irc != I40E_SUCCESS) { 14469d26e4fcSRobert Mustacchi i40e_log(i40e, "failed to read PBA string: %d", irc); 14479d26e4fcSRobert Mustacchi } else { 14489d26e4fcSRobert Mustacchi (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip, 14499d26e4fcSRobert Mustacchi "printed-board-assembly", (char *)pbanum); 14509d26e4fcSRobert Mustacchi } 14519d26e4fcSRobert Mustacchi 14529d26e4fcSRobert Mustacchi #ifdef DEBUG 14539d26e4fcSRobert Mustacchi ASSERT(snprintf(NULL, 0, "%d.%d", hw->aq.fw_maj_ver, 14549d26e4fcSRobert Mustacchi hw->aq.fw_min_ver) < sizeof (buf)); 14559d26e4fcSRobert Mustacchi ASSERT(snprintf(NULL, 0, "%x", hw->aq.fw_build) < sizeof (buf)); 14569d26e4fcSRobert Mustacchi ASSERT(snprintf(NULL, 0, "%d.%d", hw->aq.api_maj_ver, 14579d26e4fcSRobert Mustacchi hw->aq.api_min_ver) < sizeof (buf)); 14589d26e4fcSRobert Mustacchi #endif 14599d26e4fcSRobert Mustacchi 14609d26e4fcSRobert Mustacchi (void) snprintf(buf, sizeof (buf), "%d.%d", hw->aq.fw_maj_ver, 14619d26e4fcSRobert Mustacchi hw->aq.fw_min_ver); 14629d26e4fcSRobert Mustacchi (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip, 14639d26e4fcSRobert Mustacchi "firmware-version", buf); 14649d26e4fcSRobert Mustacchi (void) snprintf(buf, sizeof (buf), "%x", hw->aq.fw_build); 14659d26e4fcSRobert Mustacchi (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip, 14669d26e4fcSRobert Mustacchi "firmware-build", buf); 14679d26e4fcSRobert Mustacchi (void) snprintf(buf, sizeof (buf), "%d.%d", hw->aq.api_maj_ver, 14689d26e4fcSRobert Mustacchi hw->aq.api_min_ver); 14699d26e4fcSRobert Mustacchi (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip, 14709d26e4fcSRobert Mustacchi "api-version", buf); 14719d26e4fcSRobert Mustacchi 14729d26e4fcSRobert Mustacchi if (!i40e_set_hw_bus_info(hw)) 14739d26e4fcSRobert Mustacchi return (B_FALSE); 14749d26e4fcSRobert Mustacchi 14759d26e4fcSRobert Mustacchi if (i40e_check_acc_handle(osdep->ios_reg_handle) != DDI_FM_OK) { 14769d26e4fcSRobert Mustacchi ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST); 14779d26e4fcSRobert Mustacchi return (B_FALSE); 14789d26e4fcSRobert Mustacchi } 14799d26e4fcSRobert Mustacchi 14809d26e4fcSRobert Mustacchi return (B_TRUE); 14819d26e4fcSRobert Mustacchi } 14829d26e4fcSRobert Mustacchi 14833d75a287SRobert Mustacchi static void 14849d26e4fcSRobert Mustacchi i40e_identify_hardware(i40e_t *i40e) 14859d26e4fcSRobert Mustacchi { 14869d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 14879d26e4fcSRobert Mustacchi struct i40e_osdep *osdep = &i40e->i40e_osdep_space; 14889d26e4fcSRobert Mustacchi 14899d26e4fcSRobert Mustacchi hw->vendor_id = pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_VENID); 14909d26e4fcSRobert Mustacchi hw->device_id = pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_DEVID); 14919d26e4fcSRobert Mustacchi hw->revision_id = pci_config_get8(osdep->ios_cfg_handle, 14929d26e4fcSRobert Mustacchi PCI_CONF_REVID); 14939d26e4fcSRobert Mustacchi hw->subsystem_device_id = 14949d26e4fcSRobert Mustacchi pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_SUBSYSID); 14959d26e4fcSRobert Mustacchi hw->subsystem_vendor_id = 14969d26e4fcSRobert Mustacchi pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_SUBVENID); 14979d26e4fcSRobert Mustacchi 14989d26e4fcSRobert Mustacchi /* 14999d26e4fcSRobert Mustacchi * Note that we set the hardware's bus information later on, in 15009d26e4fcSRobert Mustacchi * i40e_get_available_resources(). The common code doesn't seem to 15019d26e4fcSRobert Mustacchi * require that it be set in any ways, it seems to be mostly for 15029d26e4fcSRobert Mustacchi * book-keeping. 15039d26e4fcSRobert Mustacchi */ 15049d26e4fcSRobert Mustacchi } 15059d26e4fcSRobert Mustacchi 15069d26e4fcSRobert Mustacchi static boolean_t 15079d26e4fcSRobert Mustacchi i40e_regs_map(i40e_t *i40e) 15089d26e4fcSRobert Mustacchi { 15099d26e4fcSRobert Mustacchi dev_info_t *devinfo = i40e->i40e_dip; 15109d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 15119d26e4fcSRobert Mustacchi struct i40e_osdep *osdep = &i40e->i40e_osdep_space; 15129d26e4fcSRobert Mustacchi off_t memsize; 15139d26e4fcSRobert Mustacchi int ret; 15149d26e4fcSRobert Mustacchi 15159d26e4fcSRobert Mustacchi if (ddi_dev_regsize(devinfo, I40E_ADAPTER_REGSET, &memsize) != 15169d26e4fcSRobert Mustacchi DDI_SUCCESS) { 15179d26e4fcSRobert Mustacchi i40e_error(i40e, "Used invalid register set to map PCIe regs"); 15189d26e4fcSRobert Mustacchi return (B_FALSE); 15199d26e4fcSRobert Mustacchi } 15209d26e4fcSRobert Mustacchi 15219d26e4fcSRobert Mustacchi if ((ret = ddi_regs_map_setup(devinfo, I40E_ADAPTER_REGSET, 15229d26e4fcSRobert Mustacchi (caddr_t *)&hw->hw_addr, 0, memsize, &i40e_regs_acc_attr, 15239d26e4fcSRobert Mustacchi &osdep->ios_reg_handle)) != DDI_SUCCESS) { 15249d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to map device registers: %d", ret); 15259d26e4fcSRobert Mustacchi return (B_FALSE); 15269d26e4fcSRobert Mustacchi } 15279d26e4fcSRobert Mustacchi 15289d26e4fcSRobert Mustacchi osdep->ios_reg_size = memsize; 15299d26e4fcSRobert Mustacchi return (B_TRUE); 15309d26e4fcSRobert Mustacchi } 15319d26e4fcSRobert Mustacchi 15329d26e4fcSRobert Mustacchi /* 15339d26e4fcSRobert Mustacchi * Update parameters required when a new MTU has been configured. Calculate the 15349d26e4fcSRobert Mustacchi * maximum frame size, as well as, size our DMA buffers which we size in 15359d26e4fcSRobert Mustacchi * increments of 1K. 15369d26e4fcSRobert Mustacchi */ 15379d26e4fcSRobert Mustacchi void 15389d26e4fcSRobert Mustacchi i40e_update_mtu(i40e_t *i40e) 15399d26e4fcSRobert Mustacchi { 15409d26e4fcSRobert Mustacchi uint32_t rx, tx; 15419d26e4fcSRobert Mustacchi 15429d26e4fcSRobert Mustacchi i40e->i40e_frame_max = i40e->i40e_sdu + 15439d26e4fcSRobert Mustacchi sizeof (struct ether_vlan_header) + ETHERFCSL; 15449d26e4fcSRobert Mustacchi 15459d26e4fcSRobert Mustacchi rx = i40e->i40e_frame_max + I40E_BUF_IPHDR_ALIGNMENT; 15469d26e4fcSRobert Mustacchi i40e->i40e_rx_buf_size = ((rx >> 10) + 15479d26e4fcSRobert Mustacchi ((rx & (((uint32_t)1 << 10) -1)) > 0 ? 1 : 0)) << 10; 15489d26e4fcSRobert Mustacchi 15499d26e4fcSRobert Mustacchi tx = i40e->i40e_frame_max; 15509d26e4fcSRobert Mustacchi i40e->i40e_tx_buf_size = ((tx >> 10) + 15519d26e4fcSRobert Mustacchi ((tx & (((uint32_t)1 << 10) -1)) > 0 ? 1 : 0)) << 10; 15529d26e4fcSRobert Mustacchi } 15539d26e4fcSRobert Mustacchi 15549d26e4fcSRobert Mustacchi static int 15559d26e4fcSRobert Mustacchi i40e_get_prop(i40e_t *i40e, char *prop, int min, int max, int def) 15569d26e4fcSRobert Mustacchi { 15579d26e4fcSRobert Mustacchi int val; 15589d26e4fcSRobert Mustacchi 15599d26e4fcSRobert Mustacchi val = ddi_prop_get_int(DDI_DEV_T_ANY, i40e->i40e_dip, DDI_PROP_DONTPASS, 15609d26e4fcSRobert Mustacchi prop, def); 15619d26e4fcSRobert Mustacchi if (val > max) 15629d26e4fcSRobert Mustacchi val = max; 15639d26e4fcSRobert Mustacchi if (val < min) 15649d26e4fcSRobert Mustacchi val = min; 15659d26e4fcSRobert Mustacchi return (val); 15669d26e4fcSRobert Mustacchi } 15679d26e4fcSRobert Mustacchi 15689d26e4fcSRobert Mustacchi static void 15699d26e4fcSRobert Mustacchi i40e_init_properties(i40e_t *i40e) 15709d26e4fcSRobert Mustacchi { 15719d26e4fcSRobert Mustacchi i40e->i40e_sdu = i40e_get_prop(i40e, "default_mtu", 15729d26e4fcSRobert Mustacchi I40E_MIN_MTU, I40E_MAX_MTU, I40E_DEF_MTU); 15739d26e4fcSRobert Mustacchi 15749d26e4fcSRobert Mustacchi i40e->i40e_intr_force = i40e_get_prop(i40e, "intr_force", 15759d26e4fcSRobert Mustacchi I40E_INTR_NONE, I40E_INTR_LEGACY, I40E_INTR_NONE); 15769d26e4fcSRobert Mustacchi 15779d26e4fcSRobert Mustacchi i40e->i40e_mr_enable = i40e_get_prop(i40e, "mr_enable", 15789d26e4fcSRobert Mustacchi B_FALSE, B_TRUE, B_TRUE); 15799d26e4fcSRobert Mustacchi 15809d26e4fcSRobert Mustacchi i40e->i40e_tx_ring_size = i40e_get_prop(i40e, "tx_ring_size", 15819d26e4fcSRobert Mustacchi I40E_MIN_TX_RING_SIZE, I40E_MAX_TX_RING_SIZE, 15829d26e4fcSRobert Mustacchi I40E_DEF_TX_RING_SIZE); 15839d26e4fcSRobert Mustacchi if ((i40e->i40e_tx_ring_size % I40E_DESC_ALIGN) != 0) { 15849d26e4fcSRobert Mustacchi i40e->i40e_tx_ring_size = P2ROUNDUP(i40e->i40e_tx_ring_size, 15859d26e4fcSRobert Mustacchi I40E_DESC_ALIGN); 15869d26e4fcSRobert Mustacchi } 15879d26e4fcSRobert Mustacchi 15889d26e4fcSRobert Mustacchi i40e->i40e_tx_block_thresh = i40e_get_prop(i40e, "tx_resched_threshold", 15899d26e4fcSRobert Mustacchi I40E_MIN_TX_BLOCK_THRESH, 15909d26e4fcSRobert Mustacchi i40e->i40e_tx_ring_size - I40E_TX_MAX_COOKIE, 15919d26e4fcSRobert Mustacchi I40E_DEF_TX_BLOCK_THRESH); 15929d26e4fcSRobert Mustacchi 1593*88628b1bSRyan Zezeski i40e->i40e_num_rx_groups = i40e_get_prop(i40e, "rx_num_groups", 1594*88628b1bSRyan Zezeski I40E_MIN_NUM_RX_GROUPS, I40E_MAX_NUM_RX_GROUPS, 1595*88628b1bSRyan Zezeski I40E_DEF_NUM_RX_GROUPS); 1596*88628b1bSRyan Zezeski 15979d26e4fcSRobert Mustacchi i40e->i40e_rx_ring_size = i40e_get_prop(i40e, "rx_ring_size", 15989d26e4fcSRobert Mustacchi I40E_MIN_RX_RING_SIZE, I40E_MAX_RX_RING_SIZE, 15999d26e4fcSRobert Mustacchi I40E_DEF_RX_RING_SIZE); 16009d26e4fcSRobert Mustacchi if ((i40e->i40e_rx_ring_size % I40E_DESC_ALIGN) != 0) { 16019d26e4fcSRobert Mustacchi i40e->i40e_rx_ring_size = P2ROUNDUP(i40e->i40e_rx_ring_size, 16029d26e4fcSRobert Mustacchi I40E_DESC_ALIGN); 16039d26e4fcSRobert Mustacchi } 16049d26e4fcSRobert Mustacchi 16059d26e4fcSRobert Mustacchi i40e->i40e_rx_limit_per_intr = i40e_get_prop(i40e, "rx_limit_per_intr", 16069d26e4fcSRobert Mustacchi I40E_MIN_RX_LIMIT_PER_INTR, I40E_MAX_RX_LIMIT_PER_INTR, 16079d26e4fcSRobert Mustacchi I40E_DEF_RX_LIMIT_PER_INTR); 16089d26e4fcSRobert Mustacchi 16099d26e4fcSRobert Mustacchi i40e->i40e_tx_hcksum_enable = i40e_get_prop(i40e, "tx_hcksum_enable", 16109d26e4fcSRobert Mustacchi B_FALSE, B_TRUE, B_TRUE); 16119d26e4fcSRobert Mustacchi 161209aee612SRyan Zezeski i40e->i40e_tx_lso_enable = i40e_get_prop(i40e, "tx_lso_enable", 161309aee612SRyan Zezeski B_FALSE, B_TRUE, B_TRUE); 161409aee612SRyan Zezeski 16159d26e4fcSRobert Mustacchi i40e->i40e_rx_hcksum_enable = i40e_get_prop(i40e, "rx_hcksum_enable", 16169d26e4fcSRobert Mustacchi B_FALSE, B_TRUE, B_TRUE); 16179d26e4fcSRobert Mustacchi 16189d26e4fcSRobert Mustacchi i40e->i40e_rx_dma_min = i40e_get_prop(i40e, "rx_dma_threshold", 16199d26e4fcSRobert Mustacchi I40E_MIN_RX_DMA_THRESH, I40E_MAX_RX_DMA_THRESH, 16209d26e4fcSRobert Mustacchi I40E_DEF_RX_DMA_THRESH); 16219d26e4fcSRobert Mustacchi 16229d26e4fcSRobert Mustacchi i40e->i40e_tx_dma_min = i40e_get_prop(i40e, "tx_dma_threshold", 16239d26e4fcSRobert Mustacchi I40E_MIN_TX_DMA_THRESH, I40E_MAX_TX_DMA_THRESH, 16249d26e4fcSRobert Mustacchi I40E_DEF_TX_DMA_THRESH); 16259d26e4fcSRobert Mustacchi 16269d26e4fcSRobert Mustacchi i40e->i40e_tx_itr = i40e_get_prop(i40e, "tx_intr_throttle", 16279d26e4fcSRobert Mustacchi I40E_MIN_ITR, I40E_MAX_ITR, I40E_DEF_TX_ITR); 16289d26e4fcSRobert Mustacchi 16299d26e4fcSRobert Mustacchi i40e->i40e_rx_itr = i40e_get_prop(i40e, "rx_intr_throttle", 16309d26e4fcSRobert Mustacchi I40E_MIN_ITR, I40E_MAX_ITR, I40E_DEF_RX_ITR); 16319d26e4fcSRobert Mustacchi 16329d26e4fcSRobert Mustacchi i40e->i40e_other_itr = i40e_get_prop(i40e, "other_intr_throttle", 16339d26e4fcSRobert Mustacchi I40E_MIN_ITR, I40E_MAX_ITR, I40E_DEF_OTHER_ITR); 16349d26e4fcSRobert Mustacchi 16359d26e4fcSRobert Mustacchi if (!i40e->i40e_mr_enable) { 16369d26e4fcSRobert Mustacchi i40e->i40e_num_trqpairs = I40E_TRQPAIR_NOMSIX; 16379d26e4fcSRobert Mustacchi i40e->i40e_num_rx_groups = I40E_GROUP_NOMSIX; 16389d26e4fcSRobert Mustacchi } 16399d26e4fcSRobert Mustacchi 16409d26e4fcSRobert Mustacchi i40e_update_mtu(i40e); 16419d26e4fcSRobert Mustacchi } 16429d26e4fcSRobert Mustacchi 16439d26e4fcSRobert Mustacchi /* 16449d26e4fcSRobert Mustacchi * There are a few constraints on interrupts that we're currently imposing, some 16459d26e4fcSRobert Mustacchi * of which are restrictions from hardware. For a fuller treatment, see 16469d26e4fcSRobert Mustacchi * i40e_intr.c. 16479d26e4fcSRobert Mustacchi * 16489d26e4fcSRobert Mustacchi * Currently, to use MSI-X we require two interrupts be available though in 16499d26e4fcSRobert Mustacchi * theory we should participate in IRM and happily use more interrupts. 16509d26e4fcSRobert Mustacchi * 16519d26e4fcSRobert Mustacchi * Hardware only supports a single MSI being programmed and therefore if we 16529d26e4fcSRobert Mustacchi * don't have MSI-X interrupts available at this time, then we ratchet down the 16539d26e4fcSRobert Mustacchi * number of rings and groups available. Obviously, we only bother with a single 16549d26e4fcSRobert Mustacchi * fixed interrupt. 16559d26e4fcSRobert Mustacchi */ 16569d26e4fcSRobert Mustacchi static boolean_t 16579d26e4fcSRobert Mustacchi i40e_alloc_intr_handles(i40e_t *i40e, dev_info_t *devinfo, int intr_type) 16589d26e4fcSRobert Mustacchi { 1659396505afSPaul Winder i40e_hw_t *hw = &i40e->i40e_hw_space; 1660396505afSPaul Winder ddi_acc_handle_t rh = i40e->i40e_osdep_space.ios_reg_handle; 16619d26e4fcSRobert Mustacchi int request, count, actual, rc, min; 1662396505afSPaul Winder uint32_t reg; 16639d26e4fcSRobert Mustacchi 16649d26e4fcSRobert Mustacchi switch (intr_type) { 16659d26e4fcSRobert Mustacchi case DDI_INTR_TYPE_FIXED: 16669d26e4fcSRobert Mustacchi case DDI_INTR_TYPE_MSI: 16679d26e4fcSRobert Mustacchi request = 1; 16689d26e4fcSRobert Mustacchi min = 1; 16699d26e4fcSRobert Mustacchi break; 16709d26e4fcSRobert Mustacchi case DDI_INTR_TYPE_MSIX: 1671396505afSPaul Winder min = 2; 1672396505afSPaul Winder if (!i40e->i40e_mr_enable) { 1673396505afSPaul Winder request = 2; 1674396505afSPaul Winder break; 1675396505afSPaul Winder } 1676396505afSPaul Winder reg = I40E_READ_REG(hw, I40E_GLPCI_CNF2); 16779d26e4fcSRobert Mustacchi /* 1678396505afSPaul Winder * Should this read fail, we will drop back to using 1679396505afSPaul Winder * MSI or fixed interrupts. 16809d26e4fcSRobert Mustacchi */ 1681396505afSPaul Winder if (i40e_check_acc_handle(rh) != DDI_FM_OK) { 1682396505afSPaul Winder ddi_fm_service_impact(i40e->i40e_dip, 1683396505afSPaul Winder DDI_SERVICE_DEGRADED); 1684396505afSPaul Winder return (B_FALSE); 1685396505afSPaul Winder } 1686396505afSPaul Winder request = (reg & I40E_GLPCI_CNF2_MSI_X_PF_N_MASK) >> 1687396505afSPaul Winder I40E_GLPCI_CNF2_MSI_X_PF_N_SHIFT; 1688396505afSPaul Winder request++; /* the register value is n - 1 */ 16899d26e4fcSRobert Mustacchi break; 16909d26e4fcSRobert Mustacchi default: 16919d26e4fcSRobert Mustacchi panic("bad interrupt type passed to i40e_alloc_intr_handles: " 16929d26e4fcSRobert Mustacchi "%d", intr_type); 16939d26e4fcSRobert Mustacchi } 16949d26e4fcSRobert Mustacchi 16959d26e4fcSRobert Mustacchi rc = ddi_intr_get_nintrs(devinfo, intr_type, &count); 16969d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS || count < min) { 16979d26e4fcSRobert Mustacchi i40e_log(i40e, "Get interrupt number failed, " 16989d26e4fcSRobert Mustacchi "returned %d, count %d", rc, count); 16999d26e4fcSRobert Mustacchi return (B_FALSE); 17009d26e4fcSRobert Mustacchi } 17019d26e4fcSRobert Mustacchi 17029d26e4fcSRobert Mustacchi rc = ddi_intr_get_navail(devinfo, intr_type, &count); 17039d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS || count < min) { 17049d26e4fcSRobert Mustacchi i40e_log(i40e, "Get AVAILABLE interrupt number failed, " 17059d26e4fcSRobert Mustacchi "returned %d, count %d", rc, count); 17069d26e4fcSRobert Mustacchi return (B_FALSE); 17079d26e4fcSRobert Mustacchi } 17089d26e4fcSRobert Mustacchi 17099d26e4fcSRobert Mustacchi actual = 0; 17109d26e4fcSRobert Mustacchi i40e->i40e_intr_count = 0; 17119d26e4fcSRobert Mustacchi i40e->i40e_intr_count_max = 0; 17129d26e4fcSRobert Mustacchi i40e->i40e_intr_count_min = 0; 17139d26e4fcSRobert Mustacchi 17149d26e4fcSRobert Mustacchi i40e->i40e_intr_size = request * sizeof (ddi_intr_handle_t); 17159d26e4fcSRobert Mustacchi ASSERT(i40e->i40e_intr_size != 0); 17169d26e4fcSRobert Mustacchi i40e->i40e_intr_handles = kmem_alloc(i40e->i40e_intr_size, KM_SLEEP); 17179d26e4fcSRobert Mustacchi 17189d26e4fcSRobert Mustacchi rc = ddi_intr_alloc(devinfo, i40e->i40e_intr_handles, intr_type, 0, 17199d26e4fcSRobert Mustacchi min(request, count), &actual, DDI_INTR_ALLOC_NORMAL); 17209d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 17219d26e4fcSRobert Mustacchi i40e_log(i40e, "Interrupt allocation failed with %d.", rc); 17229d26e4fcSRobert Mustacchi goto alloc_handle_fail; 17239d26e4fcSRobert Mustacchi } 17249d26e4fcSRobert Mustacchi 17259d26e4fcSRobert Mustacchi i40e->i40e_intr_count = actual; 17269d26e4fcSRobert Mustacchi i40e->i40e_intr_count_max = request; 17279d26e4fcSRobert Mustacchi i40e->i40e_intr_count_min = min; 17289d26e4fcSRobert Mustacchi 17299d26e4fcSRobert Mustacchi if (actual < min) { 17309d26e4fcSRobert Mustacchi i40e_log(i40e, "actual (%d) is less than minimum (%d).", 17319d26e4fcSRobert Mustacchi actual, min); 17329d26e4fcSRobert Mustacchi goto alloc_handle_fail; 17339d26e4fcSRobert Mustacchi } 17349d26e4fcSRobert Mustacchi 17359d26e4fcSRobert Mustacchi /* 17369d26e4fcSRobert Mustacchi * Record the priority and capabilities for our first vector. Once 17379d26e4fcSRobert Mustacchi * we have it, that's our priority until detach time. Even if we 17389d26e4fcSRobert Mustacchi * eventually participate in IRM, our priority shouldn't change. 17399d26e4fcSRobert Mustacchi */ 17409d26e4fcSRobert Mustacchi rc = ddi_intr_get_pri(i40e->i40e_intr_handles[0], &i40e->i40e_intr_pri); 17419d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 17429d26e4fcSRobert Mustacchi i40e_log(i40e, 17439d26e4fcSRobert Mustacchi "Getting interrupt priority failed with %d.", rc); 17449d26e4fcSRobert Mustacchi goto alloc_handle_fail; 17459d26e4fcSRobert Mustacchi } 17469d26e4fcSRobert Mustacchi 17479d26e4fcSRobert Mustacchi rc = ddi_intr_get_cap(i40e->i40e_intr_handles[0], &i40e->i40e_intr_cap); 17489d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 17499d26e4fcSRobert Mustacchi i40e_log(i40e, 17509d26e4fcSRobert Mustacchi "Getting interrupt capabilities failed with %d.", rc); 17519d26e4fcSRobert Mustacchi goto alloc_handle_fail; 17529d26e4fcSRobert Mustacchi } 17539d26e4fcSRobert Mustacchi 17549d26e4fcSRobert Mustacchi i40e->i40e_intr_type = intr_type; 17559d26e4fcSRobert Mustacchi return (B_TRUE); 17569d26e4fcSRobert Mustacchi 17579d26e4fcSRobert Mustacchi alloc_handle_fail: 17589d26e4fcSRobert Mustacchi 17599d26e4fcSRobert Mustacchi i40e_rem_intrs(i40e); 17609d26e4fcSRobert Mustacchi return (B_FALSE); 17619d26e4fcSRobert Mustacchi } 17629d26e4fcSRobert Mustacchi 17639d26e4fcSRobert Mustacchi static boolean_t 17649d26e4fcSRobert Mustacchi i40e_alloc_intrs(i40e_t *i40e, dev_info_t *devinfo) 17659d26e4fcSRobert Mustacchi { 1766234a3cfbSPaul Winder i40e_hw_t *hw = &i40e->i40e_hw_space; 17679d26e4fcSRobert Mustacchi int intr_types, rc; 1768b9d34b9dSRobert Mustacchi uint_t max_trqpairs; 1769b9d34b9dSRobert Mustacchi 1770b9d34b9dSRobert Mustacchi if (i40e_is_x722(i40e)) { 1771b9d34b9dSRobert Mustacchi max_trqpairs = I40E_722_MAX_TC_QUEUES; 1772b9d34b9dSRobert Mustacchi } else { 1773b9d34b9dSRobert Mustacchi max_trqpairs = I40E_710_MAX_TC_QUEUES; 1774b9d34b9dSRobert Mustacchi } 17759d26e4fcSRobert Mustacchi 17769d26e4fcSRobert Mustacchi rc = ddi_intr_get_supported_types(devinfo, &intr_types); 17779d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 17789d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to get supported interrupt types: %d", 17799d26e4fcSRobert Mustacchi rc); 17809d26e4fcSRobert Mustacchi return (B_FALSE); 17819d26e4fcSRobert Mustacchi } 17829d26e4fcSRobert Mustacchi 17839d26e4fcSRobert Mustacchi i40e->i40e_intr_type = 0; 17849d26e4fcSRobert Mustacchi 178509aee612SRyan Zezeski /* 178609aee612SRyan Zezeski * We need to determine the number of queue pairs per traffic 178709aee612SRyan Zezeski * class. We only have one traffic class (TC0), so we'll base 178809aee612SRyan Zezeski * this off the number of interrupts provided. Furthermore, 178909aee612SRyan Zezeski * since we only use one traffic class, the number of queues 179009aee612SRyan Zezeski * per traffic class and per VSI are the same. 179109aee612SRyan Zezeski */ 17929d26e4fcSRobert Mustacchi if ((intr_types & DDI_INTR_TYPE_MSIX) && 179309aee612SRyan Zezeski (i40e->i40e_intr_force <= I40E_INTR_MSIX) && 179409aee612SRyan Zezeski (i40e_alloc_intr_handles(i40e, devinfo, DDI_INTR_TYPE_MSIX))) { 1795234a3cfbSPaul Winder uint32_t n, qp_cap, num_trqpairs; 179609aee612SRyan Zezeski 179709aee612SRyan Zezeski /* 179809aee612SRyan Zezeski * While we want the number of queue pairs to match 179909aee612SRyan Zezeski * the number of interrupts, we must keep stay in 180009aee612SRyan Zezeski * bounds of the maximum number of queues per traffic 180109aee612SRyan Zezeski * class. We subtract one from i40e_intr_count to 180209aee612SRyan Zezeski * account for interrupt zero; which is currently 180309aee612SRyan Zezeski * restricted to admin queue commands and other 180409aee612SRyan Zezeski * interrupt causes. 180509aee612SRyan Zezeski */ 180609aee612SRyan Zezeski n = MIN(i40e->i40e_intr_count - 1, max_trqpairs); 180709aee612SRyan Zezeski ASSERT3U(n, >, 0); 180809aee612SRyan Zezeski 180909aee612SRyan Zezeski /* 181009aee612SRyan Zezeski * Round up to the nearest power of two to ensure that 181109aee612SRyan Zezeski * the QBASE aligns with the TC size which must be 181209aee612SRyan Zezeski * programmed as a power of two. See the queue mapping 181309aee612SRyan Zezeski * description in section 7.4.9.5.5.1. 181409aee612SRyan Zezeski * 181509aee612SRyan Zezeski * If i40e_intr_count - 1 is not a power of two then 181609aee612SRyan Zezeski * some queue pairs on the same VSI will have to share 181709aee612SRyan Zezeski * an interrupt. 181809aee612SRyan Zezeski * 181909aee612SRyan Zezeski * We may want to revisit this logic in a future where 182009aee612SRyan Zezeski * we have more interrupts and more VSIs. Otherwise, 182109aee612SRyan Zezeski * each VSI will use as many interrupts as possible. 182209aee612SRyan Zezeski * Using more QPs per VSI means better RSS for each 182309aee612SRyan Zezeski * group, but at the same time may require more 182409aee612SRyan Zezeski * sharing of interrupts across VSIs. This may be a 182509aee612SRyan Zezeski * good candidate for a .conf tunable. 182609aee612SRyan Zezeski */ 182709aee612SRyan Zezeski n = 0x1 << ddi_fls(n); 182809aee612SRyan Zezeski i40e->i40e_num_trqpairs_per_vsi = n; 1829234a3cfbSPaul Winder 1830234a3cfbSPaul Winder /* 1831234a3cfbSPaul Winder * Make sure the number of tx/rx qpairs does not exceed 1832234a3cfbSPaul Winder * the device's capabilities. 1833234a3cfbSPaul Winder */ 183409aee612SRyan Zezeski ASSERT3U(i40e->i40e_num_rx_groups, >, 0); 1835234a3cfbSPaul Winder qp_cap = MIN(hw->func_caps.num_rx_qp, hw->func_caps.num_tx_qp); 1836234a3cfbSPaul Winder num_trqpairs = i40e->i40e_num_trqpairs_per_vsi * 183709aee612SRyan Zezeski i40e->i40e_num_rx_groups; 1838234a3cfbSPaul Winder if (num_trqpairs > qp_cap) { 1839234a3cfbSPaul Winder i40e->i40e_num_rx_groups = MAX(1, qp_cap / 1840234a3cfbSPaul Winder i40e->i40e_num_trqpairs_per_vsi); 1841234a3cfbSPaul Winder num_trqpairs = i40e->i40e_num_trqpairs_per_vsi * 1842234a3cfbSPaul Winder i40e->i40e_num_rx_groups; 1843234a3cfbSPaul Winder i40e_log(i40e, "Rx groups restricted to %u", 1844234a3cfbSPaul Winder i40e->i40e_num_rx_groups); 1845234a3cfbSPaul Winder } 1846234a3cfbSPaul Winder ASSERT3U(num_trqpairs, >, 0); 1847234a3cfbSPaul Winder i40e->i40e_num_trqpairs = num_trqpairs; 184809aee612SRyan Zezeski return (B_TRUE); 18499d26e4fcSRobert Mustacchi } 18509d26e4fcSRobert Mustacchi 18519d26e4fcSRobert Mustacchi /* 18529d26e4fcSRobert Mustacchi * We only use multiple transmit/receive pairs when MSI-X interrupts are 18539d26e4fcSRobert Mustacchi * available due to the fact that the device basically only supports a 18549d26e4fcSRobert Mustacchi * single MSI interrupt. 18559d26e4fcSRobert Mustacchi */ 18569d26e4fcSRobert Mustacchi i40e->i40e_num_trqpairs = I40E_TRQPAIR_NOMSIX; 185709aee612SRyan Zezeski i40e->i40e_num_trqpairs_per_vsi = i40e->i40e_num_trqpairs; 18589d26e4fcSRobert Mustacchi i40e->i40e_num_rx_groups = I40E_GROUP_NOMSIX; 18599d26e4fcSRobert Mustacchi 18609d26e4fcSRobert Mustacchi if ((intr_types & DDI_INTR_TYPE_MSI) && 18619d26e4fcSRobert Mustacchi (i40e->i40e_intr_force <= I40E_INTR_MSI)) { 18629d26e4fcSRobert Mustacchi if (i40e_alloc_intr_handles(i40e, devinfo, DDI_INTR_TYPE_MSI)) 18639d26e4fcSRobert Mustacchi return (B_TRUE); 18649d26e4fcSRobert Mustacchi } 18659d26e4fcSRobert Mustacchi 18669d26e4fcSRobert Mustacchi if (intr_types & DDI_INTR_TYPE_FIXED) { 18679d26e4fcSRobert Mustacchi if (i40e_alloc_intr_handles(i40e, devinfo, DDI_INTR_TYPE_FIXED)) 18689d26e4fcSRobert Mustacchi return (B_TRUE); 18699d26e4fcSRobert Mustacchi } 18709d26e4fcSRobert Mustacchi 18719d26e4fcSRobert Mustacchi return (B_FALSE); 18729d26e4fcSRobert Mustacchi } 18739d26e4fcSRobert Mustacchi 18749d26e4fcSRobert Mustacchi /* 18759d26e4fcSRobert Mustacchi * Map different interrupts to MSI-X vectors. 18769d26e4fcSRobert Mustacchi */ 18779d26e4fcSRobert Mustacchi static boolean_t 18789d26e4fcSRobert Mustacchi i40e_map_intrs_to_vectors(i40e_t *i40e) 18799d26e4fcSRobert Mustacchi { 18809d26e4fcSRobert Mustacchi if (i40e->i40e_intr_type != DDI_INTR_TYPE_MSIX) { 18819d26e4fcSRobert Mustacchi return (B_TRUE); 18829d26e4fcSRobert Mustacchi } 18839d26e4fcSRobert Mustacchi 18849d26e4fcSRobert Mustacchi /* 188509aee612SRyan Zezeski * Each queue pair is mapped to a single interrupt, so 188609aee612SRyan Zezeski * transmit and receive interrupts for a given queue share the 188709aee612SRyan Zezeski * same vector. Vector zero is reserved for the admin queue. 18889d26e4fcSRobert Mustacchi */ 188909aee612SRyan Zezeski for (uint_t i = 0; i < i40e->i40e_num_trqpairs; i++) { 189009aee612SRyan Zezeski uint_t vector = i % (i40e->i40e_intr_count - 1); 18919d26e4fcSRobert Mustacchi 189209aee612SRyan Zezeski i40e->i40e_trqpairs[i].itrq_rx_intrvec = vector + 1; 189309aee612SRyan Zezeski i40e->i40e_trqpairs[i].itrq_tx_intrvec = vector + 1; 1894396505afSPaul Winder } 18959d26e4fcSRobert Mustacchi 18969d26e4fcSRobert Mustacchi return (B_TRUE); 18979d26e4fcSRobert Mustacchi } 18989d26e4fcSRobert Mustacchi 18999d26e4fcSRobert Mustacchi static boolean_t 19009d26e4fcSRobert Mustacchi i40e_add_intr_handlers(i40e_t *i40e) 19019d26e4fcSRobert Mustacchi { 19029d26e4fcSRobert Mustacchi int rc, vector; 19039d26e4fcSRobert Mustacchi 19049d26e4fcSRobert Mustacchi switch (i40e->i40e_intr_type) { 19059d26e4fcSRobert Mustacchi case DDI_INTR_TYPE_MSIX: 19069d26e4fcSRobert Mustacchi for (vector = 0; vector < i40e->i40e_intr_count; vector++) { 19079d26e4fcSRobert Mustacchi rc = ddi_intr_add_handler( 19089d26e4fcSRobert Mustacchi i40e->i40e_intr_handles[vector], 19099d26e4fcSRobert Mustacchi (ddi_intr_handler_t *)i40e_intr_msix, i40e, 19109d26e4fcSRobert Mustacchi (void *)(uintptr_t)vector); 19119d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 19129d26e4fcSRobert Mustacchi i40e_log(i40e, "Add interrupt handler (MSI-X) " 19139d26e4fcSRobert Mustacchi "failed: return %d, vector %d", rc, vector); 19149d26e4fcSRobert Mustacchi for (vector--; vector >= 0; vector--) { 19159d26e4fcSRobert Mustacchi (void) ddi_intr_remove_handler( 19169d26e4fcSRobert Mustacchi i40e->i40e_intr_handles[vector]); 19179d26e4fcSRobert Mustacchi } 19189d26e4fcSRobert Mustacchi return (B_FALSE); 19199d26e4fcSRobert Mustacchi } 19209d26e4fcSRobert Mustacchi } 19219d26e4fcSRobert Mustacchi break; 19229d26e4fcSRobert Mustacchi case DDI_INTR_TYPE_MSI: 19239d26e4fcSRobert Mustacchi rc = ddi_intr_add_handler(i40e->i40e_intr_handles[0], 19249d26e4fcSRobert Mustacchi (ddi_intr_handler_t *)i40e_intr_msi, i40e, NULL); 19259d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 19269d26e4fcSRobert Mustacchi i40e_log(i40e, "Add interrupt handler (MSI) failed: " 19279d26e4fcSRobert Mustacchi "return %d", rc); 19289d26e4fcSRobert Mustacchi return (B_FALSE); 19299d26e4fcSRobert Mustacchi } 19309d26e4fcSRobert Mustacchi break; 19319d26e4fcSRobert Mustacchi case DDI_INTR_TYPE_FIXED: 19329d26e4fcSRobert Mustacchi rc = ddi_intr_add_handler(i40e->i40e_intr_handles[0], 19339d26e4fcSRobert Mustacchi (ddi_intr_handler_t *)i40e_intr_legacy, i40e, NULL); 19349d26e4fcSRobert Mustacchi if (rc != DDI_SUCCESS) { 19359d26e4fcSRobert Mustacchi i40e_log(i40e, "Add interrupt handler (legacy) failed:" 19369d26e4fcSRobert Mustacchi " return %d", rc); 19379d26e4fcSRobert Mustacchi return (B_FALSE); 19389d26e4fcSRobert Mustacchi } 19399d26e4fcSRobert Mustacchi break; 19409d26e4fcSRobert Mustacchi default: 19419d26e4fcSRobert Mustacchi /* Cast to pacify lint */ 19429d26e4fcSRobert Mustacchi panic("i40e_intr_type %p contains an unknown type: %d", 19439d26e4fcSRobert Mustacchi (void *)i40e, i40e->i40e_intr_type); 19449d26e4fcSRobert Mustacchi } 19459d26e4fcSRobert Mustacchi 19469d26e4fcSRobert Mustacchi return (B_TRUE); 19479d26e4fcSRobert Mustacchi } 19489d26e4fcSRobert Mustacchi 19499d26e4fcSRobert Mustacchi /* 19509d26e4fcSRobert Mustacchi * Perform periodic checks. Longer term, we should be thinking about additional 19519d26e4fcSRobert Mustacchi * things here: 19529d26e4fcSRobert Mustacchi * 19539d26e4fcSRobert Mustacchi * o Stall Detection 19549d26e4fcSRobert Mustacchi * o Temperature sensor detection 19559d26e4fcSRobert Mustacchi * o Device resetting 19569d26e4fcSRobert Mustacchi * o Statistics updating to avoid wraparound 19579d26e4fcSRobert Mustacchi */ 19589d26e4fcSRobert Mustacchi static void 19599d26e4fcSRobert Mustacchi i40e_timer(void *arg) 19609d26e4fcSRobert Mustacchi { 19619d26e4fcSRobert Mustacchi i40e_t *i40e = arg; 19629d26e4fcSRobert Mustacchi 19639d26e4fcSRobert Mustacchi mutex_enter(&i40e->i40e_general_lock); 19649d26e4fcSRobert Mustacchi i40e_link_check(i40e); 19659d26e4fcSRobert Mustacchi mutex_exit(&i40e->i40e_general_lock); 19669d26e4fcSRobert Mustacchi } 19679d26e4fcSRobert Mustacchi 19689d26e4fcSRobert Mustacchi /* 19699d26e4fcSRobert Mustacchi * Get the hardware state, and scribble away anything that needs scribbling. 19709d26e4fcSRobert Mustacchi */ 19719d26e4fcSRobert Mustacchi static void 19729d26e4fcSRobert Mustacchi i40e_get_hw_state(i40e_t *i40e, i40e_hw_t *hw) 19739d26e4fcSRobert Mustacchi { 19749d26e4fcSRobert Mustacchi int rc; 19759d26e4fcSRobert Mustacchi 19769d26e4fcSRobert Mustacchi ASSERT(MUTEX_HELD(&i40e->i40e_general_lock)); 19779d26e4fcSRobert Mustacchi 19789d26e4fcSRobert Mustacchi (void) i40e_aq_get_link_info(hw, TRUE, NULL, NULL); 19799d26e4fcSRobert Mustacchi i40e_link_check(i40e); 19809d26e4fcSRobert Mustacchi 19819d26e4fcSRobert Mustacchi /* 19829d26e4fcSRobert Mustacchi * Try and determine our PHY. Note that we may have to retry to and 19839d26e4fcSRobert Mustacchi * delay to detect fiber correctly. 19849d26e4fcSRobert Mustacchi */ 19859d26e4fcSRobert Mustacchi rc = i40e_aq_get_phy_capabilities(hw, B_FALSE, B_TRUE, &i40e->i40e_phy, 19869d26e4fcSRobert Mustacchi NULL); 19879d26e4fcSRobert Mustacchi if (rc == I40E_ERR_UNKNOWN_PHY) { 19889d26e4fcSRobert Mustacchi i40e_msec_delay(200); 19899d26e4fcSRobert Mustacchi rc = i40e_aq_get_phy_capabilities(hw, B_FALSE, B_TRUE, 19909d26e4fcSRobert Mustacchi &i40e->i40e_phy, NULL); 19919d26e4fcSRobert Mustacchi } 19929d26e4fcSRobert Mustacchi 19939d26e4fcSRobert Mustacchi if (rc != I40E_SUCCESS) { 19949d26e4fcSRobert Mustacchi if (rc == I40E_ERR_UNKNOWN_PHY) { 19959d26e4fcSRobert Mustacchi i40e_error(i40e, "encountered unknown PHY type, " 19969d26e4fcSRobert Mustacchi "not attaching."); 19979d26e4fcSRobert Mustacchi } else { 19989d26e4fcSRobert Mustacchi i40e_error(i40e, "error getting physical capabilities: " 19999d26e4fcSRobert Mustacchi "%d, %d", rc, hw->aq.asq_last_status); 20009d26e4fcSRobert Mustacchi } 20019d26e4fcSRobert Mustacchi } 20029d26e4fcSRobert Mustacchi 20039d26e4fcSRobert Mustacchi rc = i40e_update_link_info(hw); 20049d26e4fcSRobert Mustacchi if (rc != I40E_SUCCESS) { 20059d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to update link information: %d", rc); 20069d26e4fcSRobert Mustacchi } 20079d26e4fcSRobert Mustacchi 20089d26e4fcSRobert Mustacchi /* 20099d26e4fcSRobert Mustacchi * In general, we don't want to mask off (as in stop from being a cause) 20109d26e4fcSRobert Mustacchi * any of the interrupts that the phy might be able to generate. 20119d26e4fcSRobert Mustacchi */ 20129d26e4fcSRobert Mustacchi rc = i40e_aq_set_phy_int_mask(hw, 0, NULL); 20139d26e4fcSRobert Mustacchi if (rc != I40E_SUCCESS) { 20149d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to update phy link mask: %d", rc); 20159d26e4fcSRobert Mustacchi } 20169d26e4fcSRobert Mustacchi } 20179d26e4fcSRobert Mustacchi 20189d26e4fcSRobert Mustacchi /* 20199d26e4fcSRobert Mustacchi * Go through and re-initialize any existing filters that we may have set up for 20209d26e4fcSRobert Mustacchi * this device. Note that we would only expect them to exist if hardware had 20219d26e4fcSRobert Mustacchi * already been initialized and we had just reset it. While we're not 20229d26e4fcSRobert Mustacchi * implementing this yet, we're keeping this around for when we add reset 20239d26e4fcSRobert Mustacchi * capabilities, so this isn't forgotten. 20249d26e4fcSRobert Mustacchi */ 20259d26e4fcSRobert Mustacchi /* ARGSUSED */ 20269d26e4fcSRobert Mustacchi static void 20279d26e4fcSRobert Mustacchi i40e_init_macaddrs(i40e_t *i40e, i40e_hw_t *hw) 20289d26e4fcSRobert Mustacchi { 20299d26e4fcSRobert Mustacchi } 20309d26e4fcSRobert Mustacchi 20319d26e4fcSRobert Mustacchi /* 203209aee612SRyan Zezeski * Set the properties which have common values across all the VSIs. 203309aee612SRyan Zezeski * Consult the "Add VSI" command section (7.4.9.5.5.1) for a 203409aee612SRyan Zezeski * complete description of these properties. 20359d26e4fcSRobert Mustacchi */ 203609aee612SRyan Zezeski static void 203709aee612SRyan Zezeski i40e_set_shared_vsi_props(i40e_t *i40e, 203809aee612SRyan Zezeski struct i40e_aqc_vsi_properties_data *info, uint_t vsi_idx) 20399d26e4fcSRobert Mustacchi { 204009aee612SRyan Zezeski uint_t tc_queues; 204109aee612SRyan Zezeski uint16_t vsi_qp_base; 20429d26e4fcSRobert Mustacchi 204309aee612SRyan Zezeski /* 204409aee612SRyan Zezeski * It's important that we use bitwise-OR here; callers to this 204509aee612SRyan Zezeski * function might enable other sections before calling this 204609aee612SRyan Zezeski * function. 204709aee612SRyan Zezeski */ 204809aee612SRyan Zezeski info->valid_sections |= LE_16(I40E_AQ_VSI_PROP_QUEUE_MAP_VALID | 204909aee612SRyan Zezeski I40E_AQ_VSI_PROP_VLAN_VALID); 2050396505afSPaul Winder 20519d26e4fcSRobert Mustacchi /* 205209aee612SRyan Zezeski * Calculate the starting QP index for this VSI. This base is 205309aee612SRyan Zezeski * relative to the PF queue space; so a value of 0 for PF#1 205409aee612SRyan Zezeski * represents the absolute index PFLAN_QALLOC_FIRSTQ for PF#1. 20559d26e4fcSRobert Mustacchi */ 205609aee612SRyan Zezeski vsi_qp_base = vsi_idx * i40e->i40e_num_trqpairs_per_vsi; 205709aee612SRyan Zezeski info->mapping_flags = LE_16(I40E_AQ_VSI_QUE_MAP_CONTIG); 205809aee612SRyan Zezeski info->queue_mapping[0] = 205909aee612SRyan Zezeski LE_16((vsi_qp_base << I40E_AQ_VSI_QUEUE_SHIFT) & 2060508a0e8cSRob Johnston I40E_AQ_VSI_QUEUE_MASK); 2061396505afSPaul Winder 2062396505afSPaul Winder /* 206309aee612SRyan Zezeski * tc_queues determines the size of the traffic class, where 206409aee612SRyan Zezeski * the size is 2^^tc_queues to a maximum of 64 for the X710 206509aee612SRyan Zezeski * and 128 for the X722. 2066b9d34b9dSRobert Mustacchi * 2067396505afSPaul Winder * Some examples: 2068508a0e8cSRob Johnston * i40e_num_trqpairs_per_vsi == 1 => tc_queues = 0, 2^^0 = 1. 2069508a0e8cSRob Johnston * i40e_num_trqpairs_per_vsi == 7 => tc_queues = 3, 2^^3 = 8. 2070508a0e8cSRob Johnston * i40e_num_trqpairs_per_vsi == 8 => tc_queues = 3, 2^^3 = 8. 2071508a0e8cSRob Johnston * i40e_num_trqpairs_per_vsi == 9 => tc_queues = 4, 2^^4 = 16. 2072508a0e8cSRob Johnston * i40e_num_trqpairs_per_vsi == 17 => tc_queues = 5, 2^^5 = 32. 2073508a0e8cSRob Johnston * i40e_num_trqpairs_per_vsi == 64 => tc_queues = 6, 2^^6 = 64. 2074396505afSPaul Winder */ 207509aee612SRyan Zezeski tc_queues = ddi_fls(i40e->i40e_num_trqpairs_per_vsi - 1); 2076396505afSPaul Winder 207709aee612SRyan Zezeski /* 207809aee612SRyan Zezeski * The TC queue mapping is in relation to the VSI queue space. 207909aee612SRyan Zezeski * Since we are only using one traffic class (TC0) we always 208009aee612SRyan Zezeski * start at queue offset 0. 208109aee612SRyan Zezeski */ 208209aee612SRyan Zezeski info->tc_mapping[0] = 208309aee612SRyan Zezeski LE_16(((0 << I40E_AQ_VSI_TC_QUE_OFFSET_SHIFT) & 2084508a0e8cSRob Johnston I40E_AQ_VSI_TC_QUE_OFFSET_MASK) | 2085508a0e8cSRob Johnston ((tc_queues << I40E_AQ_VSI_TC_QUE_NUMBER_SHIFT) & 2086508a0e8cSRob Johnston I40E_AQ_VSI_TC_QUE_NUMBER_MASK)); 20879d26e4fcSRobert Mustacchi 208809aee612SRyan Zezeski /* 208909aee612SRyan Zezeski * I40E_AQ_VSI_PVLAN_MODE_ALL ("VLAN driver insertion mode") 209009aee612SRyan Zezeski * 209109aee612SRyan Zezeski * Allow tagged and untagged packets to be sent to this 209209aee612SRyan Zezeski * VSI from the host. 209309aee612SRyan Zezeski * 209409aee612SRyan Zezeski * I40E_AQ_VSI_PVLAN_EMOD_NOTHING ("VLAN and UP expose mode") 209509aee612SRyan Zezeski * 209609aee612SRyan Zezeski * Leave the tag on the frame and place no VLAN 209709aee612SRyan Zezeski * information in the descriptor. We want this mode 209809aee612SRyan Zezeski * because our MAC layer will take care of the VLAN tag, 209909aee612SRyan Zezeski * if there is one. 210009aee612SRyan Zezeski */ 210109aee612SRyan Zezeski info->port_vlan_flags = I40E_AQ_VSI_PVLAN_MODE_ALL | 21029d26e4fcSRobert Mustacchi I40E_AQ_VSI_PVLAN_EMOD_NOTHING; 210309aee612SRyan Zezeski } 21049d26e4fcSRobert Mustacchi 210509aee612SRyan Zezeski /* 210609aee612SRyan Zezeski * Delete the VSI at this index, if one exists. We assume there is no 210709aee612SRyan Zezeski * action we can take if this command fails but to log the failure. 210809aee612SRyan Zezeski */ 210909aee612SRyan Zezeski static void 211009aee612SRyan Zezeski i40e_delete_vsi(i40e_t *i40e, uint_t idx) 211109aee612SRyan Zezeski { 211209aee612SRyan Zezeski i40e_hw_t *hw = &i40e->i40e_hw_space; 211309aee612SRyan Zezeski uint16_t seid = i40e->i40e_vsis[idx].iv_seid; 21149d26e4fcSRobert Mustacchi 211509aee612SRyan Zezeski if (seid != 0) { 211609aee612SRyan Zezeski int rc; 21179d26e4fcSRobert Mustacchi 211809aee612SRyan Zezeski rc = i40e_aq_delete_element(hw, seid, NULL); 211909aee612SRyan Zezeski 212009aee612SRyan Zezeski if (rc != I40E_SUCCESS) { 212109aee612SRyan Zezeski i40e_error(i40e, "Failed to delete VSI %d: %d", 212209aee612SRyan Zezeski rc, hw->aq.asq_last_status); 212309aee612SRyan Zezeski } 212409aee612SRyan Zezeski 212509aee612SRyan Zezeski i40e->i40e_vsis[idx].iv_seid = 0; 212609aee612SRyan Zezeski } 212709aee612SRyan Zezeski } 212809aee612SRyan Zezeski 212909aee612SRyan Zezeski /* 213009aee612SRyan Zezeski * Add a new VSI. 213109aee612SRyan Zezeski */ 213209aee612SRyan Zezeski static boolean_t 213309aee612SRyan Zezeski i40e_add_vsi(i40e_t *i40e, i40e_hw_t *hw, uint_t idx) 213409aee612SRyan Zezeski { 213509aee612SRyan Zezeski struct i40e_vsi_context ctx; 213609aee612SRyan Zezeski i40e_rx_group_t *rxg; 213709aee612SRyan Zezeski int rc; 213809aee612SRyan Zezeski 213909aee612SRyan Zezeski /* 214009aee612SRyan Zezeski * The default VSI is created by the controller. This function 214109aee612SRyan Zezeski * creates new, non-defualt VSIs only. 214209aee612SRyan Zezeski */ 214309aee612SRyan Zezeski ASSERT3U(idx, !=, 0); 214409aee612SRyan Zezeski 214509aee612SRyan Zezeski bzero(&ctx, sizeof (struct i40e_vsi_context)); 214609aee612SRyan Zezeski ctx.uplink_seid = i40e->i40e_veb_seid; 214709aee612SRyan Zezeski ctx.pf_num = hw->pf_id; 214809aee612SRyan Zezeski ctx.flags = I40E_AQ_VSI_TYPE_PF; 214909aee612SRyan Zezeski ctx.connection_type = I40E_AQ_VSI_CONN_TYPE_NORMAL; 215009aee612SRyan Zezeski i40e_set_shared_vsi_props(i40e, &ctx.info, idx); 215109aee612SRyan Zezeski 215209aee612SRyan Zezeski rc = i40e_aq_add_vsi(hw, &ctx, NULL); 215309aee612SRyan Zezeski if (rc != I40E_SUCCESS) { 215409aee612SRyan Zezeski i40e_error(i40e, "i40e_aq_add_vsi() failed %d: %d", rc, 215509aee612SRyan Zezeski hw->aq.asq_last_status); 21569d26e4fcSRobert Mustacchi return (B_FALSE); 21579d26e4fcSRobert Mustacchi } 21589d26e4fcSRobert Mustacchi 215909aee612SRyan Zezeski rxg = &i40e->i40e_rx_groups[idx]; 216009aee612SRyan Zezeski rxg->irg_vsi_seid = ctx.seid; 216109aee612SRyan Zezeski i40e->i40e_vsis[idx].iv_number = ctx.vsi_number; 216209aee612SRyan Zezeski i40e->i40e_vsis[idx].iv_seid = ctx.seid; 216309aee612SRyan Zezeski i40e->i40e_vsis[idx].iv_stats_id = LE_16(ctx.info.stat_counter_idx); 216409aee612SRyan Zezeski 216509aee612SRyan Zezeski if (i40e_stat_vsi_init(i40e, idx) == B_FALSE) 216609aee612SRyan Zezeski return (B_FALSE); 21679d26e4fcSRobert Mustacchi 21689d26e4fcSRobert Mustacchi return (B_TRUE); 21699d26e4fcSRobert Mustacchi } 21709d26e4fcSRobert Mustacchi 2171b9d34b9dSRobert Mustacchi /* 217209aee612SRyan Zezeski * Configure the hardware for the Default Virtual Station Interface (VSI). 2173b9d34b9dSRobert Mustacchi */ 2174b9d34b9dSRobert Mustacchi static boolean_t 217509aee612SRyan Zezeski i40e_config_def_vsi(i40e_t *i40e, i40e_hw_t *hw) 2176b9d34b9dSRobert Mustacchi { 217709aee612SRyan Zezeski struct i40e_vsi_context ctx; 217809aee612SRyan Zezeski i40e_rx_group_t *def_rxg; 217909aee612SRyan Zezeski int err; 218009aee612SRyan Zezeski struct i40e_aqc_remove_macvlan_element_data filt; 2181b9d34b9dSRobert Mustacchi 218209aee612SRyan Zezeski bzero(&ctx, sizeof (struct i40e_vsi_context)); 218309aee612SRyan Zezeski ctx.seid = I40E_DEF_VSI_SEID(i40e); 218409aee612SRyan Zezeski ctx.pf_num = hw->pf_id; 218509aee612SRyan Zezeski err = i40e_aq_get_vsi_params(hw, &ctx, NULL); 218609aee612SRyan Zezeski if (err != I40E_SUCCESS) { 218709aee612SRyan Zezeski i40e_error(i40e, "get VSI params failed with %d", err); 218809aee612SRyan Zezeski return (B_FALSE); 218909aee612SRyan Zezeski } 2190b9d34b9dSRobert Mustacchi 219109aee612SRyan Zezeski ctx.info.valid_sections = 0; 219209aee612SRyan Zezeski i40e->i40e_vsis[0].iv_number = ctx.vsi_number; 219309aee612SRyan Zezeski i40e->i40e_vsis[0].iv_stats_id = LE_16(ctx.info.stat_counter_idx); 219409aee612SRyan Zezeski if (i40e_stat_vsi_init(i40e, 0) == B_FALSE) 219509aee612SRyan Zezeski return (B_FALSE); 219609aee612SRyan Zezeski 219709aee612SRyan Zezeski i40e_set_shared_vsi_props(i40e, &ctx.info, I40E_DEF_VSI_IDX); 219809aee612SRyan Zezeski 219909aee612SRyan Zezeski err = i40e_aq_update_vsi_params(hw, &ctx, NULL); 220009aee612SRyan Zezeski if (err != I40E_SUCCESS) { 220109aee612SRyan Zezeski i40e_error(i40e, "Update VSI params failed with %d", err); 220209aee612SRyan Zezeski return (B_FALSE); 220309aee612SRyan Zezeski } 220409aee612SRyan Zezeski 220509aee612SRyan Zezeski def_rxg = &i40e->i40e_rx_groups[0]; 220609aee612SRyan Zezeski def_rxg->irg_vsi_seid = I40E_DEF_VSI_SEID(i40e); 220709aee612SRyan Zezeski 220809aee612SRyan Zezeski /* 220935c41becSRyan Zezeski * We have seen three different behaviors in regards to the 221035c41becSRyan Zezeski * Default VSI and its implicit L2 MAC+VLAN filter. 221135c41becSRyan Zezeski * 221235c41becSRyan Zezeski * 1. It has an implicit filter for the factory MAC address 221335c41becSRyan Zezeski * and this filter counts against 'ifr_nmacfilt_used'. 221435c41becSRyan Zezeski * 221535c41becSRyan Zezeski * 2. It has an implicit filter for the factory MAC address 221635c41becSRyan Zezeski * and this filter DOES NOT count against 'ifr_nmacfilt_used'. 221735c41becSRyan Zezeski * 221835c41becSRyan Zezeski * 3. It DOES NOT have an implicit filter. 221935c41becSRyan Zezeski * 222035c41becSRyan Zezeski * All three of these cases are accounted for below. If we 222135c41becSRyan Zezeski * fail to remove the L2 filter (ENOENT) then we assume there 222235c41becSRyan Zezeski * wasn't one. Otherwise, if we successfully remove the 222335c41becSRyan Zezeski * filter, we make sure to update the 'ifr_nmacfilt_used' 222435c41becSRyan Zezeski * count accordingly. 222535c41becSRyan Zezeski * 222635c41becSRyan Zezeski * We remove this filter to prevent duplicate delivery of 222735c41becSRyan Zezeski * packets destined for the primary MAC address as DLS will 222835c41becSRyan Zezeski * create the same filter on a non-default VSI for the primary 222935c41becSRyan Zezeski * MAC client. 223035c41becSRyan Zezeski * 223135c41becSRyan Zezeski * If you change the following code please test it across as 223235c41becSRyan Zezeski * many X700 series controllers and firmware revisions as you 223335c41becSRyan Zezeski * can. 223409aee612SRyan Zezeski */ 223509aee612SRyan Zezeski bzero(&filt, sizeof (filt)); 223609aee612SRyan Zezeski bcopy(hw->mac.port_addr, filt.mac_addr, ETHERADDRL); 223709aee612SRyan Zezeski filt.flags = I40E_AQC_MACVLAN_DEL_PERFECT_MATCH; 223809aee612SRyan Zezeski filt.vlan_tag = 0; 223909aee612SRyan Zezeski 224009aee612SRyan Zezeski ASSERT3U(i40e->i40e_resources.ifr_nmacfilt_used, <=, 1); 224135c41becSRyan Zezeski i40e_log(i40e, "Num L2 filters: %u", 224235c41becSRyan Zezeski i40e->i40e_resources.ifr_nmacfilt_used); 224309aee612SRyan Zezeski 224409aee612SRyan Zezeski err = i40e_aq_remove_macvlan(hw, I40E_DEF_VSI_SEID(i40e), &filt, 1, 224509aee612SRyan Zezeski NULL); 224635c41becSRyan Zezeski if (err == I40E_SUCCESS) { 224735c41becSRyan Zezeski i40e_log(i40e, 224835c41becSRyan Zezeski "Removed L2 filter from Default VSI with SEID %u", 224935c41becSRyan Zezeski I40E_DEF_VSI_SEID(i40e)); 225035c41becSRyan Zezeski } else if (hw->aq.asq_last_status == ENOENT) { 225135c41becSRyan Zezeski i40e_log(i40e, 225235c41becSRyan Zezeski "No L2 filter for Default VSI with SEID %u", 225335c41becSRyan Zezeski I40E_DEF_VSI_SEID(i40e)); 225435c41becSRyan Zezeski } else { 225535c41becSRyan Zezeski i40e_error(i40e, "Failed to remove L2 filter from" 225635c41becSRyan Zezeski " Default VSI with SEID %u: %d (%d)", 225735c41becSRyan Zezeski I40E_DEF_VSI_SEID(i40e), err, hw->aq.asq_last_status); 225835c41becSRyan Zezeski 225909aee612SRyan Zezeski return (B_FALSE); 226009aee612SRyan Zezeski } 226109aee612SRyan Zezeski 226209aee612SRyan Zezeski /* 226309aee612SRyan Zezeski * As mentioned above, the controller created an implicit L2 226409aee612SRyan Zezeski * filter for the primary MAC. We want to remove both the 226509aee612SRyan Zezeski * filter and decrement the filter count. However, not all 226609aee612SRyan Zezeski * controllers count this implicit filter against the total 226709aee612SRyan Zezeski * MAC filter count. So here we are making sure it is either 226809aee612SRyan Zezeski * one or zero. If it is one, then we know it is for the 226909aee612SRyan Zezeski * implicit filter and we should decrement since we just 227009aee612SRyan Zezeski * removed the filter above. If it is zero then we know the 227109aee612SRyan Zezeski * controller that does not count the implicit filter, and it 227209aee612SRyan Zezeski * was enough to just remove it; we leave the count alone. 227309aee612SRyan Zezeski * But if it is neither, then we have never seen a controller 227409aee612SRyan Zezeski * like this before and we should fail to attach. 227509aee612SRyan Zezeski * 227609aee612SRyan Zezeski * It is unfortunate that this code must exist but the 227709aee612SRyan Zezeski * behavior of this implicit L2 filter and its corresponding 227809aee612SRyan Zezeski * count were dicovered through empirical testing. The 227909aee612SRyan Zezeski * programming manuals hint at this filter but do not 228009aee612SRyan Zezeski * explicitly call out the exact behavior. 228109aee612SRyan Zezeski */ 228209aee612SRyan Zezeski if (i40e->i40e_resources.ifr_nmacfilt_used == 1) { 228309aee612SRyan Zezeski i40e->i40e_resources.ifr_nmacfilt_used--; 228409aee612SRyan Zezeski } else { 228509aee612SRyan Zezeski if (i40e->i40e_resources.ifr_nmacfilt_used != 0) { 228635c41becSRyan Zezeski i40e_error(i40e, "Unexpected L2 filter count: %u" 228709aee612SRyan Zezeski " (expected 0)", 228809aee612SRyan Zezeski i40e->i40e_resources.ifr_nmacfilt_used); 228935c41becSRyan Zezeski return (B_FALSE); 229009aee612SRyan Zezeski } 229109aee612SRyan Zezeski } 229209aee612SRyan Zezeski 229309aee612SRyan Zezeski return (B_TRUE); 229409aee612SRyan Zezeski } 229509aee612SRyan Zezeski 229609aee612SRyan Zezeski static boolean_t 229709aee612SRyan Zezeski i40e_config_rss_key_x722(i40e_t *i40e, i40e_hw_t *hw) 229809aee612SRyan Zezeski { 229909aee612SRyan Zezeski for (uint_t i = 0; i < i40e->i40e_num_rx_groups; i++) { 230009aee612SRyan Zezeski uint32_t seed[I40E_PFQF_HKEY_MAX_INDEX + 1]; 2301b9d34b9dSRobert Mustacchi struct i40e_aqc_get_set_rss_key_data key; 230209aee612SRyan Zezeski const char *u8seed; 2303b9d34b9dSRobert Mustacchi enum i40e_status_code status; 230409aee612SRyan Zezeski uint16_t vsi_number = i40e->i40e_vsis[i].iv_number; 230509aee612SRyan Zezeski 230609aee612SRyan Zezeski (void) random_get_pseudo_bytes((uint8_t *)seed, sizeof (seed)); 230709aee612SRyan Zezeski u8seed = (char *)seed; 2308b9d34b9dSRobert Mustacchi 2309b9d34b9dSRobert Mustacchi CTASSERT(sizeof (key) >= (sizeof (key.standard_rss_key) + 2310b9d34b9dSRobert Mustacchi sizeof (key.extended_hash_key))); 2311b9d34b9dSRobert Mustacchi 2312b9d34b9dSRobert Mustacchi bcopy(u8seed, key.standard_rss_key, 2313b9d34b9dSRobert Mustacchi sizeof (key.standard_rss_key)); 2314b9d34b9dSRobert Mustacchi bcopy(&u8seed[sizeof (key.standard_rss_key)], 2315b9d34b9dSRobert Mustacchi key.extended_hash_key, sizeof (key.extended_hash_key)); 2316b9d34b9dSRobert Mustacchi 231709aee612SRyan Zezeski ASSERT3U(vsi_number, !=, 0); 231809aee612SRyan Zezeski status = i40e_aq_set_rss_key(hw, vsi_number, &key); 231909aee612SRyan Zezeski 2320b9d34b9dSRobert Mustacchi if (status != I40E_SUCCESS) { 232109aee612SRyan Zezeski i40e_error(i40e, "failed to set RSS key for VSI %u: %d", 232209aee612SRyan Zezeski vsi_number, status); 2323b9d34b9dSRobert Mustacchi return (B_FALSE); 2324b9d34b9dSRobert Mustacchi } 232509aee612SRyan Zezeski } 232609aee612SRyan Zezeski 232709aee612SRyan Zezeski return (B_TRUE); 232809aee612SRyan Zezeski } 232909aee612SRyan Zezeski 233009aee612SRyan Zezeski /* 233109aee612SRyan Zezeski * Configure the RSS key. For the X710 controller family, this is set on a 233209aee612SRyan Zezeski * per-PF basis via registers. For the X722, this is done on a per-VSI basis 233309aee612SRyan Zezeski * through the admin queue. 233409aee612SRyan Zezeski */ 233509aee612SRyan Zezeski static boolean_t 233609aee612SRyan Zezeski i40e_config_rss_key(i40e_t *i40e, i40e_hw_t *hw) 233709aee612SRyan Zezeski { 233809aee612SRyan Zezeski if (i40e_is_x722(i40e)) { 233909aee612SRyan Zezeski if (!i40e_config_rss_key_x722(i40e, hw)) 234009aee612SRyan Zezeski return (B_FALSE); 2341b9d34b9dSRobert Mustacchi } else { 234209aee612SRyan Zezeski uint32_t seed[I40E_PFQF_HKEY_MAX_INDEX + 1]; 234309aee612SRyan Zezeski 234409aee612SRyan Zezeski (void) random_get_pseudo_bytes((uint8_t *)seed, sizeof (seed)); 234509aee612SRyan Zezeski for (uint_t i = 0; i <= I40E_PFQF_HKEY_MAX_INDEX; i++) 2346b9d34b9dSRobert Mustacchi i40e_write_rx_ctl(hw, I40E_PFQF_HKEY(i), seed[i]); 2347b9d34b9dSRobert Mustacchi } 2348b9d34b9dSRobert Mustacchi 2349b9d34b9dSRobert Mustacchi return (B_TRUE); 2350b9d34b9dSRobert Mustacchi } 2351b9d34b9dSRobert Mustacchi 2352b9d34b9dSRobert Mustacchi /* 2353b9d34b9dSRobert Mustacchi * Populate the LUT. The size of each entry in the LUT depends on the controller 2354b9d34b9dSRobert Mustacchi * family, with the X722 using a known 7-bit width. On the X710 controller, this 2355b9d34b9dSRobert Mustacchi * is programmed through its control registers where as on the X722 this is 2356b9d34b9dSRobert Mustacchi * configured through the admin queue. Also of note, the X722 allows the LUT to 235709aee612SRyan Zezeski * be set on a per-PF or VSI basis. At this time we use the PF setting. If we 235809aee612SRyan Zezeski * decide to use the per-VSI LUT in the future, then we will need to modify the 235909aee612SRyan Zezeski * i40e_add_vsi() function to set the RSS LUT bits in the queueing section. 2360b9d34b9dSRobert Mustacchi * 2361b9d34b9dSRobert Mustacchi * We populate the LUT in a round robin fashion with the rx queue indices from 0 236209aee612SRyan Zezeski * to i40e_num_trqpairs_per_vsi - 1. 2363b9d34b9dSRobert Mustacchi */ 2364b9d34b9dSRobert Mustacchi static boolean_t 2365b9d34b9dSRobert Mustacchi i40e_config_rss_hlut(i40e_t *i40e, i40e_hw_t *hw) 2366b9d34b9dSRobert Mustacchi { 2367b9d34b9dSRobert Mustacchi uint32_t *hlut; 2368b9d34b9dSRobert Mustacchi uint8_t lut_mask; 2369b9d34b9dSRobert Mustacchi uint_t i; 2370b9d34b9dSRobert Mustacchi boolean_t ret = B_FALSE; 2371b9d34b9dSRobert Mustacchi 2372b9d34b9dSRobert Mustacchi /* 2373b9d34b9dSRobert Mustacchi * We always configure the PF with a table size of 512 bytes in 2374b9d34b9dSRobert Mustacchi * i40e_chip_start(). 2375b9d34b9dSRobert Mustacchi */ 2376b9d34b9dSRobert Mustacchi hlut = kmem_alloc(I40E_HLUT_TABLE_SIZE, KM_NOSLEEP); 2377b9d34b9dSRobert Mustacchi if (hlut == NULL) { 2378b9d34b9dSRobert Mustacchi i40e_error(i40e, "i40e_config_rss() buffer allocation failed"); 2379b9d34b9dSRobert Mustacchi return (B_FALSE); 2380b9d34b9dSRobert Mustacchi } 2381b9d34b9dSRobert Mustacchi 2382b9d34b9dSRobert Mustacchi /* 2383b9d34b9dSRobert Mustacchi * The width of the X722 is apparently defined to be 7 bits, regardless 2384b9d34b9dSRobert Mustacchi * of the capability. 2385b9d34b9dSRobert Mustacchi */ 2386b9d34b9dSRobert Mustacchi if (i40e_is_x722(i40e)) { 2387b9d34b9dSRobert Mustacchi lut_mask = (1 << 7) - 1; 2388b9d34b9dSRobert Mustacchi } else { 2389b9d34b9dSRobert Mustacchi lut_mask = (1 << hw->func_caps.rss_table_entry_width) - 1; 2390b9d34b9dSRobert Mustacchi } 2391b9d34b9dSRobert Mustacchi 239209aee612SRyan Zezeski for (i = 0; i < I40E_HLUT_TABLE_SIZE; i++) { 239309aee612SRyan Zezeski ((uint8_t *)hlut)[i] = 239409aee612SRyan Zezeski (i % i40e->i40e_num_trqpairs_per_vsi) & lut_mask; 239509aee612SRyan Zezeski } 2396b9d34b9dSRobert Mustacchi 2397b9d34b9dSRobert Mustacchi if (i40e_is_x722(i40e)) { 2398b9d34b9dSRobert Mustacchi enum i40e_status_code status; 239909aee612SRyan Zezeski 240009aee612SRyan Zezeski status = i40e_aq_set_rss_lut(hw, 0, B_TRUE, (uint8_t *)hlut, 240109aee612SRyan Zezeski I40E_HLUT_TABLE_SIZE); 240209aee612SRyan Zezeski 2403b9d34b9dSRobert Mustacchi if (status != I40E_SUCCESS) { 240409aee612SRyan Zezeski i40e_error(i40e, "failed to set RSS LUT %d: %d", 240509aee612SRyan Zezeski status, hw->aq.asq_last_status); 2406b9d34b9dSRobert Mustacchi goto out; 2407b9d34b9dSRobert Mustacchi } 2408b9d34b9dSRobert Mustacchi } else { 2409b9d34b9dSRobert Mustacchi for (i = 0; i < I40E_HLUT_TABLE_SIZE >> 2; i++) { 2410b9d34b9dSRobert Mustacchi I40E_WRITE_REG(hw, I40E_PFQF_HLUT(i), hlut[i]); 2411b9d34b9dSRobert Mustacchi } 2412b9d34b9dSRobert Mustacchi } 2413b9d34b9dSRobert Mustacchi ret = B_TRUE; 2414b9d34b9dSRobert Mustacchi out: 2415b9d34b9dSRobert Mustacchi kmem_free(hlut, I40E_HLUT_TABLE_SIZE); 2416b9d34b9dSRobert Mustacchi return (ret); 2417b9d34b9dSRobert Mustacchi } 2418b9d34b9dSRobert Mustacchi 2419396505afSPaul Winder /* 2420396505afSPaul Winder * Set up RSS. 2421508a0e8cSRob Johnston * 1. Seed the hash key. 2422396505afSPaul Winder * 2. Enable PCTYPEs for the hash filter. 2423396505afSPaul Winder * 3. Populate the LUT. 2424396505afSPaul Winder */ 2425396505afSPaul Winder static boolean_t 2426396505afSPaul Winder i40e_config_rss(i40e_t *i40e, i40e_hw_t *hw) 2427396505afSPaul Winder { 2428396505afSPaul Winder uint64_t hena; 2429396505afSPaul Winder 2430396505afSPaul Winder /* 2431396505afSPaul Winder * 1. Seed the hash key 2432396505afSPaul Winder */ 2433b9d34b9dSRobert Mustacchi if (!i40e_config_rss_key(i40e, hw)) 2434b9d34b9dSRobert Mustacchi return (B_FALSE); 2435396505afSPaul Winder 2436396505afSPaul Winder /* 2437396505afSPaul Winder * 2. Configure PCTYPES 2438396505afSPaul Winder */ 2439396505afSPaul Winder hena = (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_OTHER) | 2440396505afSPaul Winder (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_TCP) | 2441b9d34b9dSRobert Mustacchi (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_SCTP) | 2442396505afSPaul Winder (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_UDP) | 2443396505afSPaul Winder (1ULL << I40E_FILTER_PCTYPE_FRAG_IPV4) | 2444396505afSPaul Winder (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_OTHER) | 2445396505afSPaul Winder (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_TCP) | 2446b9d34b9dSRobert Mustacchi (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_SCTP) | 2447396505afSPaul Winder (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_UDP) | 2448396505afSPaul Winder (1ULL << I40E_FILTER_PCTYPE_FRAG_IPV6) | 2449396505afSPaul Winder (1ULL << I40E_FILTER_PCTYPE_L2_PAYLOAD); 2450396505afSPaul Winder 2451b9d34b9dSRobert Mustacchi /* 2452b9d34b9dSRobert Mustacchi * Add additional types supported by the X722 controller. 2453b9d34b9dSRobert Mustacchi */ 2454b9d34b9dSRobert Mustacchi if (i40e_is_x722(i40e)) { 2455b9d34b9dSRobert Mustacchi hena |= (1ULL << I40E_FILTER_PCTYPE_NONF_UNICAST_IPV4_UDP) | 2456b9d34b9dSRobert Mustacchi (1ULL << I40E_FILTER_PCTYPE_NONF_MULTICAST_IPV4_UDP) | 2457b9d34b9dSRobert Mustacchi (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_TCP_SYN_NO_ACK) | 2458b9d34b9dSRobert Mustacchi (1ULL << I40E_FILTER_PCTYPE_NONF_UNICAST_IPV6_UDP) | 2459b9d34b9dSRobert Mustacchi (1ULL << I40E_FILTER_PCTYPE_NONF_MULTICAST_IPV6_UDP) | 2460b9d34b9dSRobert Mustacchi (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_TCP_SYN_NO_ACK); 2461b9d34b9dSRobert Mustacchi } 2462b9d34b9dSRobert Mustacchi 2463396505afSPaul Winder i40e_write_rx_ctl(hw, I40E_PFQF_HENA(0), (uint32_t)hena); 2464396505afSPaul Winder i40e_write_rx_ctl(hw, I40E_PFQF_HENA(1), (uint32_t)(hena >> 32)); 2465396505afSPaul Winder 2466396505afSPaul Winder /* 2467396505afSPaul Winder * 3. Populate LUT 2468396505afSPaul Winder */ 2469b9d34b9dSRobert Mustacchi return (i40e_config_rss_hlut(i40e, hw)); 2470396505afSPaul Winder } 2471396505afSPaul Winder 24729d26e4fcSRobert Mustacchi /* 24739d26e4fcSRobert Mustacchi * Wrapper to kick the chipset on. 24749d26e4fcSRobert Mustacchi */ 24759d26e4fcSRobert Mustacchi static boolean_t 24769d26e4fcSRobert Mustacchi i40e_chip_start(i40e_t *i40e) 24779d26e4fcSRobert Mustacchi { 24789d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 24799d26e4fcSRobert Mustacchi struct i40e_filter_control_settings filter; 24809d26e4fcSRobert Mustacchi int rc; 24812f79f42cSRyan Zezeski uint8_t err; 24829d26e4fcSRobert Mustacchi 24839d26e4fcSRobert Mustacchi if (((hw->aq.fw_maj_ver == 4) && (hw->aq.fw_min_ver < 33)) || 24849d26e4fcSRobert Mustacchi (hw->aq.fw_maj_ver < 4)) { 24859d26e4fcSRobert Mustacchi i40e_msec_delay(75); 24869d26e4fcSRobert Mustacchi if (i40e_aq_set_link_restart_an(hw, TRUE, NULL) != 24879d26e4fcSRobert Mustacchi I40E_SUCCESS) { 24889d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to restart link: admin queue " 24899d26e4fcSRobert Mustacchi "error: %d", hw->aq.asq_last_status); 24909d26e4fcSRobert Mustacchi return (B_FALSE); 24919d26e4fcSRobert Mustacchi } 24929d26e4fcSRobert Mustacchi } 24939d26e4fcSRobert Mustacchi 24949d26e4fcSRobert Mustacchi /* Determine hardware state */ 24959d26e4fcSRobert Mustacchi i40e_get_hw_state(i40e, hw); 24969d26e4fcSRobert Mustacchi 24972f79f42cSRyan Zezeski /* For now, we always disable Ethernet Flow Control. */ 24982f79f42cSRyan Zezeski hw->fc.requested_mode = I40E_FC_NONE; 24992f79f42cSRyan Zezeski rc = i40e_set_fc(hw, &err, B_TRUE); 25002f79f42cSRyan Zezeski if (rc != I40E_SUCCESS) { 25012f79f42cSRyan Zezeski i40e_error(i40e, "Setting flow control failed, returned %d" 25022f79f42cSRyan Zezeski " with error: 0x%x", rc, err); 25032f79f42cSRyan Zezeski return (B_FALSE); 25042f79f42cSRyan Zezeski } 25052f79f42cSRyan Zezeski 25069d26e4fcSRobert Mustacchi /* Initialize mac addresses. */ 25079d26e4fcSRobert Mustacchi i40e_init_macaddrs(i40e, hw); 25089d26e4fcSRobert Mustacchi 25099d26e4fcSRobert Mustacchi /* 2510b9d34b9dSRobert Mustacchi * Set up the filter control. If the hash lut size is changed from 2511b9d34b9dSRobert Mustacchi * I40E_HASH_LUT_SIZE_512 then I40E_HLUT_TABLE_SIZE and 2512b9d34b9dSRobert Mustacchi * i40e_config_rss_hlut() will need to be updated. 25139d26e4fcSRobert Mustacchi */ 25149d26e4fcSRobert Mustacchi bzero(&filter, sizeof (filter)); 25159d26e4fcSRobert Mustacchi filter.enable_ethtype = TRUE; 25169d26e4fcSRobert Mustacchi filter.enable_macvlan = TRUE; 2517396505afSPaul Winder filter.hash_lut_size = I40E_HASH_LUT_SIZE_512; 25189d26e4fcSRobert Mustacchi 25199d26e4fcSRobert Mustacchi rc = i40e_set_filter_control(hw, &filter); 25209d26e4fcSRobert Mustacchi if (rc != I40E_SUCCESS) { 25219d26e4fcSRobert Mustacchi i40e_error(i40e, "i40e_set_filter_control() returned %d", rc); 25229d26e4fcSRobert Mustacchi return (B_FALSE); 25239d26e4fcSRobert Mustacchi } 25249d26e4fcSRobert Mustacchi 25259d26e4fcSRobert Mustacchi i40e_intr_chip_init(i40e); 25269d26e4fcSRobert Mustacchi 252709aee612SRyan Zezeski rc = i40e_get_mac_seid(i40e); 252809aee612SRyan Zezeski if (rc == -1) { 252909aee612SRyan Zezeski i40e_error(i40e, "failed to obtain MAC Uplink SEID"); 25309d26e4fcSRobert Mustacchi return (B_FALSE); 253109aee612SRyan Zezeski } 253209aee612SRyan Zezeski i40e->i40e_mac_seid = (uint16_t)rc; 253309aee612SRyan Zezeski 253409aee612SRyan Zezeski /* 253509aee612SRyan Zezeski * Create a VEB in order to support multiple VSIs. Each VSI 253609aee612SRyan Zezeski * functions as a MAC group. This call sets the PF's MAC as 253709aee612SRyan Zezeski * the uplink port and the PF's default VSI as the default 253809aee612SRyan Zezeski * downlink port. 253909aee612SRyan Zezeski */ 254009aee612SRyan Zezeski rc = i40e_aq_add_veb(hw, i40e->i40e_mac_seid, I40E_DEF_VSI_SEID(i40e), 254109aee612SRyan Zezeski 0x1, B_TRUE, &i40e->i40e_veb_seid, B_FALSE, NULL); 254209aee612SRyan Zezeski if (rc != I40E_SUCCESS) { 254309aee612SRyan Zezeski i40e_error(i40e, "i40e_aq_add_veb() failed %d: %d", rc, 254409aee612SRyan Zezeski hw->aq.asq_last_status); 254509aee612SRyan Zezeski return (B_FALSE); 254609aee612SRyan Zezeski } 254709aee612SRyan Zezeski 254809aee612SRyan Zezeski if (!i40e_config_def_vsi(i40e, hw)) 254909aee612SRyan Zezeski return (B_FALSE); 255009aee612SRyan Zezeski 255109aee612SRyan Zezeski for (uint_t i = 1; i < i40e->i40e_num_rx_groups; i++) { 255209aee612SRyan Zezeski if (!i40e_add_vsi(i40e, hw, i)) 255309aee612SRyan Zezeski return (B_FALSE); 255409aee612SRyan Zezeski } 25559d26e4fcSRobert Mustacchi 2556396505afSPaul Winder if (!i40e_config_rss(i40e, hw)) 2557396505afSPaul Winder return (B_FALSE); 2558396505afSPaul Winder 25599d26e4fcSRobert Mustacchi i40e_flush(hw); 25609d26e4fcSRobert Mustacchi 25619d26e4fcSRobert Mustacchi return (B_TRUE); 25629d26e4fcSRobert Mustacchi } 25639d26e4fcSRobert Mustacchi 25649d26e4fcSRobert Mustacchi /* 25659d26e4fcSRobert Mustacchi * Take care of tearing down the rx ring. See 8.3.3.1.2 for more information. 25669d26e4fcSRobert Mustacchi */ 25679d26e4fcSRobert Mustacchi static void 25689d26e4fcSRobert Mustacchi i40e_shutdown_rx_rings(i40e_t *i40e) 25699d26e4fcSRobert Mustacchi { 25709d26e4fcSRobert Mustacchi int i; 25719d26e4fcSRobert Mustacchi uint32_t reg; 25729d26e4fcSRobert Mustacchi 25739d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 25749d26e4fcSRobert Mustacchi 25759d26e4fcSRobert Mustacchi /* 25769d26e4fcSRobert Mustacchi * Step 1. The interrupt linked list (see i40e_intr.c for more 25779d26e4fcSRobert Mustacchi * information) should have already been cleared before calling this 25789d26e4fcSRobert Mustacchi * function. 25799d26e4fcSRobert Mustacchi */ 25809d26e4fcSRobert Mustacchi #ifdef DEBUG 25819d26e4fcSRobert Mustacchi if (i40e->i40e_intr_type == DDI_INTR_TYPE_MSIX) { 25829d26e4fcSRobert Mustacchi for (i = 1; i < i40e->i40e_intr_count; i++) { 25839d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_PFINT_LNKLSTN(i - 1)); 25849d26e4fcSRobert Mustacchi VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL); 25859d26e4fcSRobert Mustacchi } 25869d26e4fcSRobert Mustacchi } else { 25879d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_PFINT_LNKLST0); 25889d26e4fcSRobert Mustacchi VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL); 25899d26e4fcSRobert Mustacchi } 25909d26e4fcSRobert Mustacchi 25919d26e4fcSRobert Mustacchi #endif /* DEBUG */ 25929d26e4fcSRobert Mustacchi 25939d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 25949d26e4fcSRobert Mustacchi /* 25959d26e4fcSRobert Mustacchi * Step 1. Request the queue by clearing QENA_REQ. It may not be 25969d26e4fcSRobert Mustacchi * set due to unwinding from failures and a partially enabled 25979d26e4fcSRobert Mustacchi * ring set. 25989d26e4fcSRobert Mustacchi */ 25999d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_QRX_ENA(i)); 26009d26e4fcSRobert Mustacchi if (!(reg & I40E_QRX_ENA_QENA_REQ_MASK)) 26019d26e4fcSRobert Mustacchi continue; 26029d26e4fcSRobert Mustacchi VERIFY((reg & I40E_QRX_ENA_QENA_REQ_MASK) == 26039d26e4fcSRobert Mustacchi I40E_QRX_ENA_QENA_REQ_MASK); 26049d26e4fcSRobert Mustacchi reg &= ~I40E_QRX_ENA_QENA_REQ_MASK; 26059d26e4fcSRobert Mustacchi I40E_WRITE_REG(hw, I40E_QRX_ENA(i), reg); 26069d26e4fcSRobert Mustacchi } 26079d26e4fcSRobert Mustacchi 26089d26e4fcSRobert Mustacchi /* 26099d26e4fcSRobert Mustacchi * Step 2. Wait for the disable to take, by having QENA_STAT in the FPM 26109d26e4fcSRobert Mustacchi * be cleared. Note that we could still receive data in the queue during 26119d26e4fcSRobert Mustacchi * this time. We don't actually wait for this now and instead defer this 26129d26e4fcSRobert Mustacchi * to i40e_shutdown_rings_wait(), after we've interleaved disabling the 26139d26e4fcSRobert Mustacchi * TX queues as well. 26149d26e4fcSRobert Mustacchi */ 26159d26e4fcSRobert Mustacchi } 26169d26e4fcSRobert Mustacchi 26179d26e4fcSRobert Mustacchi static void 26189d26e4fcSRobert Mustacchi i40e_shutdown_tx_rings(i40e_t *i40e) 26199d26e4fcSRobert Mustacchi { 26209d26e4fcSRobert Mustacchi int i; 26219d26e4fcSRobert Mustacchi uint32_t reg; 26229d26e4fcSRobert Mustacchi 26239d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 26249d26e4fcSRobert Mustacchi 26259d26e4fcSRobert Mustacchi /* 26269d26e4fcSRobert Mustacchi * Step 1. The interrupt linked list should already have been cleared. 26279d26e4fcSRobert Mustacchi */ 26289d26e4fcSRobert Mustacchi #ifdef DEBUG 26299d26e4fcSRobert Mustacchi if (i40e->i40e_intr_type == DDI_INTR_TYPE_MSIX) { 26309d26e4fcSRobert Mustacchi for (i = 1; i < i40e->i40e_intr_count; i++) { 26319d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_PFINT_LNKLSTN(i - 1)); 26329d26e4fcSRobert Mustacchi VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL); 26339d26e4fcSRobert Mustacchi } 26349d26e4fcSRobert Mustacchi } else { 26359d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_PFINT_LNKLST0); 26369d26e4fcSRobert Mustacchi VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL); 26379d26e4fcSRobert Mustacchi 26389d26e4fcSRobert Mustacchi } 26399d26e4fcSRobert Mustacchi #endif /* DEBUG */ 26409d26e4fcSRobert Mustacchi 26419d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 26429d26e4fcSRobert Mustacchi /* 26439d26e4fcSRobert Mustacchi * Step 2. Set the SET_QDIS flag for every queue. 26449d26e4fcSRobert Mustacchi */ 26459d26e4fcSRobert Mustacchi i40e_pre_tx_queue_cfg(hw, i, B_FALSE); 26469d26e4fcSRobert Mustacchi } 26479d26e4fcSRobert Mustacchi 26489d26e4fcSRobert Mustacchi /* 26499d26e4fcSRobert Mustacchi * Step 3. Wait at least 400 usec (can be done once for all queues). 26509d26e4fcSRobert Mustacchi */ 26519d26e4fcSRobert Mustacchi drv_usecwait(500); 26529d26e4fcSRobert Mustacchi 26539d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 26549d26e4fcSRobert Mustacchi /* 26559d26e4fcSRobert Mustacchi * Step 4. Clear the QENA_REQ flag which tells hardware to 26569d26e4fcSRobert Mustacchi * quiesce. If QENA_REQ is not already set then that means that 26579d26e4fcSRobert Mustacchi * we likely already tried to disable this queue. 26589d26e4fcSRobert Mustacchi */ 26599d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_QTX_ENA(i)); 26609d26e4fcSRobert Mustacchi if (!(reg & I40E_QTX_ENA_QENA_REQ_MASK)) 26619d26e4fcSRobert Mustacchi continue; 26629d26e4fcSRobert Mustacchi reg &= ~I40E_QTX_ENA_QENA_REQ_MASK; 26639d26e4fcSRobert Mustacchi I40E_WRITE_REG(hw, I40E_QTX_ENA(i), reg); 26649d26e4fcSRobert Mustacchi } 26659d26e4fcSRobert Mustacchi 26669d26e4fcSRobert Mustacchi /* 26679d26e4fcSRobert Mustacchi * Step 5. Wait for all drains to finish. This will be done by the 26689d26e4fcSRobert Mustacchi * hardware removing the QENA_STAT flag from the queue. Rather than 26699d26e4fcSRobert Mustacchi * waiting here, we interleave it with all the others in 26709d26e4fcSRobert Mustacchi * i40e_shutdown_rings_wait(). 26719d26e4fcSRobert Mustacchi */ 26729d26e4fcSRobert Mustacchi } 26739d26e4fcSRobert Mustacchi 26749d26e4fcSRobert Mustacchi /* 26759d26e4fcSRobert Mustacchi * Wait for all the rings to be shut down. e.g. Steps 2 and 5 from the above 26769d26e4fcSRobert Mustacchi * functions. 26779d26e4fcSRobert Mustacchi */ 26789d26e4fcSRobert Mustacchi static boolean_t 26799d26e4fcSRobert Mustacchi i40e_shutdown_rings_wait(i40e_t *i40e) 26809d26e4fcSRobert Mustacchi { 26819d26e4fcSRobert Mustacchi int i, try; 26829d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 26839d26e4fcSRobert Mustacchi 26849d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 26859d26e4fcSRobert Mustacchi uint32_t reg; 26869d26e4fcSRobert Mustacchi 26879d26e4fcSRobert Mustacchi for (try = 0; try < I40E_RING_WAIT_NTRIES; try++) { 26889d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_QRX_ENA(i)); 26899d26e4fcSRobert Mustacchi if ((reg & I40E_QRX_ENA_QENA_STAT_MASK) == 0) 26909d26e4fcSRobert Mustacchi break; 26919d26e4fcSRobert Mustacchi i40e_msec_delay(I40E_RING_WAIT_PAUSE); 26929d26e4fcSRobert Mustacchi } 26939d26e4fcSRobert Mustacchi 26949d26e4fcSRobert Mustacchi if ((reg & I40E_QRX_ENA_QENA_STAT_MASK) != 0) { 26959d26e4fcSRobert Mustacchi i40e_error(i40e, "timed out disabling rx queue %d", 26969d26e4fcSRobert Mustacchi i); 26979d26e4fcSRobert Mustacchi return (B_FALSE); 26989d26e4fcSRobert Mustacchi } 26999d26e4fcSRobert Mustacchi 27009d26e4fcSRobert Mustacchi for (try = 0; try < I40E_RING_WAIT_NTRIES; try++) { 27019d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_QTX_ENA(i)); 27029d26e4fcSRobert Mustacchi if ((reg & I40E_QTX_ENA_QENA_STAT_MASK) == 0) 27039d26e4fcSRobert Mustacchi break; 27049d26e4fcSRobert Mustacchi i40e_msec_delay(I40E_RING_WAIT_PAUSE); 27059d26e4fcSRobert Mustacchi } 27069d26e4fcSRobert Mustacchi 27079d26e4fcSRobert Mustacchi if ((reg & I40E_QTX_ENA_QENA_STAT_MASK) != 0) { 27089d26e4fcSRobert Mustacchi i40e_error(i40e, "timed out disabling tx queue %d", 27099d26e4fcSRobert Mustacchi i); 27109d26e4fcSRobert Mustacchi return (B_FALSE); 27119d26e4fcSRobert Mustacchi } 27129d26e4fcSRobert Mustacchi } 27139d26e4fcSRobert Mustacchi 27149d26e4fcSRobert Mustacchi return (B_TRUE); 27159d26e4fcSRobert Mustacchi } 27169d26e4fcSRobert Mustacchi 27179d26e4fcSRobert Mustacchi static boolean_t 27189d26e4fcSRobert Mustacchi i40e_shutdown_rings(i40e_t *i40e) 27199d26e4fcSRobert Mustacchi { 27209d26e4fcSRobert Mustacchi i40e_shutdown_rx_rings(i40e); 27219d26e4fcSRobert Mustacchi i40e_shutdown_tx_rings(i40e); 27229d26e4fcSRobert Mustacchi return (i40e_shutdown_rings_wait(i40e)); 27239d26e4fcSRobert Mustacchi } 27249d26e4fcSRobert Mustacchi 27259d26e4fcSRobert Mustacchi static void 27269d26e4fcSRobert Mustacchi i40e_setup_rx_descs(i40e_trqpair_t *itrq) 27279d26e4fcSRobert Mustacchi { 27289d26e4fcSRobert Mustacchi int i; 27299d26e4fcSRobert Mustacchi i40e_rx_data_t *rxd = itrq->itrq_rxdata; 27309d26e4fcSRobert Mustacchi 27319d26e4fcSRobert Mustacchi for (i = 0; i < rxd->rxd_ring_size; i++) { 27329d26e4fcSRobert Mustacchi i40e_rx_control_block_t *rcb; 27339d26e4fcSRobert Mustacchi i40e_rx_desc_t *rdesc; 27349d26e4fcSRobert Mustacchi 27359d26e4fcSRobert Mustacchi rcb = rxd->rxd_work_list[i]; 27369d26e4fcSRobert Mustacchi rdesc = &rxd->rxd_desc_ring[i]; 27379d26e4fcSRobert Mustacchi 27389d26e4fcSRobert Mustacchi rdesc->read.pkt_addr = 27399d26e4fcSRobert Mustacchi CPU_TO_LE64((uintptr_t)rcb->rcb_dma.dmab_dma_address); 27409d26e4fcSRobert Mustacchi rdesc->read.hdr_addr = 0; 27419d26e4fcSRobert Mustacchi } 27429d26e4fcSRobert Mustacchi } 27439d26e4fcSRobert Mustacchi 27449d26e4fcSRobert Mustacchi static boolean_t 27459d26e4fcSRobert Mustacchi i40e_setup_rx_hmc(i40e_trqpair_t *itrq) 27469d26e4fcSRobert Mustacchi { 27479d26e4fcSRobert Mustacchi i40e_rx_data_t *rxd = itrq->itrq_rxdata; 27489d26e4fcSRobert Mustacchi i40e_t *i40e = itrq->itrq_i40e; 27499d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 27509d26e4fcSRobert Mustacchi 27519d26e4fcSRobert Mustacchi struct i40e_hmc_obj_rxq rctx; 27529d26e4fcSRobert Mustacchi int err; 27539d26e4fcSRobert Mustacchi 27549d26e4fcSRobert Mustacchi bzero(&rctx, sizeof (struct i40e_hmc_obj_rxq)); 27559d26e4fcSRobert Mustacchi rctx.base = rxd->rxd_desc_area.dmab_dma_address / 27569d26e4fcSRobert Mustacchi I40E_HMC_RX_CTX_UNIT; 27579d26e4fcSRobert Mustacchi rctx.qlen = rxd->rxd_ring_size; 27589d26e4fcSRobert Mustacchi VERIFY(i40e->i40e_rx_buf_size >= I40E_HMC_RX_DBUFF_MIN); 27599d26e4fcSRobert Mustacchi VERIFY(i40e->i40e_rx_buf_size <= I40E_HMC_RX_DBUFF_MAX); 27609d26e4fcSRobert Mustacchi rctx.dbuff = i40e->i40e_rx_buf_size >> I40E_RXQ_CTX_DBUFF_SHIFT; 27619d26e4fcSRobert Mustacchi rctx.hbuff = 0 >> I40E_RXQ_CTX_HBUFF_SHIFT; 27629d26e4fcSRobert Mustacchi rctx.dtype = I40E_HMC_RX_DTYPE_NOSPLIT; 27639d26e4fcSRobert Mustacchi rctx.dsize = I40E_HMC_RX_DSIZE_32BYTE; 27649d26e4fcSRobert Mustacchi rctx.crcstrip = I40E_HMC_RX_CRCSTRIP_ENABLE; 27659d26e4fcSRobert Mustacchi rctx.fc_ena = I40E_HMC_RX_FC_DISABLE; 27669d26e4fcSRobert Mustacchi rctx.l2tsel = I40E_HMC_RX_L2TAGORDER; 27679d26e4fcSRobert Mustacchi rctx.hsplit_0 = I40E_HMC_RX_HDRSPLIT_DISABLE; 27689d26e4fcSRobert Mustacchi rctx.hsplit_1 = I40E_HMC_RX_HDRSPLIT_DISABLE; 27699d26e4fcSRobert Mustacchi rctx.showiv = I40E_HMC_RX_INVLAN_DONTSTRIP; 27709d26e4fcSRobert Mustacchi rctx.rxmax = i40e->i40e_frame_max; 27719d26e4fcSRobert Mustacchi rctx.tphrdesc_ena = I40E_HMC_RX_TPH_DISABLE; 27729d26e4fcSRobert Mustacchi rctx.tphwdesc_ena = I40E_HMC_RX_TPH_DISABLE; 27739d26e4fcSRobert Mustacchi rctx.tphdata_ena = I40E_HMC_RX_TPH_DISABLE; 27749d26e4fcSRobert Mustacchi rctx.tphhead_ena = I40E_HMC_RX_TPH_DISABLE; 27759d26e4fcSRobert Mustacchi rctx.lrxqthresh = I40E_HMC_RX_LOWRXQ_NOINTR; 27769d26e4fcSRobert Mustacchi 27779d26e4fcSRobert Mustacchi /* 27789d26e4fcSRobert Mustacchi * This must be set to 0x1, see Table 8-12 in section 8.3.3.2.2. 27799d26e4fcSRobert Mustacchi */ 27809d26e4fcSRobert Mustacchi rctx.prefena = I40E_HMC_RX_PREFENA; 27819d26e4fcSRobert Mustacchi 27829d26e4fcSRobert Mustacchi err = i40e_clear_lan_rx_queue_context(hw, itrq->itrq_index); 27839d26e4fcSRobert Mustacchi if (err != I40E_SUCCESS) { 27849d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to clear rx queue %d context: %d", 27859d26e4fcSRobert Mustacchi itrq->itrq_index, err); 27869d26e4fcSRobert Mustacchi return (B_FALSE); 27879d26e4fcSRobert Mustacchi } 27889d26e4fcSRobert Mustacchi 27899d26e4fcSRobert Mustacchi err = i40e_set_lan_rx_queue_context(hw, itrq->itrq_index, &rctx); 27909d26e4fcSRobert Mustacchi if (err != I40E_SUCCESS) { 27919d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to set rx queue %d context: %d", 27929d26e4fcSRobert Mustacchi itrq->itrq_index, err); 27939d26e4fcSRobert Mustacchi return (B_FALSE); 27949d26e4fcSRobert Mustacchi } 27959d26e4fcSRobert Mustacchi 27969d26e4fcSRobert Mustacchi return (B_TRUE); 27979d26e4fcSRobert Mustacchi } 27989d26e4fcSRobert Mustacchi 27999d26e4fcSRobert Mustacchi /* 28009d26e4fcSRobert Mustacchi * Take care of setting up the descriptor rings and actually programming the 28019d26e4fcSRobert Mustacchi * device. See 8.3.3.1.1 for the full list of steps we need to do to enable the 28029d26e4fcSRobert Mustacchi * rx rings. 28039d26e4fcSRobert Mustacchi */ 28049d26e4fcSRobert Mustacchi static boolean_t 28059d26e4fcSRobert Mustacchi i40e_setup_rx_rings(i40e_t *i40e) 28069d26e4fcSRobert Mustacchi { 28079d26e4fcSRobert Mustacchi int i; 28089d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 28099d26e4fcSRobert Mustacchi 28109d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 28119d26e4fcSRobert Mustacchi i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[i]; 28129d26e4fcSRobert Mustacchi i40e_rx_data_t *rxd = itrq->itrq_rxdata; 28139d26e4fcSRobert Mustacchi uint32_t reg; 28149d26e4fcSRobert Mustacchi 28159d26e4fcSRobert Mustacchi /* 28169d26e4fcSRobert Mustacchi * Step 1. Program all receive ring descriptors. 28179d26e4fcSRobert Mustacchi */ 28189d26e4fcSRobert Mustacchi i40e_setup_rx_descs(itrq); 28199d26e4fcSRobert Mustacchi 28209d26e4fcSRobert Mustacchi /* 28219d26e4fcSRobert Mustacchi * Step 2. Program the queue's FPM/HMC context. 28229d26e4fcSRobert Mustacchi */ 28239d26e4fcSRobert Mustacchi if (i40e_setup_rx_hmc(itrq) == B_FALSE) 28249d26e4fcSRobert Mustacchi return (B_FALSE); 28259d26e4fcSRobert Mustacchi 28269d26e4fcSRobert Mustacchi /* 28279d26e4fcSRobert Mustacchi * Step 3. Clear the queue's tail pointer and set it to the end 28289d26e4fcSRobert Mustacchi * of the space. 28299d26e4fcSRobert Mustacchi */ 28309d26e4fcSRobert Mustacchi I40E_WRITE_REG(hw, I40E_QRX_TAIL(i), 0); 28319d26e4fcSRobert Mustacchi I40E_WRITE_REG(hw, I40E_QRX_TAIL(i), rxd->rxd_ring_size - 1); 28329d26e4fcSRobert Mustacchi 28339d26e4fcSRobert Mustacchi /* 28349d26e4fcSRobert Mustacchi * Step 4. Enable the queue via the QENA_REQ. 28359d26e4fcSRobert Mustacchi */ 28369d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_QRX_ENA(i)); 28379d26e4fcSRobert Mustacchi VERIFY0(reg & (I40E_QRX_ENA_QENA_REQ_MASK | 28389d26e4fcSRobert Mustacchi I40E_QRX_ENA_QENA_STAT_MASK)); 28399d26e4fcSRobert Mustacchi reg |= I40E_QRX_ENA_QENA_REQ_MASK; 28409d26e4fcSRobert Mustacchi I40E_WRITE_REG(hw, I40E_QRX_ENA(i), reg); 28419d26e4fcSRobert Mustacchi } 28429d26e4fcSRobert Mustacchi 28439d26e4fcSRobert Mustacchi /* 28449d26e4fcSRobert Mustacchi * Note, we wait for every queue to be enabled before we start checking. 28459d26e4fcSRobert Mustacchi * This will hopefully cause most queues to be enabled at this point. 28469d26e4fcSRobert Mustacchi */ 28479d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 28489d26e4fcSRobert Mustacchi uint32_t j, reg; 28499d26e4fcSRobert Mustacchi 28509d26e4fcSRobert Mustacchi /* 28519d26e4fcSRobert Mustacchi * Step 5. Verify that QENA_STAT has been set. It's promised 28529d26e4fcSRobert Mustacchi * that this should occur within about 10 us, but like other 28539d26e4fcSRobert Mustacchi * systems, we give the card a bit more time. 28549d26e4fcSRobert Mustacchi */ 28559d26e4fcSRobert Mustacchi for (j = 0; j < I40E_RING_WAIT_NTRIES; j++) { 28569d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_QRX_ENA(i)); 28579d26e4fcSRobert Mustacchi 28589d26e4fcSRobert Mustacchi if (reg & I40E_QRX_ENA_QENA_STAT_MASK) 28599d26e4fcSRobert Mustacchi break; 28609d26e4fcSRobert Mustacchi i40e_msec_delay(I40E_RING_WAIT_PAUSE); 28619d26e4fcSRobert Mustacchi } 28629d26e4fcSRobert Mustacchi 28639d26e4fcSRobert Mustacchi if ((reg & I40E_QRX_ENA_QENA_STAT_MASK) == 0) { 28649d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to enable rx queue %d, timed " 28659d26e4fcSRobert Mustacchi "out.", i); 28669d26e4fcSRobert Mustacchi return (B_FALSE); 28679d26e4fcSRobert Mustacchi } 28689d26e4fcSRobert Mustacchi } 28699d26e4fcSRobert Mustacchi 28709d26e4fcSRobert Mustacchi return (B_TRUE); 28719d26e4fcSRobert Mustacchi } 28729d26e4fcSRobert Mustacchi 28739d26e4fcSRobert Mustacchi static boolean_t 28749d26e4fcSRobert Mustacchi i40e_setup_tx_hmc(i40e_trqpair_t *itrq) 28759d26e4fcSRobert Mustacchi { 28769d26e4fcSRobert Mustacchi i40e_t *i40e = itrq->itrq_i40e; 28779d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 28789d26e4fcSRobert Mustacchi 28799d26e4fcSRobert Mustacchi struct i40e_hmc_obj_txq tctx; 28809d26e4fcSRobert Mustacchi struct i40e_vsi_context context; 28819d26e4fcSRobert Mustacchi int err; 28829d26e4fcSRobert Mustacchi 28839d26e4fcSRobert Mustacchi bzero(&tctx, sizeof (struct i40e_hmc_obj_txq)); 28849d26e4fcSRobert Mustacchi tctx.new_context = I40E_HMC_TX_NEW_CONTEXT; 28859d26e4fcSRobert Mustacchi tctx.base = itrq->itrq_desc_area.dmab_dma_address / 28869d26e4fcSRobert Mustacchi I40E_HMC_TX_CTX_UNIT; 28879d26e4fcSRobert Mustacchi tctx.fc_ena = I40E_HMC_TX_FC_DISABLE; 28889d26e4fcSRobert Mustacchi tctx.timesync_ena = I40E_HMC_TX_TS_DISABLE; 28899d26e4fcSRobert Mustacchi tctx.fd_ena = I40E_HMC_TX_FD_DISABLE; 28909d26e4fcSRobert Mustacchi tctx.alt_vlan_ena = I40E_HMC_TX_ALT_VLAN_DISABLE; 28919d26e4fcSRobert Mustacchi tctx.head_wb_ena = I40E_HMC_TX_WB_ENABLE; 28929d26e4fcSRobert Mustacchi tctx.qlen = itrq->itrq_tx_ring_size; 28939d26e4fcSRobert Mustacchi tctx.tphrdesc_ena = I40E_HMC_TX_TPH_DISABLE; 28949d26e4fcSRobert Mustacchi tctx.tphrpacket_ena = I40E_HMC_TX_TPH_DISABLE; 28959d26e4fcSRobert Mustacchi tctx.tphwdesc_ena = I40E_HMC_TX_TPH_DISABLE; 28969d26e4fcSRobert Mustacchi tctx.head_wb_addr = itrq->itrq_desc_area.dmab_dma_address + 28979d26e4fcSRobert Mustacchi sizeof (i40e_tx_desc_t) * itrq->itrq_tx_ring_size; 28989d26e4fcSRobert Mustacchi 28999d26e4fcSRobert Mustacchi /* 29009d26e4fcSRobert Mustacchi * This field isn't actually documented, like crc, but it suggests that 29019d26e4fcSRobert Mustacchi * it should be zeroed. We leave both of these here because of that for 29029d26e4fcSRobert Mustacchi * now. We should check with Intel on why these are here even. 29039d26e4fcSRobert Mustacchi */ 29049d26e4fcSRobert Mustacchi tctx.crc = 0; 29059d26e4fcSRobert Mustacchi tctx.rdylist_act = 0; 29069d26e4fcSRobert Mustacchi 29079d26e4fcSRobert Mustacchi /* 29089d26e4fcSRobert Mustacchi * We're supposed to assign the rdylist field with the value of the 29099d26e4fcSRobert Mustacchi * traffic class index for the first device. We query the VSI parameters 29109d26e4fcSRobert Mustacchi * again to get what the handle is. Note that every queue is always 29119d26e4fcSRobert Mustacchi * assigned to traffic class zero, because we don't actually use them. 29129d26e4fcSRobert Mustacchi */ 29139d26e4fcSRobert Mustacchi bzero(&context, sizeof (struct i40e_vsi_context)); 291409aee612SRyan Zezeski context.seid = I40E_DEF_VSI_SEID(i40e); 29159d26e4fcSRobert Mustacchi context.pf_num = hw->pf_id; 29169d26e4fcSRobert Mustacchi err = i40e_aq_get_vsi_params(hw, &context, NULL); 29179d26e4fcSRobert Mustacchi if (err != I40E_SUCCESS) { 29189d26e4fcSRobert Mustacchi i40e_error(i40e, "get VSI params failed with %d", err); 29199d26e4fcSRobert Mustacchi return (B_FALSE); 29209d26e4fcSRobert Mustacchi } 29219d26e4fcSRobert Mustacchi tctx.rdylist = LE_16(context.info.qs_handle[0]); 29229d26e4fcSRobert Mustacchi 29239d26e4fcSRobert Mustacchi err = i40e_clear_lan_tx_queue_context(hw, itrq->itrq_index); 29249d26e4fcSRobert Mustacchi if (err != I40E_SUCCESS) { 29259d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to clear tx queue %d context: %d", 29269d26e4fcSRobert Mustacchi itrq->itrq_index, err); 29279d26e4fcSRobert Mustacchi return (B_FALSE); 29289d26e4fcSRobert Mustacchi } 29299d26e4fcSRobert Mustacchi 29309d26e4fcSRobert Mustacchi err = i40e_set_lan_tx_queue_context(hw, itrq->itrq_index, &tctx); 29319d26e4fcSRobert Mustacchi if (err != I40E_SUCCESS) { 29329d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to set tx queue %d context: %d", 29339d26e4fcSRobert Mustacchi itrq->itrq_index, err); 29349d26e4fcSRobert Mustacchi return (B_FALSE); 29359d26e4fcSRobert Mustacchi } 29369d26e4fcSRobert Mustacchi 29379d26e4fcSRobert Mustacchi return (B_TRUE); 29389d26e4fcSRobert Mustacchi } 29399d26e4fcSRobert Mustacchi 29409d26e4fcSRobert Mustacchi /* 29419d26e4fcSRobert Mustacchi * Take care of setting up the descriptor rings and actually programming the 29429d26e4fcSRobert Mustacchi * device. See 8.4.3.1.1 for what we need to do here. 29439d26e4fcSRobert Mustacchi */ 29449d26e4fcSRobert Mustacchi static boolean_t 29459d26e4fcSRobert Mustacchi i40e_setup_tx_rings(i40e_t *i40e) 29469d26e4fcSRobert Mustacchi { 29479d26e4fcSRobert Mustacchi int i; 29489d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 29499d26e4fcSRobert Mustacchi 29509d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 29519d26e4fcSRobert Mustacchi i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[i]; 29529d26e4fcSRobert Mustacchi uint32_t reg; 29539d26e4fcSRobert Mustacchi 29549d26e4fcSRobert Mustacchi /* 29559d26e4fcSRobert Mustacchi * Step 1. Clear the queue disable flag and verify that the 29569d26e4fcSRobert Mustacchi * index is set correctly. 29579d26e4fcSRobert Mustacchi */ 29589d26e4fcSRobert Mustacchi i40e_pre_tx_queue_cfg(hw, i, B_TRUE); 29599d26e4fcSRobert Mustacchi 29609d26e4fcSRobert Mustacchi /* 29619d26e4fcSRobert Mustacchi * Step 2. Prepare the queue's FPM/HMC context. 29629d26e4fcSRobert Mustacchi */ 29639d26e4fcSRobert Mustacchi if (i40e_setup_tx_hmc(itrq) == B_FALSE) 29649d26e4fcSRobert Mustacchi return (B_FALSE); 29659d26e4fcSRobert Mustacchi 29669d26e4fcSRobert Mustacchi /* 29679d26e4fcSRobert Mustacchi * Step 3. Verify that it's clear that this PF owns this queue. 29689d26e4fcSRobert Mustacchi */ 29699d26e4fcSRobert Mustacchi reg = I40E_QTX_CTL_PF_QUEUE; 29709d26e4fcSRobert Mustacchi reg |= (hw->pf_id << I40E_QTX_CTL_PF_INDX_SHIFT) & 29719d26e4fcSRobert Mustacchi I40E_QTX_CTL_PF_INDX_MASK; 29729d26e4fcSRobert Mustacchi I40E_WRITE_REG(hw, I40E_QTX_CTL(itrq->itrq_index), reg); 29739d26e4fcSRobert Mustacchi i40e_flush(hw); 29749d26e4fcSRobert Mustacchi 29759d26e4fcSRobert Mustacchi /* 29769d26e4fcSRobert Mustacchi * Step 4. Set the QENA_REQ flag. 29779d26e4fcSRobert Mustacchi */ 29789d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_QTX_ENA(i)); 29799d26e4fcSRobert Mustacchi VERIFY0(reg & (I40E_QTX_ENA_QENA_REQ_MASK | 29809d26e4fcSRobert Mustacchi I40E_QTX_ENA_QENA_STAT_MASK)); 29819d26e4fcSRobert Mustacchi reg |= I40E_QTX_ENA_QENA_REQ_MASK; 29829d26e4fcSRobert Mustacchi I40E_WRITE_REG(hw, I40E_QTX_ENA(i), reg); 29839d26e4fcSRobert Mustacchi } 29849d26e4fcSRobert Mustacchi 29859d26e4fcSRobert Mustacchi /* 29869d26e4fcSRobert Mustacchi * Note, we wait for every queue to be enabled before we start checking. 29879d26e4fcSRobert Mustacchi * This will hopefully cause most queues to be enabled at this point. 29889d26e4fcSRobert Mustacchi */ 29899d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 29909d26e4fcSRobert Mustacchi uint32_t j, reg; 29919d26e4fcSRobert Mustacchi 29929d26e4fcSRobert Mustacchi /* 29939d26e4fcSRobert Mustacchi * Step 5. Verify that QENA_STAT has been set. It's promised 29949d26e4fcSRobert Mustacchi * that this should occur within about 10 us, but like BSD, 29959d26e4fcSRobert Mustacchi * we'll try for up to 100 ms for this queue. 29969d26e4fcSRobert Mustacchi */ 29979d26e4fcSRobert Mustacchi for (j = 0; j < I40E_RING_WAIT_NTRIES; j++) { 29989d26e4fcSRobert Mustacchi reg = I40E_READ_REG(hw, I40E_QTX_ENA(i)); 29999d26e4fcSRobert Mustacchi 30009d26e4fcSRobert Mustacchi if (reg & I40E_QTX_ENA_QENA_STAT_MASK) 30019d26e4fcSRobert Mustacchi break; 30029d26e4fcSRobert Mustacchi i40e_msec_delay(I40E_RING_WAIT_PAUSE); 30039d26e4fcSRobert Mustacchi } 30049d26e4fcSRobert Mustacchi 30059d26e4fcSRobert Mustacchi if ((reg & I40E_QTX_ENA_QENA_STAT_MASK) == 0) { 30069d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to enable tx queue %d, timed " 30079d26e4fcSRobert Mustacchi "out", i); 30089d26e4fcSRobert Mustacchi return (B_FALSE); 30099d26e4fcSRobert Mustacchi } 30109d26e4fcSRobert Mustacchi } 30119d26e4fcSRobert Mustacchi 30129d26e4fcSRobert Mustacchi return (B_TRUE); 30139d26e4fcSRobert Mustacchi } 30149d26e4fcSRobert Mustacchi 30159d26e4fcSRobert Mustacchi void 30169d26e4fcSRobert Mustacchi i40e_stop(i40e_t *i40e, boolean_t free_allocations) 30179d26e4fcSRobert Mustacchi { 301809aee612SRyan Zezeski uint_t i; 301909aee612SRyan Zezeski i40e_hw_t *hw = &i40e->i40e_hw_space; 30209d26e4fcSRobert Mustacchi 30219d26e4fcSRobert Mustacchi ASSERT(MUTEX_HELD(&i40e->i40e_general_lock)); 30229d26e4fcSRobert Mustacchi 30239d26e4fcSRobert Mustacchi /* 30249d26e4fcSRobert Mustacchi * Shutdown and drain the tx and rx pipeline. We do this using the 30259d26e4fcSRobert Mustacchi * following steps. 30269d26e4fcSRobert Mustacchi * 30279d26e4fcSRobert Mustacchi * 1) Shutdown interrupts to all the queues (trying to keep the admin 30289d26e4fcSRobert Mustacchi * queue alive). 30299d26e4fcSRobert Mustacchi * 30309d26e4fcSRobert Mustacchi * 2) Remove all of the interrupt tx and rx causes by setting the 30319d26e4fcSRobert Mustacchi * interrupt linked lists to zero. 30329d26e4fcSRobert Mustacchi * 30339d26e4fcSRobert Mustacchi * 2) Shutdown the tx and rx rings. Because i40e_shutdown_rings() should 30349d26e4fcSRobert Mustacchi * wait for all the queues to be disabled, once we reach that point 30359d26e4fcSRobert Mustacchi * it should be safe to free associated data. 30369d26e4fcSRobert Mustacchi * 30379d26e4fcSRobert Mustacchi * 4) Wait 50ms after all that is done. This ensures that the rings are 30389d26e4fcSRobert Mustacchi * ready for programming again and we don't have to think about this 30399d26e4fcSRobert Mustacchi * in other parts of the driver. 30409d26e4fcSRobert Mustacchi * 30419d26e4fcSRobert Mustacchi * 5) Disable remaining chip interrupts, (admin queue, etc.) 30429d26e4fcSRobert Mustacchi * 30439d26e4fcSRobert Mustacchi * 6) Verify that FM is happy with all the register accesses we 30449d26e4fcSRobert Mustacchi * performed. 30459d26e4fcSRobert Mustacchi */ 30469d26e4fcSRobert Mustacchi i40e_intr_io_disable_all(i40e); 30479d26e4fcSRobert Mustacchi i40e_intr_io_clear_cause(i40e); 30489d26e4fcSRobert Mustacchi 30499d26e4fcSRobert Mustacchi if (i40e_shutdown_rings(i40e) == B_FALSE) { 30509d26e4fcSRobert Mustacchi ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST); 30519d26e4fcSRobert Mustacchi } 30529d26e4fcSRobert Mustacchi 30539d26e4fcSRobert Mustacchi delay(50 * drv_usectohz(1000)); 30549d26e4fcSRobert Mustacchi 305509aee612SRyan Zezeski /* 305609aee612SRyan Zezeski * We don't delete the default VSI because it replaces the VEB 305709aee612SRyan Zezeski * after VEB deletion (see the "Delete Element" section). 305809aee612SRyan Zezeski * Furthermore, since the default VSI is provided by the 305909aee612SRyan Zezeski * firmware, we never attempt to delete it. 306009aee612SRyan Zezeski */ 306109aee612SRyan Zezeski for (i = 1; i < i40e->i40e_num_rx_groups; i++) { 306209aee612SRyan Zezeski i40e_delete_vsi(i40e, i); 306309aee612SRyan Zezeski } 306409aee612SRyan Zezeski 306509aee612SRyan Zezeski if (i40e->i40e_veb_seid != 0) { 306609aee612SRyan Zezeski int rc = i40e_aq_delete_element(hw, i40e->i40e_veb_seid, NULL); 306709aee612SRyan Zezeski 306809aee612SRyan Zezeski if (rc != I40E_SUCCESS) { 306909aee612SRyan Zezeski i40e_error(i40e, "Failed to delete VEB %d: %d", rc, 307009aee612SRyan Zezeski hw->aq.asq_last_status); 307109aee612SRyan Zezeski } 307209aee612SRyan Zezeski 307309aee612SRyan Zezeski i40e->i40e_veb_seid = 0; 307409aee612SRyan Zezeski } 307509aee612SRyan Zezeski 30769d26e4fcSRobert Mustacchi i40e_intr_chip_fini(i40e); 30779d26e4fcSRobert Mustacchi 30789d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 30799d26e4fcSRobert Mustacchi mutex_enter(&i40e->i40e_trqpairs[i].itrq_rx_lock); 30809d26e4fcSRobert Mustacchi mutex_enter(&i40e->i40e_trqpairs[i].itrq_tx_lock); 30819d26e4fcSRobert Mustacchi } 30829d26e4fcSRobert Mustacchi 30839d26e4fcSRobert Mustacchi /* 30849d26e4fcSRobert Mustacchi * We should consider refactoring this to be part of the ring start / 30859d26e4fcSRobert Mustacchi * stop routines at some point. 30869d26e4fcSRobert Mustacchi */ 30879d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 30889d26e4fcSRobert Mustacchi i40e_stats_trqpair_fini(&i40e->i40e_trqpairs[i]); 30899d26e4fcSRobert Mustacchi } 30909d26e4fcSRobert Mustacchi 30919d26e4fcSRobert Mustacchi if (i40e_check_acc_handle(i40e->i40e_osdep_space.ios_cfg_handle) != 30929d26e4fcSRobert Mustacchi DDI_FM_OK) { 30939d26e4fcSRobert Mustacchi ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST); 30949d26e4fcSRobert Mustacchi } 30959d26e4fcSRobert Mustacchi 30969d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 30979d26e4fcSRobert Mustacchi i40e_tx_cleanup_ring(&i40e->i40e_trqpairs[i]); 30989d26e4fcSRobert Mustacchi } 30999d26e4fcSRobert Mustacchi 31009d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 31019d26e4fcSRobert Mustacchi mutex_exit(&i40e->i40e_trqpairs[i].itrq_rx_lock); 31029d26e4fcSRobert Mustacchi mutex_exit(&i40e->i40e_trqpairs[i].itrq_tx_lock); 31039d26e4fcSRobert Mustacchi } 31049d26e4fcSRobert Mustacchi 310509aee612SRyan Zezeski for (i = 0; i < i40e->i40e_num_rx_groups; i++) { 310609aee612SRyan Zezeski i40e_stat_vsi_fini(i40e, i); 310709aee612SRyan Zezeski } 31089d26e4fcSRobert Mustacchi 31099d26e4fcSRobert Mustacchi i40e->i40e_link_speed = 0; 31109d26e4fcSRobert Mustacchi i40e->i40e_link_duplex = 0; 31119d26e4fcSRobert Mustacchi i40e_link_state_set(i40e, LINK_STATE_UNKNOWN); 31129d26e4fcSRobert Mustacchi 31139d26e4fcSRobert Mustacchi if (free_allocations) { 31149d26e4fcSRobert Mustacchi i40e_free_ring_mem(i40e, B_FALSE); 31159d26e4fcSRobert Mustacchi } 31169d26e4fcSRobert Mustacchi } 31179d26e4fcSRobert Mustacchi 31189d26e4fcSRobert Mustacchi boolean_t 31199d26e4fcSRobert Mustacchi i40e_start(i40e_t *i40e, boolean_t alloc) 31209d26e4fcSRobert Mustacchi { 31219d26e4fcSRobert Mustacchi i40e_hw_t *hw = &i40e->i40e_hw_space; 31229d26e4fcSRobert Mustacchi boolean_t rc = B_TRUE; 31239d26e4fcSRobert Mustacchi int i, err; 31249d26e4fcSRobert Mustacchi 31259d26e4fcSRobert Mustacchi ASSERT(MUTEX_HELD(&i40e->i40e_general_lock)); 31269d26e4fcSRobert Mustacchi 31279d26e4fcSRobert Mustacchi if (alloc) { 31289d26e4fcSRobert Mustacchi if (i40e_alloc_ring_mem(i40e) == B_FALSE) { 31299d26e4fcSRobert Mustacchi i40e_error(i40e, 31309d26e4fcSRobert Mustacchi "Failed to allocate ring memory"); 31319d26e4fcSRobert Mustacchi return (B_FALSE); 31329d26e4fcSRobert Mustacchi } 31339d26e4fcSRobert Mustacchi } 31349d26e4fcSRobert Mustacchi 31359d26e4fcSRobert Mustacchi /* 31369d26e4fcSRobert Mustacchi * This should get refactored to be part of ring start and stop at 31379d26e4fcSRobert Mustacchi * some point, along with most of the logic here. 31389d26e4fcSRobert Mustacchi */ 31399d26e4fcSRobert Mustacchi for (i = 0; i < i40e->i40e_num_trqpairs; i++) { 31409d26e4fcSRobert Mustacchi if (i40e_stats_trqpair_init(&i40e->i40e_trqpairs[i]) == 31419d26e4fcSRobert Mustacchi B_FALSE) { 31429d26e4fcSRobert Mustacchi int j; 31439d26e4fcSRobert Mustacchi 31449d26e4fcSRobert Mustacchi for (j = 0; j < i; j++) { 31459d26e4fcSRobert Mustacchi i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[j]; 31469d26e4fcSRobert Mustacchi i40e_stats_trqpair_fini(itrq); 31479d26e4fcSRobert Mustacchi } 31489d26e4fcSRobert Mustacchi return (B_FALSE); 31499d26e4fcSRobert Mustacchi } 31509d26e4fcSRobert Mustacchi } 31519d26e4fcSRobert Mustacchi 31529d26e4fcSRobert Mustacchi if (!i40e_chip_start(i40e)) { 31539d26e4fcSRobert Mustacchi i40e_fm_ereport(i40e, DDI_FM_DEVICE_INVAL_STATE); 31549d26e4fcSRobert Mustacchi rc = B_FALSE; 31559d26e4fcSRobert Mustacchi goto done; 31569d26e4fcSRobert Mustacchi } 31579d26e4fcSRobert Mustacchi 31589d26e4fcSRobert Mustacchi if (i40e_setup_rx_rings(i40e) == B_FALSE) { 31599d26e4fcSRobert Mustacchi rc = B_FALSE; 31609d26e4fcSRobert Mustacchi goto done; 31619d26e4fcSRobert Mustacchi } 31629d26e4fcSRobert Mustacchi 31639d26e4fcSRobert Mustacchi if (i40e_setup_tx_rings(i40e) == B_FALSE) { 31649d26e4fcSRobert Mustacchi rc = B_FALSE; 31659d26e4fcSRobert Mustacchi goto done; 31669d26e4fcSRobert Mustacchi } 31679d26e4fcSRobert Mustacchi 31689d26e4fcSRobert Mustacchi /* 31699d26e4fcSRobert Mustacchi * Enable broadcast traffic; however, do not enable multicast traffic. 31709d26e4fcSRobert Mustacchi * That's handle exclusively through MAC's mc_multicst routines. 31719d26e4fcSRobert Mustacchi */ 317209aee612SRyan Zezeski err = i40e_aq_set_vsi_broadcast(hw, I40E_DEF_VSI_SEID(i40e), B_TRUE, 317309aee612SRyan Zezeski NULL); 31749d26e4fcSRobert Mustacchi if (err != I40E_SUCCESS) { 31759d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to set default VSI: %d", err); 31769d26e4fcSRobert Mustacchi rc = B_FALSE; 31779d26e4fcSRobert Mustacchi goto done; 31789d26e4fcSRobert Mustacchi } 31799d26e4fcSRobert Mustacchi 31809d26e4fcSRobert Mustacchi err = i40e_aq_set_mac_config(hw, i40e->i40e_frame_max, B_TRUE, 0, NULL); 31819d26e4fcSRobert Mustacchi if (err != I40E_SUCCESS) { 31829d26e4fcSRobert Mustacchi i40e_error(i40e, "failed to set MAC config: %d", err); 31839d26e4fcSRobert Mustacchi rc = B_FALSE; 31849d26e4fcSRobert Mustacchi goto done; 31859d26e4fcSRobert Mustacchi } 31869d26e4fcSRobert Mustacchi 31879d26e4fcSRobert Mustacchi /* 31889d26e4fcSRobert Mustacchi * Finally, make sure that we're happy from an FM perspective. 31899d26e4fcSRobert Mustacchi */ 31909d26e4fcSRobert Mustacchi if (i40e_check_acc_handle(i40e->i40e_osdep_space.ios_reg_handle) != 31919d26e4fcSRobert Mustacchi DDI_FM_OK) { 31929d26e4fcSRobert Mustacchi rc = B_FALSE; 31939d26e4fcSRobert Mustacchi goto done; 31949d26e4fcSRobert Mustacchi } 31959d26e4fcSRobert Mustacchi 31969d26e4fcSRobert Mustacchi /* Clear state bits prior to final interrupt enabling. */ 31979d26e4fcSRobert Mustacchi atomic_and_32(&i40e->i40e_state, 31989d26e4fcSRobert Mustacchi ~(I40E_ERROR | I40E_STALL | I40E_OVERTEMP)); 31999d26e4fcSRobert Mustacchi 32009d26e4fcSRobert Mustacchi i40e_intr_io_enable_all(i40e); 32019d26e4fcSRobert Mustacchi 32029d26e4fcSRobert Mustacchi done: 32039d26e4fcSRobert Mustacchi if (rc == B_FALSE) { 32049d26e4fcSRobert Mustacchi i40e_stop(i40e, B_FALSE); 32059d26e4fcSRobert Mustacchi if (alloc == B_TRUE) { 32069d26e4fcSRobert Mustacchi i40e_free_ring_mem(i40e, B_TRUE); 32079d26e4fcSRobert Mustacchi } 32089d26e4fcSRobert Mustacchi ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST); 32099d26e4fcSRobert Mustacchi } 32109d26e4fcSRobert Mustacchi 32119d26e4fcSRobert Mustacchi return (rc); 32129d26e4fcSRobert Mustacchi } 32139d26e4fcSRobert Mustacchi 32149d26e4fcSRobert Mustacchi /* 32159d26e4fcSRobert Mustacchi * We may have loaned up descriptors to the stack. As such, if we still have 32169d26e4fcSRobert Mustacchi * them outstanding, then we will not continue with detach. 32179d26e4fcSRobert Mustacchi */ 32189d26e4fcSRobert Mustacchi static boolean_t 32199d26e4fcSRobert Mustacchi i40e_drain_rx(i40e_t *i40e) 32209d26e4fcSRobert Mustacchi { 32219d26e4fcSRobert Mustacchi mutex_enter(&i40e->i40e_rx_pending_lock); 32229d26e4fcSRobert Mustacchi while (i40e->i40e_rx_pending > 0) { 32239d26e4fcSRobert Mustacchi if (cv_reltimedwait(&i40e->i40e_rx_pending_cv, 32249d26e4fcSRobert Mustacchi &i40e->i40e_rx_pending_lock, 32259d26e4fcSRobert Mustacchi drv_usectohz(I40E_DRAIN_RX_WAIT), TR_CLOCK_TICK) == -1) { 32269d26e4fcSRobert Mustacchi mutex_exit(&i40e->i40e_rx_pending_lock); 32279d26e4fcSRobert Mustacchi return (B_FALSE); 32289d26e4fcSRobert Mustacchi } 32299d26e4fcSRobert Mustacchi } 32309d26e4fcSRobert Mustacchi mutex_exit(&i40e->i40e_rx_pending_lock); 32319d26e4fcSRobert Mustacchi 32329d26e4fcSRobert Mustacchi return (B_TRUE); 32339d26e4fcSRobert Mustacchi } 32349d26e4fcSRobert Mustacchi 3235508a0e8cSRob Johnston /* 3236508a0e8cSRob Johnston * DDI UFM Callbacks 3237508a0e8cSRob Johnston */ 3238508a0e8cSRob Johnston static int 3239508a0e8cSRob Johnston i40e_ufm_fill_image(ddi_ufm_handle_t *ufmh, void *arg, uint_t imgno, 3240508a0e8cSRob Johnston ddi_ufm_image_t *img) 3241508a0e8cSRob Johnston { 3242508a0e8cSRob Johnston if (imgno != 0) 3243508a0e8cSRob Johnston return (EINVAL); 3244508a0e8cSRob Johnston 3245508a0e8cSRob Johnston ddi_ufm_image_set_desc(img, "Firmware"); 3246508a0e8cSRob Johnston ddi_ufm_image_set_nslots(img, 1); 3247508a0e8cSRob Johnston 3248508a0e8cSRob Johnston return (0); 3249508a0e8cSRob Johnston } 3250508a0e8cSRob Johnston 3251508a0e8cSRob Johnston static int 3252508a0e8cSRob Johnston i40e_ufm_fill_slot(ddi_ufm_handle_t *ufmh, void *arg, uint_t imgno, 3253508a0e8cSRob Johnston uint_t slotno, ddi_ufm_slot_t *slot) 3254508a0e8cSRob Johnston { 3255508a0e8cSRob Johnston i40e_t *i40e = (i40e_t *)arg; 3256508a0e8cSRob Johnston char *fw_ver = NULL, *fw_bld = NULL, *api_ver = NULL; 3257508a0e8cSRob Johnston nvlist_t *misc = NULL; 3258508a0e8cSRob Johnston uint_t flags = DDI_PROP_DONTPASS; 3259508a0e8cSRob Johnston int err; 3260508a0e8cSRob Johnston 3261508a0e8cSRob Johnston if (imgno != 0 || slotno != 0 || 3262508a0e8cSRob Johnston ddi_prop_lookup_string(DDI_DEV_T_ANY, i40e->i40e_dip, flags, 3263508a0e8cSRob Johnston "firmware-version", &fw_ver) != DDI_PROP_SUCCESS || 3264508a0e8cSRob Johnston ddi_prop_lookup_string(DDI_DEV_T_ANY, i40e->i40e_dip, flags, 3265508a0e8cSRob Johnston "firmware-build", &fw_bld) != DDI_PROP_SUCCESS || 3266508a0e8cSRob Johnston ddi_prop_lookup_string(DDI_DEV_T_ANY, i40e->i40e_dip, flags, 3267508a0e8cSRob Johnston "api-version", &api_ver) != DDI_PROP_SUCCESS) { 3268508a0e8cSRob Johnston err = EINVAL; 3269508a0e8cSRob Johnston goto err; 3270508a0e8cSRob Johnston } 3271508a0e8cSRob Johnston 3272508a0e8cSRob Johnston ddi_ufm_slot_set_attrs(slot, DDI_UFM_ATTR_ACTIVE); 3273508a0e8cSRob Johnston ddi_ufm_slot_set_version(slot, fw_ver); 3274508a0e8cSRob Johnston 3275508a0e8cSRob Johnston (void) nvlist_alloc(&misc, NV_UNIQUE_NAME, KM_SLEEP); 3276508a0e8cSRob Johnston if ((err = nvlist_add_string(misc, "firmware-build", fw_bld)) != 0 || 3277508a0e8cSRob Johnston (err = nvlist_add_string(misc, "api-version", api_ver)) != 0) { 3278508a0e8cSRob Johnston goto err; 3279508a0e8cSRob Johnston } 3280508a0e8cSRob Johnston ddi_ufm_slot_set_misc(slot, misc); 3281508a0e8cSRob Johnston 3282508a0e8cSRob Johnston ddi_prop_free(fw_ver); 3283508a0e8cSRob Johnston ddi_prop_free(fw_bld); 3284508a0e8cSRob Johnston ddi_prop_free(api_ver); 3285508a0e8cSRob Johnston 3286508a0e8cSRob Johnston return (0); 3287508a0e8cSRob Johnston err: 3288508a0e8cSRob Johnston nvlist_free(misc); 3289508a0e8cSRob Johnston if (fw_ver != NULL) 3290508a0e8cSRob Johnston ddi_prop_free(fw_ver); 3291508a0e8cSRob Johnston if (fw_bld != NULL) 3292508a0e8cSRob Johnston ddi_prop_free(fw_bld); 3293508a0e8cSRob Johnston if (api_ver != NULL) 3294508a0e8cSRob Johnston ddi_prop_free(api_ver); 3295508a0e8cSRob Johnston 3296508a0e8cSRob Johnston return (err); 3297508a0e8cSRob Johnston } 3298508a0e8cSRob Johnston 3299508a0e8cSRob Johnston static int 3300508a0e8cSRob Johnston i40e_ufm_getcaps(ddi_ufm_handle_t *ufmh, void *arg, ddi_ufm_cap_t *caps) 3301508a0e8cSRob Johnston { 3302508a0e8cSRob Johnston *caps = DDI_UFM_CAP_REPORT; 3303508a0e8cSRob Johnston 3304508a0e8cSRob Johnston return (0); 3305508a0e8cSRob Johnston } 3306508a0e8cSRob Johnston 3307508a0e8cSRob Johnston static ddi_ufm_ops_t i40e_ufm_ops = { 3308508a0e8cSRob Johnston NULL, 3309508a0e8cSRob Johnston i40e_ufm_fill_image, 3310508a0e8cSRob Johnston i40e_ufm_fill_slot, 3311508a0e8cSRob Johnston i40e_ufm_getcaps 3312508a0e8cSRob Johnston }; 3313508a0e8cSRob Johnston 33149d26e4fcSRobert Mustacchi static int 33159d26e4fcSRobert Mustacchi i40e_attach(dev_info_t *devinfo, ddi_attach_cmd_t cmd) 33169d26e4fcSRobert Mustacchi { 33179d26e4fcSRobert Mustacchi i40e_t *i40e; 33189d26e4fcSRobert Mustacchi struct i40e_osdep *osdep; 33199d26e4fcSRobert Mustacchi i40e_hw_t *hw; 33209d26e4fcSRobert Mustacchi int instance; 33219d26e4fcSRobert Mustacchi 33229d26e4fcSRobert Mustacchi if (cmd != DDI_ATTACH) 33239d26e4fcSRobert Mustacchi return (DDI_FAILURE); 33249d26e4fcSRobert Mustacchi 33259d26e4fcSRobert Mustacchi instance = ddi_get_instance(devinfo); 33269d26e4fcSRobert Mustacchi i40e = kmem_zalloc(sizeof (i40e_t), KM_SLEEP); 33279d26e4fcSRobert Mustacchi 33289d26e4fcSRobert Mustacchi i40e->i40e_aqbuf = kmem_zalloc(I40E_ADMINQ_BUFSZ, KM_SLEEP); 33299d26e4fcSRobert Mustacchi i40e->i40e_instance = instance; 33309d26e4fcSRobert Mustacchi i40e->i40e_dip = devinfo; 33319d26e4fcSRobert Mustacchi 33329d26e4fcSRobert Mustacchi hw = &i40e->i40e_hw_space; 33339d26e4fcSRobert Mustacchi osdep = &i40e->i40e_osdep_space; 33349d26e4fcSRobert Mustacchi hw->back = osdep; 33359d26e4fcSRobert Mustacchi osdep->ios_i40e = i40e; 33369d26e4fcSRobert Mustacchi 33379d26e4fcSRobert Mustacchi ddi_set_driver_private(devinfo, i40e); 33389d26e4fcSRobert Mustacchi 33399d26e4fcSRobert Mustacchi i40e_fm_init(i40e); 33409d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_FM_INIT; 33419d26e4fcSRobert Mustacchi 33429d26e4fcSRobert Mustacchi if (pci_config_setup(devinfo, &osdep->ios_cfg_handle) != DDI_SUCCESS) { 33439d26e4fcSRobert Mustacchi i40e_error(i40e, "Failed to map PCI configurations."); 33449d26e4fcSRobert Mustacchi goto attach_fail; 33459d26e4fcSRobert Mustacchi } 33469d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_PCI_CONFIG; 33479d26e4fcSRobert Mustacchi 33483d75a287SRobert Mustacchi i40e_identify_hardware(i40e); 33499d26e4fcSRobert Mustacchi 33509d26e4fcSRobert Mustacchi if (!i40e_regs_map(i40e)) { 33519d26e4fcSRobert Mustacchi i40e_error(i40e, "Failed to map device registers."); 33529d26e4fcSRobert Mustacchi goto attach_fail; 33539d26e4fcSRobert Mustacchi } 33549d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_REGS_MAP; 33559d26e4fcSRobert Mustacchi 33569d26e4fcSRobert Mustacchi i40e_init_properties(i40e); 33579d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_PROPS; 33589d26e4fcSRobert Mustacchi 33599d26e4fcSRobert Mustacchi if (!i40e_common_code_init(i40e, hw)) 33609d26e4fcSRobert Mustacchi goto attach_fail; 33619d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_COMMON_CODE; 33629d26e4fcSRobert Mustacchi 33639d26e4fcSRobert Mustacchi /* 33649d26e4fcSRobert Mustacchi * When we participate in IRM, we should make sure that we register 33659d26e4fcSRobert Mustacchi * ourselves with it before callbacks. 33669d26e4fcSRobert Mustacchi */ 33679d26e4fcSRobert Mustacchi if (!i40e_alloc_intrs(i40e, devinfo)) { 33689d26e4fcSRobert Mustacchi i40e_error(i40e, "Failed to allocate interrupts."); 33699d26e4fcSRobert Mustacchi goto attach_fail; 33709d26e4fcSRobert Mustacchi } 33719d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_ALLOC_INTR; 33729d26e4fcSRobert Mustacchi 33739d26e4fcSRobert Mustacchi if (!i40e_alloc_trqpairs(i40e)) { 33749d26e4fcSRobert Mustacchi i40e_error(i40e, 33759d26e4fcSRobert Mustacchi "Failed to allocate receive & transmit rings."); 33769d26e4fcSRobert Mustacchi goto attach_fail; 33779d26e4fcSRobert Mustacchi } 33789d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_ALLOC_RINGSLOCKS; 33799d26e4fcSRobert Mustacchi 33809d26e4fcSRobert Mustacchi if (!i40e_map_intrs_to_vectors(i40e)) { 33819d26e4fcSRobert Mustacchi i40e_error(i40e, "Failed to map interrupts to vectors."); 33829d26e4fcSRobert Mustacchi goto attach_fail; 33839d26e4fcSRobert Mustacchi } 33849d26e4fcSRobert Mustacchi 33859d26e4fcSRobert Mustacchi if (!i40e_add_intr_handlers(i40e)) { 33869d26e4fcSRobert Mustacchi i40e_error(i40e, "Failed to add the interrupt handlers."); 33879d26e4fcSRobert Mustacchi goto attach_fail; 33889d26e4fcSRobert Mustacchi } 33899d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_ADD_INTR; 33909d26e4fcSRobert Mustacchi 33919d26e4fcSRobert Mustacchi if (!i40e_final_init(i40e)) { 33929d26e4fcSRobert Mustacchi i40e_error(i40e, "Final initialization failed."); 33939d26e4fcSRobert Mustacchi goto attach_fail; 33949d26e4fcSRobert Mustacchi } 33959d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_INIT; 33969d26e4fcSRobert Mustacchi 33979d26e4fcSRobert Mustacchi if (i40e_check_acc_handle(i40e->i40e_osdep_space.ios_cfg_handle) != 33989d26e4fcSRobert Mustacchi DDI_FM_OK) { 33999d26e4fcSRobert Mustacchi ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST); 34009d26e4fcSRobert Mustacchi goto attach_fail; 34019d26e4fcSRobert Mustacchi } 34029d26e4fcSRobert Mustacchi 34039d26e4fcSRobert Mustacchi if (!i40e_stats_init(i40e)) { 34049d26e4fcSRobert Mustacchi i40e_error(i40e, "Stats initialization failed."); 34059d26e4fcSRobert Mustacchi goto attach_fail; 34069d26e4fcSRobert Mustacchi } 34079d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_STATS; 34089d26e4fcSRobert Mustacchi 34099d26e4fcSRobert Mustacchi if (!i40e_register_mac(i40e)) { 34109d26e4fcSRobert Mustacchi i40e_error(i40e, "Failed to register to MAC/GLDv3"); 34119d26e4fcSRobert Mustacchi goto attach_fail; 34129d26e4fcSRobert Mustacchi } 34139d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_MAC; 34149d26e4fcSRobert Mustacchi 34159d26e4fcSRobert Mustacchi i40e->i40e_periodic_id = ddi_periodic_add(i40e_timer, i40e, 34169d26e4fcSRobert Mustacchi I40E_CYCLIC_PERIOD, DDI_IPL_0); 34179d26e4fcSRobert Mustacchi if (i40e->i40e_periodic_id == 0) { 34189d26e4fcSRobert Mustacchi i40e_error(i40e, "Failed to add the link-check timer"); 34199d26e4fcSRobert Mustacchi goto attach_fail; 34209d26e4fcSRobert Mustacchi } 34219d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_LINK_TIMER; 34229d26e4fcSRobert Mustacchi 34239d26e4fcSRobert Mustacchi if (!i40e_enable_interrupts(i40e)) { 34249d26e4fcSRobert Mustacchi i40e_error(i40e, "Failed to enable DDI interrupts"); 34259d26e4fcSRobert Mustacchi goto attach_fail; 34269d26e4fcSRobert Mustacchi } 34279d26e4fcSRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_ENABLE_INTR; 34289d26e4fcSRobert Mustacchi 342944b0ba91SRobert Mustacchi if (i40e->i40e_hw_space.bus.func == 0) { 343044b0ba91SRobert Mustacchi if (ddi_ufm_init(i40e->i40e_dip, DDI_UFM_CURRENT_VERSION, 343144b0ba91SRobert Mustacchi &i40e_ufm_ops, &i40e->i40e_ufmh, i40e) != 0) { 343244b0ba91SRobert Mustacchi i40e_error(i40e, "failed to initialize UFM subsystem"); 343344b0ba91SRobert Mustacchi goto attach_fail; 343444b0ba91SRobert Mustacchi } 343544b0ba91SRobert Mustacchi ddi_ufm_update(i40e->i40e_ufmh); 343644b0ba91SRobert Mustacchi i40e->i40e_attach_progress |= I40E_ATTACH_UFM_INIT; 3437508a0e8cSRob Johnston } 3438508a0e8cSRob Johnston 34399d26e4fcSRobert Mustacchi atomic_or_32(&i40e->i40e_state, I40E_INITIALIZED); 34409d26e4fcSRobert Mustacchi 34419d26e4fcSRobert Mustacchi mutex_enter(&i40e_glock); 34429d26e4fcSRobert Mustacchi list_insert_tail(&i40e_glist, i40e); 34439d26e4fcSRobert Mustacchi mutex_exit(&i40e_glock); 34449d26e4fcSRobert Mustacchi 34459d26e4fcSRobert Mustacchi return (DDI_SUCCESS); 34469d26e4fcSRobert Mustacchi 34479d26e4fcSRobert Mustacchi attach_fail: 34489d26e4fcSRobert Mustacchi i40e_unconfigure(devinfo, i40e); 34499d26e4fcSRobert Mustacchi return (DDI_FAILURE); 34509d26e4fcSRobert Mustacchi } 34519d26e4fcSRobert Mustacchi 34529d26e4fcSRobert Mustacchi static int 34539d26e4fcSRobert Mustacchi i40e_detach(dev_info_t *devinfo, ddi_detach_cmd_t cmd) 34549d26e4fcSRobert Mustacchi { 34559d26e4fcSRobert Mustacchi i40e_t *i40e; 34569d26e4fcSRobert Mustacchi 34579d26e4fcSRobert Mustacchi if (cmd != DDI_DETACH) 34589d26e4fcSRobert Mustacchi return (DDI_FAILURE); 34599d26e4fcSRobert Mustacchi 34609d26e4fcSRobert Mustacchi i40e = (i40e_t *)ddi_get_driver_private(devinfo); 34619d26e4fcSRobert Mustacchi if (i40e == NULL) { 34629d26e4fcSRobert Mustacchi i40e_log(NULL, "i40e_detach() called with no i40e pointer!"); 34639d26e4fcSRobert Mustacchi return (DDI_FAILURE); 34649d26e4fcSRobert Mustacchi } 34659d26e4fcSRobert Mustacchi 34669d26e4fcSRobert Mustacchi if (i40e_drain_rx(i40e) == B_FALSE) { 34679d26e4fcSRobert Mustacchi i40e_log(i40e, "timed out draining DMA resources, %d buffers " 34689d26e4fcSRobert Mustacchi "remain", i40e->i40e_rx_pending); 34699d26e4fcSRobert Mustacchi return (DDI_FAILURE); 34709d26e4fcSRobert Mustacchi } 34719d26e4fcSRobert Mustacchi 34729d26e4fcSRobert Mustacchi mutex_enter(&i40e_glock); 34739d26e4fcSRobert Mustacchi list_remove(&i40e_glist, i40e); 34749d26e4fcSRobert Mustacchi mutex_exit(&i40e_glock); 34759d26e4fcSRobert Mustacchi 34769d26e4fcSRobert Mustacchi i40e_unconfigure(devinfo, i40e); 34779d26e4fcSRobert Mustacchi 34789d26e4fcSRobert Mustacchi return (DDI_SUCCESS); 34799d26e4fcSRobert Mustacchi } 34809d26e4fcSRobert Mustacchi 34819d26e4fcSRobert Mustacchi static struct cb_ops i40e_cb_ops = { 34829d26e4fcSRobert Mustacchi nulldev, /* cb_open */ 34839d26e4fcSRobert Mustacchi nulldev, /* cb_close */ 34849d26e4fcSRobert Mustacchi nodev, /* cb_strategy */ 34859d26e4fcSRobert Mustacchi nodev, /* cb_print */ 34869d26e4fcSRobert Mustacchi nodev, /* cb_dump */ 34879d26e4fcSRobert Mustacchi nodev, /* cb_read */ 34889d26e4fcSRobert Mustacchi nodev, /* cb_write */ 34899d26e4fcSRobert Mustacchi nodev, /* cb_ioctl */ 34909d26e4fcSRobert Mustacchi nodev, /* cb_devmap */ 34919d26e4fcSRobert Mustacchi nodev, /* cb_mmap */ 34929d26e4fcSRobert Mustacchi nodev, /* cb_segmap */ 34939d26e4fcSRobert Mustacchi nochpoll, /* cb_chpoll */ 34949d26e4fcSRobert Mustacchi ddi_prop_op, /* cb_prop_op */ 34959d26e4fcSRobert Mustacchi NULL, /* cb_stream */ 34969d26e4fcSRobert Mustacchi D_MP | D_HOTPLUG, /* cb_flag */ 34979d26e4fcSRobert Mustacchi CB_REV, /* cb_rev */ 34989d26e4fcSRobert Mustacchi nodev, /* cb_aread */ 34999d26e4fcSRobert Mustacchi nodev /* cb_awrite */ 35009d26e4fcSRobert Mustacchi }; 35019d26e4fcSRobert Mustacchi 35029d26e4fcSRobert Mustacchi static struct dev_ops i40e_dev_ops = { 35039d26e4fcSRobert Mustacchi DEVO_REV, /* devo_rev */ 35049d26e4fcSRobert Mustacchi 0, /* devo_refcnt */ 35059d26e4fcSRobert Mustacchi NULL, /* devo_getinfo */ 35069d26e4fcSRobert Mustacchi nulldev, /* devo_identify */ 35079d26e4fcSRobert Mustacchi nulldev, /* devo_probe */ 35089d26e4fcSRobert Mustacchi i40e_attach, /* devo_attach */ 35099d26e4fcSRobert Mustacchi i40e_detach, /* devo_detach */ 35109d26e4fcSRobert Mustacchi nodev, /* devo_reset */ 35119d26e4fcSRobert Mustacchi &i40e_cb_ops, /* devo_cb_ops */ 35129d26e4fcSRobert Mustacchi NULL, /* devo_bus_ops */ 35139d26e4fcSRobert Mustacchi ddi_power, /* devo_power */ 35149d26e4fcSRobert Mustacchi ddi_quiesce_not_supported /* devo_quiesce */ 35159d26e4fcSRobert Mustacchi }; 35169d26e4fcSRobert Mustacchi 35179d26e4fcSRobert Mustacchi static struct modldrv i40e_modldrv = { 35189d26e4fcSRobert Mustacchi &mod_driverops, 35199d26e4fcSRobert Mustacchi i40e_ident, 35209d26e4fcSRobert Mustacchi &i40e_dev_ops 35219d26e4fcSRobert Mustacchi }; 35229d26e4fcSRobert Mustacchi 35239d26e4fcSRobert Mustacchi static struct modlinkage i40e_modlinkage = { 35249d26e4fcSRobert Mustacchi MODREV_1, 35259d26e4fcSRobert Mustacchi &i40e_modldrv, 35269d26e4fcSRobert Mustacchi NULL 35279d26e4fcSRobert Mustacchi }; 35289d26e4fcSRobert Mustacchi 35299d26e4fcSRobert Mustacchi /* 35309d26e4fcSRobert Mustacchi * Module Initialization Functions. 35319d26e4fcSRobert Mustacchi */ 35329d26e4fcSRobert Mustacchi int 35339d26e4fcSRobert Mustacchi _init(void) 35349d26e4fcSRobert Mustacchi { 35359d26e4fcSRobert Mustacchi int status; 35369d26e4fcSRobert Mustacchi 35379d26e4fcSRobert Mustacchi list_create(&i40e_glist, sizeof (i40e_t), offsetof(i40e_t, i40e_glink)); 35389d26e4fcSRobert Mustacchi list_create(&i40e_dlist, sizeof (i40e_device_t), 35399d26e4fcSRobert Mustacchi offsetof(i40e_device_t, id_link)); 35409d26e4fcSRobert Mustacchi mutex_init(&i40e_glock, NULL, MUTEX_DRIVER, NULL); 35419d26e4fcSRobert Mustacchi mac_init_ops(&i40e_dev_ops, I40E_MODULE_NAME); 35429d26e4fcSRobert Mustacchi 35439d26e4fcSRobert Mustacchi status = mod_install(&i40e_modlinkage); 35449d26e4fcSRobert Mustacchi if (status != DDI_SUCCESS) { 35459d26e4fcSRobert Mustacchi mac_fini_ops(&i40e_dev_ops); 35469d26e4fcSRobert Mustacchi mutex_destroy(&i40e_glock); 35479d26e4fcSRobert Mustacchi list_destroy(&i40e_dlist); 35489d26e4fcSRobert Mustacchi list_destroy(&i40e_glist); 35499d26e4fcSRobert Mustacchi } 35509d26e4fcSRobert Mustacchi 35519d26e4fcSRobert Mustacchi return (status); 35529d26e4fcSRobert Mustacchi } 35539d26e4fcSRobert Mustacchi 35549d26e4fcSRobert Mustacchi int 35559d26e4fcSRobert Mustacchi _info(struct modinfo *modinfop) 35569d26e4fcSRobert Mustacchi { 35579d26e4fcSRobert Mustacchi return (mod_info(&i40e_modlinkage, modinfop)); 35589d26e4fcSRobert Mustacchi } 35599d26e4fcSRobert Mustacchi 35609d26e4fcSRobert Mustacchi int 35619d26e4fcSRobert Mustacchi _fini(void) 35629d26e4fcSRobert Mustacchi { 35639d26e4fcSRobert Mustacchi int status; 35649d26e4fcSRobert Mustacchi 35659d26e4fcSRobert Mustacchi status = mod_remove(&i40e_modlinkage); 35669d26e4fcSRobert Mustacchi if (status == DDI_SUCCESS) { 35679d26e4fcSRobert Mustacchi mac_fini_ops(&i40e_dev_ops); 35689d26e4fcSRobert Mustacchi mutex_destroy(&i40e_glock); 35699d26e4fcSRobert Mustacchi list_destroy(&i40e_dlist); 35709d26e4fcSRobert Mustacchi list_destroy(&i40e_glist); 35719d26e4fcSRobert Mustacchi } 35729d26e4fcSRobert Mustacchi 35739d26e4fcSRobert Mustacchi return (status); 35749d26e4fcSRobert Mustacchi } 3575