History log of /freebsd-head/sys/arm64/
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
1c19645dc2492e3412a6ea01b17992ab4228d9cd 09-Oct-2020 imp <imp@FreeBSD.org> Create in-tree LINT files

Now that config(8) has supported include for 19 years, transition to
including the NOTES files. include support didn't exist at the time,
nor did the envvar stuff recently added. Now that it does, eliminate
the building of LINT files by just including everything you need.

Note: This may cause conflicts with updating in some cases.
find sys -name LINT\* -rm
is suggested across this commit to remove the generated LINT
files.

Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D26540
onf/LINT
onf/Makefile
ecb56912df2b529f1cc38d58eae0b9b1139395b2 08-Oct-2020 mhorne <mhorne@FreeBSD.org> Add a routine to dump boot metadata

The boot metadata (also referred to as modinfo, or preload metadata)
provides information about the size and location of the kernel,
pre-loaded modules, and other metadata (e.g. the EFI framebuffer) to be
consumed during by the kernel during early boot. It is encoded as a
series of type-length-value entries and is usually constructed by
loader(8) and passed to the kernel. It is also faked on some
architectures when booted by other means.

Although much of the module information is available via kldstat(8),
there is no easy way to debug the metadata in its entirety. Add some
routines to parse this data and allow it to be printed to the console
during early boot or output via a sysctl.

Since the output can be lengthly, printing to the console is gated
behind the debug.dump_modinfo_at_boot kenv variable as well as the
BOOTVERBOSE flag. The sysctl to print the metadata is named
debug.dump_modinfo.

Reviewed by: tsoome
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D26687
rm64/machdep.c
66fb704e75989a621f47c46efa55572299a17401 07-Oct-2020 mhorne <mhorne@FreeBSD.org> Print symbol index for unsupported relocation types

It is unlikely, but possible, that an unrecognized or unsupported
relocation type is encountered while trying to load a kernel module. If
this occurs we should offer the symbol index as a hint to the user.

While here, fix some small style issues.

Reviewed by: markj, kib (amd64 part, in D26701)
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
rm64/elf_machdep.c
be830cb3a5753af3cb3913fd087e4380329c3375 05-Oct-2020 trasz <trasz@FreeBSD.org> Tweak arm64's cpu_fetch_syscall_args(). This should make it possible
for the compiler to inline the memcpy().

Reviewed by: andrew
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26629
rm64/trap.c
88d21a5afef3005babe238f324ce2ff6ff5222af 02-Oct-2020 manu <manu@FreeBSD.org> Fix LINT: Add backlight to NOTES
onf/NOTES
69badeb2fb3ae160c681568b2d7b3e013c4a1be5 01-Oct-2020 emaste <emaste@FreeBSD.org> Add cd device to arm64 GENERIC

Big-iron arm64 machines might have a CD, possibly provided by some IPMI
emulation.

Reported by: scottph
onf/GENERIC
2c90fae07c16ea847fe3f962cd39492cebbc3c27 29-Sep-2020 mhorne <mhorne@FreeBSD.org> arm64: set the correct HWCAP

This appears to be a typo. The AdvSIMD field encodes support for
half-precision floating point SIMD instructions, which corresponds to
HWCAP_ASIMDHP, not HWCAP_ASIMDDP.

MFC after: 3 days
Sponsored by: The FreeBSD Foundation
rm64/identcpu.c
8dd571af49b59f3aaf54dcb2fe1e01e416541f37 28-Sep-2020 mmel <mmel@FreeBSD.org> Fix booting arm64 EFI with LINUX_BOOT_ABI enabled.
Use address of the pointer passed to kernel to determine whether the pointer
is a FDT block (physical address) or a module pointer (virtual kernel address).
This fragment was supposed to be committed before r366196, but I accidentally
skipped it in a patch series.

Reported by: bz
rm64/locore.S
5fcaff9445045ecff2813c72a36282e4f940cd21 27-Sep-2020 trasz <trasz@FreeBSD.org> Get rid of sa->narg. It serves no purpose; use sa->callp->sy_narg instead.

Reviewed by: kib
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26458
rm64/elf32_machdep.c
rm64/trap.c
loudabi32/cloudabi32_sysvec.c
loudabi64/cloudabi64_sysvec.c
nclude/proc.h
inux/linux_sysvec.c
09ef42aec356b5ef8a7ebb4360f817dee1105b6c 27-Sep-2020 mmel <mmel@FreeBSD.org> Check the result of the function, not the pointer to it.
oriq/clk/qoriq_clkgen.c
2d46e857d345071f589d5d7bd9f5ae43e2bffb8a 27-Sep-2020 mmel <mmel@FreeBSD.org> Add LINUX_BOOT_ABI back to arm64 GENERIC kernel.

It was removed in r355289 but forgot to return it back when new u-boot booti
support was committed. Although booti is not the preferred method of
booting the kernel, it is very useful for the initial phase of porting
FreeBSD to a new platform or booting the kernel on various embedded boards
in an industrial environment.
onf/GENERIC
1623e08904b49ca55496ee7f60028aa7c4255887 27-Sep-2020 mmel <mmel@FreeBSD.org> Reapply r366193 with proper commit log.

Don't map same physical memory multiple times with different cache attributes.
This is explicitly stated as architectural undefined behavior, leading to
coherency issues sooner or later.
rm64/locore.S
6e0392a0fadb49dda6a822b46bd3629a515b2522 27-Sep-2020 mmel <mmel@FreeBSD.org> Revert r366193, it was committed with unsaved commit log.
rm64/locore.S
ede14d4f0bc2f1d339642e27bbc5b9a77ee2011f 27-Sep-2020 mmel <mmel@FreeBSD.org> Don't map same physical memory multiple times with different cache attributes.
This is explicitly stated as architectural undefined behavior, leadint to
coherencz issues sonner or later.
rm64/locore.S
caf553399ca51efb4abdb439c3df8e845ba6c312 27-Sep-2020 mmel <mmel@FreeBSD.org> Don't try to print EFI memeory map if it doesn't exist.

MFC after: 1 week
rm64/machdep.c
96407d0cdcc3fa4007d5aea02e623aea54b8bf68 25-Sep-2020 trasz <trasz@FreeBSD.org> Regen after r366145.

Sponsored by: DARPA
inux/linux_proto.h
inux/linux_sysent.c
40406f53607c243239b380c7f1a9849a66307271 24-Sep-2020 andrew <andrew@FreeBSD.org> Clean up the arm64 bus_dma_run_filter

- We can exit the loop as soon as the filter check passes.
- The alignment check has already passed so there is no need to also run
it here.

Sponsored by: Innovate UK
rm64/busdma_machdep.c
43a5ac8ba6e47eccf1677e8d2760962e0ee2ed09 24-Sep-2020 andrew <andrew@FreeBSD.org> Ensure arm64 DMA alignment is passed from parents to children

This ensures the alignment check will take these alignments into account.

Sponsored by: Innovate UK
rm64/busdma_machdep.c
bba7c438c3fefcefabc8691af51a97e94206f61f 24-Sep-2020 andrew <andrew@FreeBSD.org> Bounce in more cases in the arm64 busdma

We need to use a bounce buffer when the memory we are operating on is not
aligned to a cacheline, and not aligned to the maps alignment.

The former is to stop other threads from dirtying the cacheline while we
are performing DMA operations with it. The latter is to check memory
passed in by a driver is correctly aligned for the device.

Reviewed by: mmel
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D26496
rm64/busdma_bounce.c
60d42ebd226ff61148fdf9dfb4959a5b205b8a90 24-Sep-2020 andrew <andrew@FreeBSD.org> Ensure we always align and size arm64 busdma allocations to a cacheline

This will ensure nothing modifies the cacheline while DMA is in progress
so we won't need to bounce the data.

Reviewed by: mmel
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D26495
rm64/busdma_bounce.c
6a1bec1256a2df374faa7fa304835b08795396f5 24-Sep-2020 andrew <andrew@FreeBSD.org> Add a coherent flag on the arm64 dma map struct

Use it to decide if we can skip cache management.

While here remove the DMAMAP_COULD_BOUNCE flag as it's unneeded.

Reviewed by: mmel
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D26494
rm64/busdma_bounce.c
cf17917d836e54715c8cb8465beff24ef047ab24 24-Sep-2020 andrew <andrew@FreeBSD.org> Add bounce helpers to the arm64 busdma

Add helper functions to the arm64 busdma for common cases of checking if
we may need to bounce, and if we must bounce for a given address.

These will be expanded later as we handle cache-misaligned memory.

Reported by: mmel
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D26493
rm64/busdma_bounce.c
11e8e14a4ad80186f5564c3d2076998e3440784f 23-Sep-2020 markj <markj@FreeBSD.org> Add a vmparam.h constant indicating pmap support for large pages.

Enable SHM_LARGEPAGE support on arm64.

Reviewed by: alc, kib
Sponsored by: Juniper Networks, Inc., Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D26467
nclude/vmparam.h
a62800de2d5641a6fb8673e0b5c1aeff6d804253 23-Sep-2020 markj <markj@FreeBSD.org> Add largepage support to the arm64 pmap.

Reviewed by: alc, kib
Sponsored by: Juniper Networks, Inc., Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D26466
rm64/pmap.c
nclude/param.h
977a449bfd49e4d08d78f8af5ad178eb1b168fe4 21-Sep-2020 scottph <scottph@FreeBSD.org> arm64/pmap: Sparsify pv_table

Reviewed by: markj, kib
Approved by: scottl (implicit)
MFC after: 1 week
Sponsored by: Ampere Computing, Inc.
Differential Revision: https://reviews.freebsd.org/D26132
rm64/pmap.c
1f181d2512f60098ed7cde3326efb9a745de419d 21-Sep-2020 scottph <scottph@FreeBSD.org> Sparsify the vm_page_dump bitmap

On Ampere Altra systems, the sparse population of RAM within the
physical address space causes the vm_page_dump bitmap to be much
larger than necessary, increasing the size from ~8 Mib to > 2 Gib
(and overflowing `int` for the size).

Changing the page dump bitmap also changes the minidump file
format, so changes are also necessary in libkvm.

Reviewed by: jhb
Approved by: scottl (implicit)
MFC after: 1 week
Sponsored by: Ampere Computing, Inc.
Differential Revision: https://reviews.freebsd.org/D26131
rm64/minidump_machdep.c
nclude/minidump.h
f0deb41a04afb8b7457e115376b532d860cbe1ad 21-Sep-2020 scottph <scottph@FreeBSD.org> Move vm_page_dump bitset array definition to MI code

These definitions were repeated by all architectures, with small
variations. Consolidate the common definitons in machine
independent code and use bitset(9) macros for manipulation. Many
opportunities for deduplication remain in the machine dependent
minidump logic. The only intended functional change is increasing
the bit index type to vm_pindex_t, allowing the indexing of pages
with address of 8 TiB and greater.

Reviewed by: kib, markj
Approved by: scottl (implicit)
MFC after: 1 week
Sponsored by: Ampere Computing, Inc.
Differential Revision: https://reviews.freebsd.org/D26129
rm64/minidump_machdep.c
nclude/md_var.h
nclude/vmparam.h
002066e7a8cdda192818f5292d245df95a8637a0 21-Sep-2020 markj <markj@FreeBSD.org> Weaken assertions in pmap_l1_to_l2() and pmap_l2_to_l3().

pmap_update_entry() will temporarily clear the valid bit of page table
entries in order to satisfy the arm64 pmap's break-before-make
constraint. pmap_kextract() may operate concurrently on kernel page
table pages, introducing windows where the assertions added in r365879
may fail incorrectly since they implicitly assert that the valid bit is
set. Modify the assertions to handle this.

Reviewed by: andrew, mmel (previous version)
Reviewed by: alc, kib
Reported by: mmel, scottph
MFC with: r365879
rm64/pmap.c
2513d3914cd6d1ef186e3d28ceaa08bfc3b65480 20-Sep-2020 mmel <mmel@FreeBSD.org> Adjust DMA alignment for USB stack.
It should be at least as large as the maximum value of caheline size
for currently known CPUs.

MFC after: 2 weeks
onf/GENERIC
f0b88592cbf55481781c2c9615d8e3ab9d11774a 19-Sep-2020 mmel <mmel@FreeBSD.org> Implement workaround for broken access to configuration space.
Due to a HW bug in the RockChip PCIe implementation, attempting to access
a non-existent register in the configuration space will throw an exception.
Use new bus functions bus_peek() and bus_poke() to overcomme this limitation.
ockchip/rk_pcie.c
269d6a3111c43244daf123478a61d5c9564124c1 19-Sep-2020 mmel <mmel@FreeBSD.org> Add NetBSD compatible bus_space_peek_N() and bus_space_poke_N() functions.
One problem with the bus_space_read_N() and bus_space_write_N() family of
functions is that they provide no protection against exceptions which can
occur when no physical hardware or device responds to the read or write
cycles. In such a situation, the system typically would panic due to a
kernel-mode bus error. The bus_space_peek_N() and bus_space_poke_N() family
of functions provide a mechanism to handle these exceptions gracefully
without the risk of crashing the system.

Typical example is access to PCI(e) configuration space in bus enumeration
function on badly implemented PCI(e) root complexes (RK3399 or Neoverse
N1 N1SDP and/or access to PCI(e) register when device is in deep sleep state.

This commit adds a real implementation for arm64 only. The remaining
architectures have bus_space_peek()/bus_space_poke() emulated by using
bus_space_read()/bus_space_write() (without exception handling).

MFC after: 1 month
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D25371
rm64/bus_machdep.c
rm64/bus_space_asm.S
rm64/trap.c
nclude/bus.h
nclude/md_var.h
c6ab1f6e61cc58b3da568a8e616e98aabdef8d54 18-Sep-2020 markj <markj@FreeBSD.org> Assert we are not traversing through superpages in the arm64 pmap.

Reviewed by: alc, andrew
MFC after: 1 week
Sponsored by: Juniper Networks, Inc., Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D26465
rm64/pmap.c
371726d1752f5ce493011c64efd99ea737a379b2 17-Sep-2020 trasz <trasz@FreeBSD.org> Get rid of sv_errtbl and SV_ABI_ERRNO().

Reviewed by: kib
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26388
rm64/elf32_machdep.c
rm64/elf_machdep.c
inux/linux_sysvec.c
9e7fe3bb3320ce49ab4a72598bc7e6db9ae678a7 15-Sep-2020 trasz <trasz@FreeBSD.org> Move SV_ABI_ERRNO translation into linux-specific code, to simplify
the syscall path and declutter it a bit. No functional changes intended.

Reviewed by: kib (earlier version)
MFC after: 2 weeks
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26378
rm64/vm_machdep.c
inux/linux_sysvec.c
85867877360a47d52cc52a01ca5101611987f59d 15-Sep-2020 andrew <andrew@FreeBSD.org> Use ATTR_DEFAULT in the arm64 locore.S

We can use ATTR_DEFAULT directly in locore.S as it fits within an orr
instruction operand.

Sponsored by: Innovate UK
rm64/locore.S
c9d5bb55c0271ae672779416f8bb4d35914db7bf 14-Sep-2020 scottl <scottl@FreeBSD.org> Refine the busdma template interface. Provide tools for filling in fields
that can be extended, but also ensure compile-time type checking. Refactor
common code out of arch-specific implementations. Move the mpr and mps
drivers to this new API. The template type remains visible to the consumer
so that it can be allocated on the stack, but should be considered opaque.
rm64/busdma_machdep.c
d0a461d47027915b2da5dfeb1fd74c352e6d0b2e 10-Sep-2020 andrew <andrew@FreeBSD.org> Move the pl061 acpi attachment earlier

As the pl061 driver can be an interrupt controller attach it earlier in the
boot so other drivers can use it.

Use a new GPIO xref to not conflict with the existing root interrupt
controller.

Sponsored by: Innovate UK
nclude/intr.h
5740cdb43735343554e05230508f2aa859ba14c1 08-Sep-2020 mhorne <mhorne@FreeBSD.org> arm64: export new HWCAP features

Expose some of the new HWCAP features added in r65304. This includes the
addition of elf_hwcap2 into the sysvec, and a separate function to parse
for those features.

This only exposes features which require no further configuration, e.g.
indicating the presence of certain instructions. Larger features (SVE)
will not be advertised until we actually support them. The exact list of
features/extensions this patch exposes is:
- ARMv8.0-DGH
- ARMv8.0-SB
- ARMv8.2-BF16
- ARMv8.2-DCCVADP
- ARMv8.2-I8MM
- ARMv8.4-LRCPC
- ARMv8.5-CondM
- ARMv8.5-FRINT
- ARMv8.5-RNG
- PSTATE.SSBS

While here, annotate elf_hwcap and elf_hwcap2 as __read_frequently, and
move the declarations to the machine/md_var.h header.

Submitted by: mikael@ (D22314 portion)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26031
Differential Revision: https://reviews.freebsd.org/D22314
rm64/elf_machdep.c
rm64/identcpu.c
nclude/md_var.h
b571f343ee35a4ae9d92ae83193d11e720fa55f6 08-Sep-2020 mhorne <mhorne@FreeBSD.org> arm64: fix incorrect HWCAP definitions

FreeBSD exports CPU features as bits in the AT_HWCAP and AT_HWCAP2
vectors via elf_aux_info(3). This interface is similar to getauxval(3)
on Linux, and for simplicity to consumers we try to maintain an
identical set of feature flags on arm64.

The first batch of AT_HWCAP flags were added in r350166, corresponding
to definitions that already existed in Linux. Unfortunately, one flag
was missed, and a portion of the values are shifted one bit to the right
as a result.

Add the missing definition for HWCAP_ASIMDHP, and adjust the affected
values to match their Linux counterparts.

Although this is an ABI-breaking change, there is no plan to provide
compat code for old binaries. An audit of our ports tree and other
software via Debian code search indicates that there are not yet any
consumers of this interface for FreeBSD/arm64.

Bump __FreeBSD_version to be on the safe side, in case compat code needs
to be added in the future.

Reviewed by: emaste, manu
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26329
nclude/elf.h
137f829c18651ac64aeacdfa7541dc3aafc9bb9a 08-Sep-2020 andrew <andrew@FreeBSD.org> Add a GPIO driver for the Arm pl061 controller

A PL061 is a simple 8 pin GPIO controller. This GPIO device is used to
signal an internal request for shutdown on some virtual machines including
Arm-based Amazon EC2 instances.

Submitted by: Ali Saidi <alisaidi_amazon.com> (previouss version)
Reviewed by: Ali Saidi, manu
Differential Revision: https://reviews.freebsd.org/D24065
onf/GENERIC
21f5fb2cdcf2395f6149024e4755b6460579206a 07-Sep-2020 avg <avg@FreeBSD.org> Allwinner USB DRD support (musb_otg)

Allwinner USB DRD is based on the Mentor USB OTG controller, with a
different register layout and a few missing registers.

The code is by Andrew Turner (andrew).

Reviewed by: hselasky, manu
Obtained from: andrew
MFC after: 5 weeks
Differential Revision: https://reviews.freebsd.org/D5881
onf/GENERIC
45cd82112f9912fbd821ab2748e9fd266d2198b0 03-Sep-2020 mhorne <mhorne@FreeBSD.org> arm64: update the set of HWCAP definitions

This is in sync with what is defined for Linux 5.8. Note that all bits
in HWCAP are exhausted, and HWCAP2 has been added.

This also revealed an error in some of the existing definitions. We are
missing HWCAP_ASIMDHP, and as a result a portion of the HWCAP values are
shifted right by one bit. This will be fixed in an upcoming change, but
the values being added now are compatible with what Linux defines.

Reviewed by: emaste, markj, manu
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26030
nclude/elf.h
5929e8a15aab6c2602d5d8a7b5f12b1c1d14cbe8 03-Sep-2020 andrew <andrew@FreeBSD.org> Switch to an empty ttbr0 pagetable when the MMU is enabled

We don't need these pagetables after the early boot. Remove the chance we
write to memory we didn't expect to and remove architectural undefined
behaviour.

Reviewed by: alc (earlier version), mmel
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D22606
rm64/locore.S
rm64/machdep.c
nclude/machdep.h
06b00ceaa914a3907e4e27bad924f44612bae1d7 02-Sep-2020 markj <markj@FreeBSD.org> Include the psind in data returned by mincore(2).

Currently we use a single bit to indicate whether the virtual page is
part of a superpage. To support a forthcoming implementation of
non-transparent 1GB superpages, it is useful to provide more detailed
information about large page sizes.

The change converts MINCORE_SUPER into a mask for MINCORE_PSIND(psind)
values, indicating a mapping of size psind, where psind is an index into
the pagesizes array returned by getpagesizes(3), which in turn comes
from the hw.pagesizes sysctl. MINCORE_PSIND(1) is equal to the old
value of MINCORE_SUPER.

For now, two bits are used to record the page size, permitting values
of MAXPAGESIZES up to 4.

Reviewed by: alc, kib
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D26238
rm64/pmap.c
97764532b89054bbc7a4dc32fb58ced68595d370 02-Sep-2020 markj <markj@FreeBSD.org> Add the MEM_EXTRACT_PADDR ioctl to /dev/mem.

This allows privileged userspace processes to find information about the
physical page backing a given mapping. It is useful in applications
such as DPDK which perform some of their own memory management.

Reviewed by: kib, jhb (previous version)
MFC after: 2 weeks
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara Inc.
Differential Revision: https://reviews.freebsd.org/D26237
rm64/mem.c
nclude/memdev.h
3e310e90ef137a8e39b140aeb9a4dd14e461282e 02-Sep-2020 andrew <andrew@FreeBSD.org> Partially revert r365069.

This whitespace was intentionally added to help differentiate the different
register groups within this file.

While here add missing whitespace from earlier in the file,

Sponsored by: Innovate UK
rm64/identcpu.c
ce768ede1110bc282696c9865b06ca826dab0432 01-Sep-2020 mjg <mjg@FreeBSD.org> arm64: clean up empty lines in .c and .h files
rm64/busdma_bounce.c
rm64/freebsd32_machdep.c
rm64/gic_v3_acpi.c
rm64/gic_v3_var.h
rm64/gicv3_its.c
rm64/identcpu.c
rm64/machdep.c
rm64/machdep_boot.c
rm64/nexus.c
rm64/pmap.c
rm64/sys_machdep.c
rm64/vfp.c
roadcom/genet/if_genet.c
avium/thunder_pcie_common.c
avium/thunder_pcie_fdt.c
avium/thunder_pcie_pem.c
oresight/coresight_etm4x.c
oresight/coresight_fdt.c
reescale/imx/clk/imx_clk_mux.c
reescale/imx/imx7gpc.c
reescale/imx/imx8mq_ccm.c
nclude/acpica_machdep.h
nclude/armreg.h
nclude/bus.h
nclude/cpu.h
nclude/endian.h
nclude/ieeefp.h
nclude/ucontext.h
nclude/vfp.h
inux/linux_proto.h
oriq/clk/qoriq_clk_pll.h
oriq/ls1046_gpio.c
ockchip/clk/rk3399_cru.c
ockchip/clk/rk3399_pmucru.c
ockchip/clk/rk_clk_fract.c
ockchip/clk/rk_clk_mux.c
ockchip/rk805reg.h
ockchip/rk_dwc3.c
ockchip/rk_i2c.c
ockchip/rk_pcie.c
ockchip/rk_pcie_phy.c
ockchip/rk_pinctrl.c
ockchip/rk_tsadc.c
fa3f07d38ef4d5dbe70f54c75b57f32353da4abd 01-Sep-2020 andrew <andrew@FreeBSD.org> Ensure the tlbi has completed before setting SCTLR

When enabling the MMU on arm64 we need to ensure the tlb invalidation has
completed before setting the enable bit in the SCTLR register.

Reported by: alc
Sponsored by: Innovate UK
rm64/locore.S
b42c826a95c14d0893d6e5126209e03bb944f429 01-Sep-2020 andrew <andrew@FreeBSD.org> Support stage 2 arm64 pmap in more places

Add support for stage 2 pmap to pmap_pte_dirty, pmap_release, and more
of pmap_enter. This adds support in all placess I have hit while testing
bhyve ehile faulting pages in as needed.

Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D26065
rm64/pmap.c
e1e53035f7a31d3e842299bc2d3f4bdc55468cd2 28-Aug-2020 mmacy <mmacy@FreeBSD.org> ZFS: add to arm64 NOTES to minimize potential for missing symbols
onf/NOTES
d7704f9e75aafd312a3a1f03835b9b5ff92a3436 26-Aug-2020 scottph <scottph@FreeBSD.org> arm64: Increase dmap size to 95 TiB

The Ampere Altra has physical memory populated sparsely within the
physical address space. Increase the size of the dmap to cover all
physical memory.

Reviewed by: andrew
Approved by: scottl (implicit)
MFC after: 1 week
Sponsored by: Ampere Computing, Inc.
Differential Revision: https://reviews.freebsd.org/D26134
nclude/vmparam.h
4315b294de5493b6349cf3ef4151a94464f55419 26-Aug-2020 scottph <scottph@FreeBSD.org> arm64/acpi: Give the real PA limit to ACPI

Read PA bits from ID_AA64MMFR0_EL1.PARange.

Reviewed by: andrew, markj
Approved by: scottl (implicit)
MFC after: 1 week
Sponsored by: Ampere Computing, Inc.
Differential Revision: https://reviews.freebsd.org/D26133
cpica/acpi_machdep.c
rm64/identcpu.c
b56073a89e765faa5b4281ab83ce1d3fa3dffa1d 26-Aug-2020 scottph <scottph@FreeBSD.org> arm64: Make local stores observable before sending IPIs

Add a synchronizing instruction to flush and wait until the local
CPU's writes are observable to other CPUs before sending IPIs.

This fixes an issue where recipient CPUs doing a rendezvous could
enter the rendezvous handling code before the initiator's writes
to the smp_rv_* variables were visible. This manifested as a
system hang, where a single CPU's increment of smp_rv_waiters[0]
actually happened "before" the initiator's zeroing of that field,
so all CPUs were stuck with the field appearing to be at
ncpus - 1.

Reviewed by: andrew, markj
Approved by: scottl (implicit)
MFC after: 1 week
Sponsored by: Ampere Computing, Inc.
Differential Revision: https://reviews.freebsd.org/D25798
rm64/mp_machdep.c
3dc95128fb041e43a51927902622286189c345e4 23-Aug-2020 kib <kib@FreeBSD.org> Pass pointers to info parsed from notes, to brandinfo->header_supported filter.

Currently, we parse notes for the values of ELF FreeBSD feature flags
and osrel. Knowing these values, or knowing that image does not carry
the note if pointers are NULL, is useful to decide which ABI variant
(brand) we want to activate for the image.

Right now this is only a plumbing change

Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D25273
rm64/elf32_machdep.c
49a56a02be7b5b2d229378eee22830ac21b71efd 23-Aug-2020 kib <kib@FreeBSD.org> Style.
rm64/elf32_machdep.c
8513d6ed1456c4446238df8a49b7695c477d30ec 22-Aug-2020 emaste <emaste@FreeBSD.org> acpi_iort: fix mapping end calculation

According to the ARM Design Document "IO Remapping Table Platform"
(DEN 0049D), the "Number of IDs" field of the ID mapping format means
"The number of IDs in the range minus one".

Submitted by: Greg V <greg@unrelenting.technology>
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D25179
cpica/acpi_iort.c
e67784058a4458e1b70c256a32298374e86c9d12 20-Aug-2020 markj <markj@FreeBSD.org> Remove an unused parameter from map_table().

MFC after: 1 week
Sponsored by: The FreeBSD Foundation
cpica/acpi_machdep.c
2de558f01eb520839faaa70be6af16410e5dbcca 19-Aug-2020 mav <mav@FreeBSD.org> Remove some noisy ACPI tables messages from verbose dmesg.

Those messages were printed hundreds of times during boot, often multiple
times for each table. We already print information about the tables in
more organized form once to not duplicate it when random ACPI drivers are
attaching.

MFC after: 1 week
cpica/acpi_machdep.c
3501867eb29102709ef9ea10cffa124f61b33247 18-Aug-2020 mjg <mjg@FreeBSD.org> linux: add sysctl compat.linux.use_emul_path

This is a step towards facilitating jails with only Linux binaries.
Supporting emul_path adds path lookups which are completely spurious
if the binary at hand runs in a Linux-based root directory.

It defaults to on (== current behavior).

make -C /root/linux-5.3-rc8 -s -j 1 bzImage:

use_emul_path=1: 101.65s user 68.68s system 100% cpu 2:49.62 total
use_emul_path=0: 101.41s user 64.32s system 100% cpu 2:45.02 total
inux/linux_machdep.c
b15bb282a592da78223c1e5da100e5612ee98353 18-Aug-2020 mjg <mjg@FreeBSD.org> linux: consistently use LFREEPATH instead of open-coding it
inux/linux_machdep.c
f59bf9b56477346816fc81ae6a4c19191e7cfef9 15-Aug-2020 mhorne <mhorne@FreeBSD.org> arm64: parse HWCAP values using user_cpu_desc

The hard work of parsing fields per-CPU, handling heterogeneous
features, and excluding features from userspace is already done by
update_special_regs. We can build our set of HWCAPs from the result.

This exposed a small bug in update_special_regs, in which the
generated bitmask was not wide enough, and as a result some bits
weren't being exposed in user_cpu_desc. Fix this.

While here, adjust some formatting.

Reviewed by: andrew
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26069
rm64/identcpu.c
889e26f81f781706f7579afd6f54ada7be7e46fc 15-Aug-2020 mhorne <mhorne@FreeBSD.org> arm64: update instruction set attribute register definitions

This adds definitions for the latest additions to the AA64ISAR[01] ID
registers. This brings these registers in sync with ARMv8.6 initial spec
release.

An future change will parse many of these fields for HWCAP features.

Reviewed by: andrew, manu, markj (all previous versions)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26029
rm64/identcpu.c
nclude/armreg.h
b177c8876546d54f93c61cafffeb60cfc1aaa26a 12-Aug-2020 andrew <andrew@FreeBSD.org> Add support for Cortex-A76/Neoverse-N1 to hwpmc

This adds support for the Cortex-A76 and Neoverse-N1 PMU counters to pmc.

While here add more PMCR_IDCODE values and check the implementers code is
correct before setting the PMU type.

Reviewed by: bz, emaste (looks reasonable to me)
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D25959
nclude/armreg.h
2bae8d09b3c0d9923660500f8f8ce6f9c4e58f4e 10-Aug-2020 gonzo <gonzo@FreeBSD.org> Improve Rockchip's integration of if_dwc

- Do not rely on U-Boot for clocks configuration, enable and set frequencies
in the driver's attach method.
- Adjust MAC settings according to detected linespeed on RK3399 and RK3328.
- Add support for RMII PHY mode on RK3328.

Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D26006
ockchip/if_dwc_rk.c
6ceab08125b5af6b842005c9ffcbf42929e40b24 05-Aug-2020 mjg <mjg@FreeBSD.org> arm64: fix uintfptr_t

Fixes compilation after r363932
nclude/profile.h
e87c28bd3945c36be54b7d4c6f9094a639086db2 05-Aug-2020 gonzo <gonzo@FreeBSD.org> Add clocks for ethernet controllers on RK3328

Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D25918
ockchip/clk/rk3328_cru.c
15e7eff0a0a8fec3b20c886ca6da80bad3f67fac 05-Aug-2020 gonzo <gonzo@FreeBSD.org> Add flag for SYSCON-controlled clocks on Rockhip platform

Ethernet clocks on RK3328 are controlled by SYSCON registers, so add
RK_CLK_COMPOSITE_GRF flag to indicate that clock node should access grf
registers instead of CRU's

Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D25918
ockchip/clk/rk_clk_composite.c
ockchip/clk/rk_clk_composite.h
4320555f0152f968795de29af1442b09b37eba58 05-Aug-2020 andrew <andrew@FreeBSD.org> Add DDB_CTF to the arm64 and riscv kernel configs

This allows DTrace fbt probes to find arguments.

Sponsored by: Innovate UK
onf/GENERIC
92d4928dac80c79b4c2f675d0fa3e833f01448a5 27-Jul-2020 mav <mav@FreeBSD.org> Add initial driver for ACPI Platform Error Interfaces.

APEI allows platform to report different kinds of errors to OS in several
ways. We've found that Supermicro X10/X11 motherboards report PCIe errors
appearing on hot-unplug via this interface using NMI. Without respective
driver it ended up in kernel panic without any additional information.

This driver introduces support for the APEI Generic Hardware Error Source
reporting via NMI, SCI or polling. It decodes the reported errors and
either pass them to pci(4) for processing or just logs otherwise. Errors
marked as fatal still end up in kernel panic, but some more informative.

When somebody get to native PCIe AER support implementation both of the
reporting mechanisms should get common error recovery code. Since in our
case errors happen when the device is already gone, there is nothing to
recover, so the code just clears the error statuses, practically ignoring
the otherwise destructive NMIs in nicer way.

MFC after: 2 weeks
Relnotes: yes
Sponsored by: iXsystems, Inc.
rm64/machdep.c
nclude/acpica_machdep.h
265ebdf0714e1ae53c3ab622c701e30a36942e18 24-Jul-2020 arichardson <arichardson@FreeBSD.org> Include TMPFS in all the GENERIC kernel configs

Being able to use tmpfs without kernel modules is very useful when building
small MFS_ROOT kernels without a real file system.
Including TMPFS also matches arm/GENERIC and the MIPS std.MALTA configs.

Compiling TMPFS only adds 4 .c files so this should not make much of a
difference to NO_MODULES build times (as we do for our minimal RISC-V
images).

Reviewed By: br (earlier version for riscv), brooks, emaste
Differential Revision: https://reviews.freebsd.org/D25317
onf/GENERIC
a9c79eb484303cb02ea6a134ad0437d15ef17d17 23-Jul-2020 jhb <jhb@FreeBSD.org> Set si_trapno to the exception code from esr.

Reviewed by: kib
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D25771
rm64/trap.c
143dbd196464bde0a98b96215b98af98af59b60c 22-Jul-2020 jhb <jhb@FreeBSD.org> Add missing space after switch.

Reviewed by: br, emaste
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D25778
rm64/trap.c
b7f9188c1bd8fe3061caab6d5d6da0ecb7425772 18-Jul-2020 trasz <trasz@FreeBSD.org> Regen after r363304.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
inux/linux_proto.h
inux/linux_sysent.c
inux/linux_systrace_args.c
3e5218c3622c7c0955f8374027e9d0e0f92df2e4 18-Jul-2020 trasz <trasz@FreeBSD.org> Add a trivial linux(4) splice(2) implementation, which simply
returns EINVAL. Fixes grep (grep-3.1-2build1).

PR: kern/218699
Reported by: avos
Reviewed by: emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25636
inux/linux_dummy.c
inux/syscalls.master
e2a60fc8d06acf65c63b1435aa66908312ae3944 17-Jul-2020 br <br@FreeBSD.org> Add acpi_iort_map_pci_smmuv3().

This new function allows us to find the SMMU instance assigned
for a particular PCI RID.

Reviewed by: andrew
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D25687
cpica/acpi_iort.c
d1ddcf911cb844b4d2947e13337df0c86d5887ed 16-Jul-2020 markj <markj@FreeBSD.org> Switch from SCTP to SCTP_SUPPORT in GENERIC configs.

This removes SCTP from in-tree kernel configuration files. Now, SCTP
can be enabled by simply loading the module, as discussed on
freebsd-net@.

Reviewed by: tuexen
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25611
onf/GENERIC
194561dc8ec646977aabef1f6c29ecd21e3c361a 14-Jul-2020 andrew <andrew@FreeBSD.org> Print the arm64 registers in more exception handling panics

It can be useful to get a dump of all registers when investigating why we
received an exception that we are unable to handle. In these cases we
already call panic, however we don't always print the registers.

Add calls to print_registers and print esr and far when applicable.

Sponsored by: Innovate UK
rm64/trap.c
8989e07d36ab05ffe2d4f855c19eb31430dd33e1 14-Jul-2020 markj <markj@FreeBSD.org> Add a driver for the SafeXcel EIP-97.

The EIP-97 is a packet processing module found on the ESPRESSObin. This
commit adds a crypto(9) driver for the crypto and hash engine in this
device. An initial skeleton driver that could attach and submit
requests was written by loos and others at Netgate, and the driver was
finished by me.

Support for separate AAD and output buffers will be added in a separate
commit, to simplify merging to stable/12 (where those features don't
exist).

Reviewed by: gnn, jhb
Feedback from: andrew, cem, manu
MFC after: 1 week
Sponsored by: Rubicon Communications, LLC (Netgate)
Differential Revision: https://reviews.freebsd.org/D25417
onf/GENERIC
dba84dc5773f90c88275b7fff9b9d33be53df34d 13-Jul-2020 markj <markj@FreeBSD.org> Print arm64 physmem info during boot.

PR: 243682
Reviewed by: andrew, emaste
MFC after: 1 week
Event: July 2020 Bugathon
Differential Revision: https://reviews.freebsd.org/D25625
rm64/machdep.c
2a4046fc498a98ac31c14b040c72172bd30cc46e 05-Jul-2020 andrew <andrew@FreeBSD.org> Rerun kernel ifunc resolvers after all CPUs have started

On architectures that use RELA relocations it is safe to rerun the ifunc
resolvers on after all CPUs have started, but while they are sill parked.

On arm64 with big.LITTLE this is needed as some SoCs have shipped with
different ID register values the big and little clusters meaning we were
unable to rely on the register values from the boot CPU.

Add support for rerunning the resolvers on arm64 and amd64 as these are
both RELA using architectures.

Reviewed by: kib
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D25455
rm64/elf_machdep.c
rm64/machdep.c
33523e6a37c2ce5e2e688ceb3c2309068d3293b6 01-Jul-2020 andrew <andrew@FreeBSD.org> Read the CPU 0 arm64 ID registers early in initarm

We also update the kernel view early in the boot. This will allow the
use of the common kernel view in ifunc resolvers.

Sponsored by: Innovate UK
rm64/identcpu.c
rm64/machdep.c
rm64/mp_machdep.c
nclude/cpu.h
22822c3e20e4fbd27c215fca46e5788f8ab9fe18 01-Jul-2020 andrew <andrew@FreeBSD.org> Move ID reading signatures to a better header

The functions to read the common user and kernel ID registers should be
in cpu.h rather than undefined.h as they are related to CPU details and
used by undefined instruction handlers.

Sponsored by: Innovate UK
nclude/cpu.h
nclude/undefined.h
927b7f2b8d761a642c7cafea9ea95e762eb22bba 01-Jul-2020 andrew <andrew@FreeBSD.org> Read the arm64 ID registers earlier in the boot process.

Also move parsing the registers to just after the secondary CPUs have
started. This means the kernel register view from all CPUs is available
after the CPU SYSINITs have finished, e.g. for use by ifunc resolvers.

Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D25505
rm64/identcpu.c
rm64/mp_machdep.c
902a8d2c7ad508a58b51e9cb0df09e325781393a 01-Jul-2020 gonzo <gonzo@FreeBSD.org> Add i.MX 8M Quad support

- Add CCM driver and clocks implementations for i.MX 8M
- Add GPC driver for iMX8
- Add clock tree for i.MX 8M Quad
- Add clocks support and new compat strings (where required) for existing i.MX 6 UART, I2C, and GPIO drivers
- Enable aarch64-compatible drivers form i.MX 6 in arm64 GENERIC kernel config
- Add dtb/imx8 kernel module with DTBs for Nitrogen8M and iMX8MQ EVK

With this patch both Nitrogen8M and iMX8MQ EVK boot with NFS root up to multiuser login prompt

Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D25274
onf/GENERIC
reescale/imx/clk/imx_clk_composite.c
reescale/imx/clk/imx_clk_composite.h
reescale/imx/clk/imx_clk_frac_pll.c
reescale/imx/clk/imx_clk_frac_pll.h
reescale/imx/clk/imx_clk_gate.c
reescale/imx/clk/imx_clk_gate.h
reescale/imx/clk/imx_clk_mux.c
reescale/imx/clk/imx_clk_mux.h
reescale/imx/clk/imx_clk_sscg_pll.c
reescale/imx/clk/imx_clk_sscg_pll.h
reescale/imx/imx7gpc.c
reescale/imx/imx8mq_ccm.c
reescale/imx/imx8mq_ccm.h
reescale/imx/imx_ccm_clk.h
8103ba690af99456be0d79fcf83157da3dcad225 29-Jun-2020 br <br@FreeBSD.org> Coresight: provide device_attach method for FDT bus.

Sponsored by: DARPA, AFRL
oresight/coresight_funnel_fdt.c
oresight/coresight_replicator_fdt.c
4a05a28459980e638c393f85f2d3bccba0fcfc31 29-Jun-2020 andrew <andrew@FreeBSD.org> Fix the spelling of identify in the arm64 identcpu code

Sponsored by: Innovate UK
rm64/identcpu.c
3addfde2b0e393c2f0ed300048222deac6672ece 29-Jun-2020 andrew <andrew@FreeBSD.org> Create a kernel arm64 ID register view

In preparation for using ifuncs in the kernel is is useful to have a common
view of the arm64 ID registers across all CPUs. Add this and extract the
logic for finding the lower value of two fields to a new helper function.

Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D25463
rm64/identcpu.c
nclude/undefined.h
145ddba8bee2bb9e4affa5e3b9acdf21c6c7283d 29-Jun-2020 kevans <kevans@FreeBSD.org> linuxolator: implement memfd_create syscall

This effectively mirrors our libc implementation, but with minor fudging --
name needs to be copied in from userspace, so we just copy it straight into
stack-allocated memfd_name into the correct position rather than allocating
memory that needs to be cleaned up.

The sealing-related fcntl(2) commands, F_GET_SEALS and F_ADD_SEALS, have
also been implemented now that we support them.

Note that this implementation is still not quite at feature parity w.r.t.
the actual Linux version; some caveats, from my foggy memory:

- Need to implement SHM_GROW_ON_WRITE, default for memfd (in progress)
- LTP wants the memfd name exposed to fdescfs
- Linux allows open() of an fdescfs fd with O_TRUNC to truncate after dup.
(?)

Interested parties can install and run LTP from ports (devel/linux-ltp) to
confirm any fixes.

PR: 240874
Reviewed by: kib, trasz
Differential Revision: https://reviews.freebsd.org/D21845
inux/linux_dummy.c
7b80a32eb1e41dcf83d2b544866042d491a57039 28-Jun-2020 gonzo <gonzo@FreeBSD.org> Configure rx_delay/tx_delay values for RK3399/RK3328 GMAC

For 1000Mb mode to work reliably TX/RX delays need to be configured
between the TX/RX clock and the respective signals on the PHY
to compensate for differing trace lengths on the PCB.

Reviewed by: manu
MFC after: 1 week
ockchip/if_dwc_rk.c
1ee69c6890082e3ea16df85ff31c111a5528dacc 28-Jun-2020 andrew <andrew@FreeBSD.org> Use EFI memory map to determine attributes for Acpi mappings on arm64.

AcpiOsMapMemory is used for device memory when e.g. an _INI method wants
to access physical memory, however, aarch64 pmap_mapbios is hardcoded to
writeback. Search for the correct memory type to use in pmap_mapbios.

Submitted by: Greg V <greg_unrelenting.technology>
Differential Revision: https://reviews.freebsd.org/D25201
rm64/machdep.c
rm64/pmap.c
nclude/machdep.h
1b54e5a8061141ce1892a53a05d7445ff99904f7 27-Jun-2020 trasz <trasz@FreeBSD.org> Regen.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
inux/linux_proto.h
inux/linux_sysent.c
inux/linux_systrace_args.c
030afd142c60119df882288c1154d7369b79592a 27-Jun-2020 trasz <trasz@FreeBSD.org> Add proper types for linux message queue syscalls; mostly taken
from 32-bit Linuxulator.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25386
inux/linux.h
inux/syscalls.master
79cc6cb3f644b4f161fc3ec95f7fc483ad48c487 27-Jun-2020 trasz <trasz@FreeBSD.org> Add syscall definitions for linux xattr syscalls.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25387
inux/syscalls.master
950b3fd53713949dfc826d8c9dbd1a09e82ad6f3 27-Jun-2020 trasz <trasz@FreeBSD.org> Adjust types of linuxulator syscalls, to match include/linux/syscalls.h
in vanilla Linux git tree.

Reviewed by: markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25385
inux/syscalls.master
510fa23149357a0084ed4b7eaa0c18de2963e816 23-Jun-2020 emaste <emaste@FreeBSD.org> arm64 armreg.h: fix TCR_TBI1 definition

Submitted by: Greg V <greg@unrelenting.technology>
Differential Revision: https://reviews.freebsd.org/D25411
nclude/armreg.h
02bc014a200a5f6c4ee5885661625cd27d0849f5 21-Jun-2020 bdragon <bdragon@FreeBSD.org> [PowerPC] More relocation fixes

It turns out relocating the symbol table itself can cause issues, like fbt
crashing because it applies the offsets to the kernel twice.

This had been previously brought up in rS333447 when the stoffs hack was
added, but I had been unaware of this and reimplemented symtab relocation.

Instead of relocating the symbol table, keep track of the relocation base
in ddb, so the ddb symbols behave like the kernel linker-provided symbols.

This is intended to be NFC on platforms other than PowerPC, which do not
use fully relocatable kernels. (The relbase will always be 0)

* Remove the rest of the stoffs hack.
* Remove my half-baked displace_symbol_table() function.
* Extend ddb initialization to cope with having a relocation offset on the
kernel symbol table.
* Fix my kernel-as-initrd hack to work with booke64 by using a temporary
mapping to access the data.
* Fix another instance of __powerpc__ that is actually RELOCATABLE_KERNEL.
* Change the behavior or X_db_symbol_values to apply the relocation base
when updating valp, to match link_elf_symbol_values() behavior.

Reviewed by: jhibbits
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D25223
rm64/machdep_boot.c
7dac5ce5e017abf4e5de33121ac9e1da2158ceaa 20-Jun-2020 trasz <trasz@FreeBSD.org> Regen after r362440.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
inux/linux_proto.h
inux/linux_syscall.h
inux/linux_syscalls.c
inux/linux_sysent.c
inux/linux_systrace_args.c
d8f6e5f667a1da216d9997a246b8fd66af464d26 20-Jun-2020 trasz <trasz@FreeBSD.org> Add linux_madvise(2) instead of having Linux apps call the native
FreeBSD madvise(2) directly. While some of the flag values match,
most don't.

PR: kern/230160
Reported by: markj
Reviewed by: markj
Discussed with: brooks, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25272
inux/linux_machdep.c
inux/syscalls.master
3d7ca48f655f52e390fe275481b6ce69714ef379 19-Jun-2020 mmel <mmel@FreeBSD.org> Finish renaming in if_dwc.
By using DWC TRM terminology, normal descriptor format should be named
extended and alternate descriptor format should be named normal.

Should not been functional change.

MFC after: 4 weeks
ockchip/if_dwc_rk.c
ed9ac562f75b25f0da67ffde8a930ea30d078e84 19-Jun-2020 mmel <mmel@FreeBSD.org> Use native-sized accesses when accessing memory from kdb.
Not all MMIO mapped devices supports byte access.

MFC after: 4 weeks
rm64/db_interface.c
7604ae54d1710f953a62a7dbac46a4bd6f4f600c 18-Jun-2020 karels <karels@FreeBSD.org> Add support for bcm54213PE in brgphy.

This chip is used in the Rasperry Pi 4, and is supported by the if_genet
driver. Currently we use the ukphy mii driver, this patch switches over
to the brgphy mii driver instead. To support the rgmii-rxid phy mode,
which is now the default in the Linux dtb, we add support for clock
skewing.

These changes are taken from OpenBSD and NetBSD, except for the bailout
in brgphy_bcm54xx_clock_delay() in rgmii mode, which was found necessary
after testing.

Submitted by: Robert Crowston, crowston at protomail.com
Differential Revision: https://reviews.freebsd.org/D25251
roadcom/genet/if_genet.c
341beabe4f76e096321427e56e78afe2c1f1c2ae 17-Jun-2020 andrew <andrew@FreeBSD.org> Support pmap_extract_and_hold on arm64 stage 2 mappings

Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D24469
rm64/pmap.c
8ce15a8c3559bdd52ae005718214d0274e87db23 17-Jun-2020 br <br@FreeBSD.org> Complete the ACPI support for ARM Coresight:
o Parse the ACPI DSD (Device Specific Data) graph property and record
device connections.
o Split-out FDT support to a separate file.
o Get the corresponding (FDT/ACPI) Coresight platform data in
the device drivers.

Sponsored by: DARPA, AFRL
oresight/coresight.c
oresight/coresight.h
oresight/coresight_acpi.c
oresight/coresight_cmd.c
oresight/coresight_cpu_debug.c
oresight/coresight_etm4x.c
oresight/coresight_etm4x.h
oresight/coresight_etm4x_acpi.c
oresight/coresight_etm4x_fdt.c
oresight/coresight_fdt.c
oresight/coresight_funnel.c
oresight/coresight_funnel.h
oresight/coresight_funnel_acpi.c
oresight/coresight_funnel_fdt.c
oresight/coresight_replicator.c
oresight/coresight_replicator.h
oresight/coresight_replicator_acpi.c
oresight/coresight_replicator_fdt.c
oresight/coresight_tmc.c
oresight/coresight_tmc.h
oresight/coresight_tmc_acpi.c
oresight/coresight_tmc_fdt.c
aaac1b142fcb54afb57bf655da4d6482c76e251f 17-Jun-2020 andrew <andrew@FreeBSD.org> Add all the TCR_EL1 fields

These will be used when adding support for new Armv8 extensions.

Sponsored by: Innovate UK
nclude/armreg.h
85476c13784ee7bedab990906a7e1711afbfbe40 12-Jun-2020 br <br@FreeBSD.org> Coresight replicator:
o Add a header file;
o Split-out FDT attachment to a separate file;
o Add ACPI attachment.

Sponsored by: DARPA, AFRL
oresight/coresight_replicator.c
oresight/coresight_replicator.h
oresight/coresight_replicator_acpi.c
oresight/coresight_replicator_fdt.c
e2e0e65b410de9886622e086f1b2a5654196de05 12-Jun-2020 br <br@FreeBSD.org> ARM Coresight Trace Memory Controller (TMC):
o Split-out FDT attachment to a separate file;
o Add ACPI attachment.

Sponsored by: DARPA, AFRL
oresight/coresight_tmc.c
oresight/coresight_tmc.h
oresight/coresight_tmc_acpi.c
oresight/coresight_tmc_fdt.c
da219c81695dbc71d59d52dd80cfc43b91a00f57 12-Jun-2020 andrew <andrew@FreeBSD.org> Teach the arm64 vfp.h about struct thread.

Ensure struct thread is defined in vfp.h. In some cases it is not and stops
the kernel from building.

Sponsored by: Innovate UK
nclude/vfp.h
4fe68d15fec78d202109673fc92d7a6f7fe81854 11-Jun-2020 br <br@FreeBSD.org> Shorten the filename of the coresight replicator driver.

Sponsored by: DARPA, AFRL
oresight/coresight_dynamic_replicator.c
oresight/coresight_replicator.c
7714c166dc124f472124a03a2cd912e1e17a3d51 10-Jun-2020 br <br@FreeBSD.org> ARM Coresight Funnel device:
o Split-out FDT attachment to a separate file;
o Add ACPI attachment;
o Add support for the Static Funnel device.

Sponsored by: DARPA, AFRL
oresight/coresight_funnel.c
oresight/coresight_funnel.h
oresight/coresight_funnel_acpi.c
oresight/coresight_funnel_fdt.c
619482f1e757586976208b156d554f8c70f595a9 09-Jun-2020 br <br@FreeBSD.org> ARM Embedded Trace Macrocell v4.x driver:
o Split-out FDT attachment to a separate file;
o Add ACPI attachment.

Sponsored by: DARPA, AFRL
oresight/coresight_etm4x.c
oresight/coresight_etm4x.h
oresight/coresight_etm4x_acpi.c
oresight/coresight_etm4x_fdt.c
c8c25848151c4e087438b40e02959716bb7d70b9 09-Jun-2020 br <br@FreeBSD.org> Fix style: wrap long lines.

Sponsored by: DARPA, AFRL
oresight/coresight.c
606c623464ee83a82ab13c98a3c369a0597b96c4 09-Jun-2020 br <br@FreeBSD.org> Rename coresight drivers: use underscores in filenames.

Sponsored by: DARPA, AFRL
oresight/coresight-cmd.c
oresight/coresight-cpu-debug.c
oresight/coresight-dynamic-replicator.c
oresight/coresight-etm4x.c
oresight/coresight-etm4x.h
oresight/coresight-funnel.c
oresight/coresight-funnel.h
oresight/coresight-tmc.c
oresight/coresight-tmc.h
oresight/coresight_cmd.c
oresight/coresight_cpu_debug.c
oresight/coresight_dynamic_replicator.c
oresight/coresight_etm4x.c
oresight/coresight_etm4x.h
oresight/coresight_funnel.c
oresight/coresight_funnel.h
oresight/coresight_tmc.c
oresight/coresight_tmc.h
a9d9f7629ebb2711b537b783c160fdfdcc399eca 31-May-2020 mhorne <mhorne@FreeBSD.org> Remove remnant of arm's ELF trampoline

The trampoline code used for loading gzipped a.out kernels on arm was
removed in r350436. A portion of this code allowed for DDB to find the
symbol tables when booting without loader(8), and some of this was
untouched in the removal. Remove it now.

Differential Revision: https://reviews.freebsd.org/D24950
rm64/machdep_boot.c
82b2ae11e9d4ff281aaa679d6884ea087d829932 30-May-2020 karels <karels@FreeBSD.org> genet: workaround for problem with ICMPv6 echo replies

The ICMPv6 echo reply is constructed with the IPv6 header too close to
the beginning of a packet for an Ethernet header to be prepended, so we
end up with an mbuf containing just the Ethernet header. The GENET
controller doesn't seem to handle this, with or without transmit checksum
offload. At least until we have chip documentation, do a pullup to
satisfy the chip. Hopefully this can be fixed properly in the future.
roadcom/genet/if_genet.c
f9c09aa267cb6555d9454231cee4ee279d5ac565 30-May-2020 karels <karels@FreeBSD.org> genet: fix issues with transmit checksum offload

Fix problem with ICMP echo replies: check only deferred data checksum
flags, and not the received checksum status bits, when checking whether
a packet has a deferred checksum; otherwise echo replies are corrupted
because the received checksum status bits are still present.

Fix some unhandled cases in packet shuffling for checksum offload.
roadcom/genet/if_genet.c
b128a802d77ebc21f52cbbce5b089f9845f600ad 27-May-2020 andrew <andrew@FreeBSD.org> Support creating and using arm64 pmap at stage 2

Add minimal support for creating stage 2 IPA -> PA mappings. For this we
need to:

- Create a new vmid set to allocate a vmid for each Virtual Machine
- Add the missing stage 2 attributes
- Use these in pmap_enter to create a new mapping
- Handle stage 2 faults

The vmid set is based on the current asid set that was generalised in
r358328. It adds a function pointer for bhyve to use when the kernel needs
to reset the vmid set. This will need to call into EL2 and invalidate the
TLB.

The stage 2 attributes have been added. To simplify setting these fields
two new functions are added to get the memory type and protection fields.
These are slightly different on stage 1 and stage 2 tables. We then use
them in pmap_enter to set the new level 3 entry to be stored.

The D-cache on all entries is cleaned to the point of coherency. This is
to allow the data to be visible to the VM. To allow for userspace to load
code when creating a new executable entry an invalid entry is created. When
the VM tried to use it the I-cache is invalidated. As the D-cache has
already been cleaned this will ensure the I-cache is synchronised with the
D-cache.

When the hardware implements a VPIPT I-cache we need to either have the
correct VMID set or invalidate it from EL2. As the host kernel will have
the wrong VMID set we need to call into EL2 to clean it. For this a second
function pointer is added that is called when this invalidation is needed.

Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D23875
rm64/pmap.c
nclude/cpufunc.h
nclude/pcpu.h
nclude/pmap.h
nclude/pte.h
ba15208e9900da80bc253e960b2e2d6e3de42d98 26-May-2020 erj <erj@FreeBSD.org> ice(4): Introduce new driver for Intel E800 Ethernet controllers

The ice(4) driver is the driver for the Intel E8xx series Ethernet
controllers; currently with codenames Columbiaville and
Columbia Park.

These new controllers support 100G speeds, as well as introducing
more queues, better virtualization support, and more offload
capabilities. Future work will enable virtual functions (like
in ixl(4)) and the other functionality outlined above.

For full functionality, the kernel should be compiled with
"device ice_ddp" like in the amd64 NOTES file, and/or
ice_ddp_load="YES" should be added to /boot/loader.conf so that
the DDP package file included in this commit can be downloaded
to the adapter. Otherwise, the adapter will fall back to a single
queue mode with limited functionality.

A man page for this driver will be forthcoming.

MFC after: 1 month
Relnotes: yes
Sponsored by: Intel Corporation
Differential Revision: https://reviews.freebsd.org/D21959
onf/NOTES
6aef78af13c3cb5f2bdf0355e2479fd225db0a99 25-May-2020 cem <cem@FreeBSD.org> copystr(9): Move to deprecate (attempt #2)

This reapplies logical r360944 and r360946 (reverting r360955), with fixed
copystr() stand-in replacement macro. Eventually the goal is to convert
consumers and kill the macro, but for a first step it helps if the macro is
correct.

Prior commit message:

Unlike the other copy*() functions, it does not serve to copy from one
address space to another or protect against potential faults. It's just
an older incarnation of the now-more-common strlcpy().

Add a coccinelle script to tools/ which can be used to mechanically
convert existing instances where replacement with strlcpy is trivial.
In the two cases which matched, fuse_vfsops.c and union_vfsops.c, the
code was further refactored manually to simplify.

Replace the declaration of copystr() in systm.h with a small macro
wrapper around strlcpy (with correction from brooks@ -- thanks).

Remove N redundant MI implementations of copystr. For MIPS, this
entailed inlining the assembler copystr into the only consumer,
copyinstr, and making the latter a leaf function.

Reviewed by: jhb (earlier version)
Discussed with: brooks (thanks!)
Differential Revision: https://reviews.freebsd.org/D24672
rm64/copystr.c
82826ce70b79296d75877c0d4a4aa83fdc937c4f 25-May-2020 mw <mw@FreeBSD.org> Introduce VF610 I2C controller support.

NXP LS1046A contains I2C controller compatible with Vybrid VF610.
Existing Vybrid MVF600 driver can be used to support it. For that purpose
declare driver as ofw_iicbus and add methods associated with ofw_iicbus.

For VF610 add dynamic clock prescaler calculation using clock information
from clock driver and clock frequency requested in device tree.

On the occasion add detach function and add additional error handling
in i2c_attach function.

Submitted by: Dawid Gorecki <dgr@semihalf.com>
Reviewed by: manu
Obtained from: Semihalf
Sponsored by: Alstom Group
Differential Revision: https://reviews.freebsd.org/D24361
onf/GENERIC
8ab8ca1c2557cd1e005c434f08e8e540d1c53f16 25-May-2020 mw <mw@FreeBSD.org> Add GPIO support for QorIQ boards.

This patch adds a GPIO controller support targeted for NXP LS1046A
SoC. The driver implements the following features:
* setting direction of each pin (IN or OUT)
* setting the mode of output pins (PUSHPULL or OPENDRAIN)
* setting the state of each output pin (1 or 0)
* reading the state of each input pin (1 or 0)

Submitted by: Kamil Koczurek <kek@semihalf.com>
Dawid Gorecki <dgr@semihalf.com>
Reviewed by: manu
Obtained from: Semihalf
Sponsored by: Alstom Group
Differential Revision: https://reviews.freebsd.org/D24353
onf/GENERIC
oriq/ls1046_gpio.c
fea63adc7d4baa2881e17a5e9292cd41f10db8d0 25-May-2020 mw <mw@FreeBSD.org> Add LS1046A clockgen driver.

Driver provides probe and attach functions for LS1046A clockgen and passes
configuration information to QorIQ clockgen class. It may be used as
a reference implementation for different QorIQ clockgen devices.

Submitted by: Dawid Gorecki <dgr@semihalf.com>
Reviewed by: mmel, manu
Obtained from: Semihalf
Sponsored by: Alstom Group
Differential Revision: https://reviews.freebsd.org/D24352
onf/GENERIC
oriq/clk/ls1046a_clkgen.c
35a1c5000bfc73d813952f9577c20e722fdbf5b5 25-May-2020 mw <mw@FreeBSD.org> Add QorIQ platform clockgen driver.

This patch adds classes and functions that can be used with various NXP
QorIQ Layerscape SoCs.

As for the clock topology - there is single platform PLL, which supplies
clocks for the peripheral bus and additional PLLs for CPU cores. There
can be multiple core PLLs (For example - LS1046A has two PLLs - CGAPLL1
and CGAPLL2). Each PLL has fixed dividers on output. The core PLLs
are not accessible from dts.

This is a preparation patch for NXP LS1046A SoC support.

Submitted by: Dawid Gorecki <dgr@semihalf.com>
Reviewed by: mmel
Obtained from: Semihalf
Sponsored by: Alstom Group
Differential Revision: https://reviews.freebsd.org/D24351
oriq/clk/qoriq_clk_pll.c
oriq/clk/qoriq_clk_pll.h
oriq/clk/qoriq_clkgen.c
oriq/clk/qoriq_clkgen.h
9daa38339c8a65244f72afb730f1bee52975ae30 19-May-2020 andrew <andrew@FreeBSD.org> Stop performing a full icache sync when the DIC and IDC flags are set

The DIC and IDC bits in the CTR_EL0 register signal to the kernel when it
can relax the instruction cache synchronisation operations. The IDC bit
means we can relax cleaning the data cache to the point of unification
while the DIC bit means we don't need to invalidate the instruction cache
for data coherence. In both cases an appropriate barrier is still needed.

For now only implement the case where both bits are set, as is the case
on the Neoverse-N1 as used in the Amazon AWS Graviton 2 CPU. Note that
this behaviour is a optional on the N1 so we may later need to implement
only one or the other bit being set.

There is a tunable to disable each flag on boot.

Testing on a 4 core Graviton 2 instance found a significant improvement
in sys and real time when running "make buildkernel -j4", with no
significant difference in user time.

Reviewed by: markj
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D24853
rm64/cpufunc_asm.S
rm64/identcpu.c
nclude/cpufunc.h
0388e6d768d838ce622a4bdffa842c2bc465279b 19-May-2020 andrew <andrew@FreeBSD.org> Create MSI/MSI-X isrcs as needed in the GICv3 ITS driver

Previously we would create an isrc for each MSI/MSI-X interrupt. This
causes issues for other interrupt sources in the system, e.g. a GPIO
driver, as they may be unable to allocate interrupts. This works around
this by allocating the isrc only when needed.

Reported by: alisaidi@amazon.com
Reviewed by: mmel
Sponsored by: Innovaate UK
Differential Revision: https://reviews.freebsd.org/D24876
rm64/gicv3_its.c
90285c8b57b98194753a50f7323d44a0db66a7aa 15-May-2020 andrew <andrew@FreeBSD.org> Remove arm64_idcache_wbinv_range as it's unused.

Sponsored by: Innovate UK
rm64/cpufunc_asm.S
nclude/cpufunc.h
e395c5cb1f35c4b760baea21d50625aca9ae033a 14-May-2020 markj <markj@FreeBSD.org> Call acpi_pxm_set_proximity_info() slightly earlier on x86.

This function is responsible for setting pc_domain in each pcpu
structure. Call it from the main function that starts APs, rather than
a separate SYSINIT. This makes it easier to close the window where
UMA's per-CPU slab allocator may be called while pc_domain is
uninitialized. In particular, the allocator uses pc_domain to allocate
domain-local pages, so allocations before this point end up using domain
0 for everything.

Reviewed by: kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D24757
rm64/mp_machdep.c
e5a341694e236c5439eba2b5d9889ec17548e158 12-May-2020 andrew <andrew@FreeBSD.org> Fix the name reported when the core supports a 64-bit CCIDX
rm64/identcpu.c
225d1d90b81a222a8423f31e90f0966a6df8e332 12-May-2020 cem <cem@FreeBSD.org> Revert r360944 and r360946 until reported issues can be resolved

Reported by: cy
rm64/copystr.c
63b0acef4e4f60b7dc101b72a042f5d91c7f9f51 11-May-2020 cem <cem@FreeBSD.org> copystr(9): Move to deprecate [2/2]

Unlike the other copy*() functions, it does not serve to copy from one
address space to another or protect against potential faults. It's just
an older incarnation of the now-more-common strlcpy().

Add a coccinelle script to tools/ which can be used to mechanically
convert existing instances where replacement with strlcpy is trivial.
In the two cases which matched, fuse_vfsops.c and union_vfsops.c, the
code was further refactored manually to simplify.

Replace the declaration of copystr() in systm.h with a small macro
wrapper around strlcpy.

Remove N redundant MI implementations of copystr. For MIPS, this
entailed inlining the assembler copystr into the only consumer,
copyinstr, and making the latter a leaf function.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D24672
rm64/copystr.c
87f880a382dbbf8c57e6fc3dc1d802693f92a048 06-May-2020 markj <markj@FreeBSD.org> Simplify arm64's pmap_bootstrap() a bit.

locore constructs an L2 page mapping the kernel and preloaded data
starting a KERNBASE (the same as VM_MIN_KERNEL_ADDRESS on arm64).
initarm() and pmap_bootstrap() use the preloaded metadata to
tell it where it can start allocating from.

pmap_bootstrap() currently iterates over the L2 page to find the last
valid entry, but doesn't do anything with the result. Remove the loop
and zap some now-unused local variables.

MFC after: 2 weeks
Sponsored by: Juniper Networks, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D24559
rm64/pmap.c
8dfd2d145731bec5e3501b1859c2d34e3b017591 29-Apr-2020 mmel <mmel@FreeBSD.org> Export tracing facility of GIC500 ITS block.
Possibility of tracing of processing message based interrupts is very
useful for debugging of PCIe driver, mainly for its MSI part.

MFC after: 1 week
rm64/gic_v3_reg.h
rm64/gicv3_its.c
c14a3754670655e864b21bc4155a5a4b064cce2c 29-Apr-2020 mmel <mmel@FreeBSD.org> Don't try to re-initialize already preseted regulator.

Don't set initial voltage for regulators having their voltage already
in allowed range. As side effect of this change, we don't try to set
initial voltage for fixed voltage regulators - these don't have impemented
voltage set method so their initialization has always failed.

MFC after: 3 weeks
ockchip/rk805.c
db4e94860d0d5e98e77ee06e9b164d106655cc08 29-Apr-2020 mmel <mmel@FreeBSD.org> Multiple fixes for rockchip iodomain driver:
- always initialize selector of voltage signaling standard.
Various versions of U-boot leaves voltage signaling standard settings
for PMUIO2 domain in different state. Always initialize it
into expected state.
- start the driver as early as possible, the IO domains should be
initialized before other drivers are attached.
- rename RK3399 register to its name founds in TRM.

This is the second part of fixes for serial port corruption observed after
DT 5.6 import.

Reviewed by: manu
MFC after: 1 week
ockchip/rk_iodomain.c
1bb1283aafd8646b295cba06c0eb4936d20a8668 25-Apr-2020 manu <manu@FreeBSD.org> arm64: rockchip: rk805: Use a tailq for the attached regulator

Store the attached regulator in a tailq to later find them in ofw_map.
While here, do not attempt to attach a regulator without a name, a node
might exists but if it doesn't have a name the regulator is unused.

MFC after: 1 month
ockchip/rk805.c
c1dd8c2500572cefa8d17682820539b8ceeb8ec6 25-Apr-2020 mmel <mmel@FreeBSD.org> Reorder initialization steps for given pin.
If pin is switched from fixed function to GPIO, it should have prepared
direction, pull-up/down and default value before function gets switched.
Otherwise we may produce unwanted glitch on output pin.
Right order of drive strength settings is questionable, but I think that
is slightly safer to do it also before function switch.

This fixes serial port corruption observed after DT 5.6 import.

MFC after: 1 week
ockchip/rk_pinctrl.c
890e35e73a9634129a09527b07e8e8dbc85fd366 24-Apr-2020 markj <markj@FreeBSD.org> Fix handling of 1GB mappings in the arm64 minidump code.

For such mappings we need to dump 512 page table pages, not one, and
they need to be included in the pmap size recorded in the minidump
header.

MFC after: 2 weeks
Sponsored by: Juniper Networks, Klara Inc.
rm64/minidump_machdep.c
143b039d73f1b9c4bc5a61644594ac5701a9e6cb 24-Apr-2020 markj <markj@FreeBSD.org> Remove an obsolete TODO comment from several minidump implementations.

The comment referenced a non-existent function, and these minidump
implementations already buffer discontiguous physical data pages by
mapping them into a single VA range that gets passed to the dump device,
so there is no real advantage in batching calls to blk_write().

The RISC-V and MIPS minidump implementations still write a page at a
time and so would benefit from some form of batching.

MFC after: 2 weeks
Sponsored by: Juniper Networks, Klara Inc.
rm64/minidump_machdep.c
f0197bb22b46c53f3d0a50825d1f0139c406490d 23-Apr-2020 manu <manu@FreeBSD.org> arm64: rockchip: Fix TSADC on RK3328

The TSADC familiy is a little bit more complex than V2 and V3.
Early revision do not use syscon and do not use qsel (RK3288).
Next revision still do not use syscon but uses qsel (RK3328).
Final revision use both.

Submitted by: peterj
MFC after: 1 month
ockchip/rk_tsadc.c
b36f05da5b7ff750c2db6930b05992503525d893 22-Apr-2020 karels <karels@FreeBSD.org> Add genet driver for Raspberry Pi 4B Ethernet

Add driver for Broadcom "GENET" version 5, as found in BCM-2711 on
Raspberry Pi 4B. The driver is derived in part from the bcmgenet.c
driver in NetBSD, along with bcmgenetreg.h.

Reviewed by: manu
Obtained from: in part from NetBSD
Relnotes: yes, note addition
Differential Revision: https://reviews.freebsd.org/D24436
roadcom/genet/if_genet.c
roadcom/genet/if_genetreg.h
onf/GENERIC
nclude/bus.h
9957fe200822f42487e225996f7289f548394e09 19-Apr-2020 mhorne <mhorne@FreeBSD.org> Convert arm's physmem interface to MI code

The arm_physmem interface found in arm's MD code provides a convenient
set of routines for adding/excluding physical memory regions and
initializing important kernel globals such as Maxmem, realmem,
phys_avail[], and dump_avail[]. It is especially convenient for FDT
systems, since we can use FDT parsing functions and pass the result
directly to one of these physmem routines. This interface is already in
use on arm and arm64, and can be used to simplify this early
initialization on RISC-V as well.

This requires only a couple trivial changes:
- Move arm_physmem_kernel_addr to arm/machdep.c. It is unused on arm64,
and manipulated entirely in arm MD code.
- Convert arm32_btop/arm64_btop to atop. This is equivalently defined
on all architectures.
- Drop the "arm" prefix.

Reviewed by: manu, emaste ("looks reasonable")
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D24153
rm64/machdep.c
rm64/pmap.c
92f82df12b2680225b1e7827584d1cd2628eaa44 16-Apr-2020 brooks <brooks@FreeBSD.org> Convert canary, execpathp, and pagesizes to pointers.

Use AUXARGS_ENTRY_PTR to export these pointers. This is a followup to
r359987 and r359988.

Reviewed by: jhb
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24446
inux/linux_sysvec.c
cfb2be0cff25af0150a209059991a791ac7c42fa 14-Apr-2020 brooks <brooks@FreeBSD.org> Centralize compatability translation macros.

Copy the CP, PTRIN, etc macros from freebsd32.h into a sys/abi_compat.h
and replace existing definitation with includes where required. This
eliminates duplicate code and allows Linux and FreeBSD compatability
headers to be included in the same files.

Input from: cem, jhb
Obtained from: CheriBSD
MFC after: 2 weeks
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D24275
inux/linux.h
50ad80db3a783c80667acf768b86832f7707cab8 06-Apr-2020 wma <wma@FreeBSD.org> Add MDIO PHY driver for NS2 ARM64 platform.

Obtained from: Semihalf
Authored by: Kornel Duleba <mindal@semihalf.com>
Approved by: wma
Differential Revision: https://reviews.freebsd.org/D21335
roadcom/brcmmdio/mdio_mux_iproc.c
roadcom/brcmmdio/mdio_nexus_iproc.c
roadcom/brcmmdio/mdio_ns2_pcie_phy.c
efd93357ab78846fb426a7d9dda6f986d9680281 01-Apr-2020 jhb <jhb@FreeBSD.org> Retire procfs-based process debugging.

Modern debuggers and process tracers use ptrace() rather than procfs
for debugging. ptrace() has a supserset of functionality available
via procfs and new debugging features are only added to ptrace().
While the two debugging services share some fields in struct proc,
they each use dedicated fields and separate code. This results in
extra complexity to support a feature that hasn't been enabled in the
default install for several years.

PR: 244939 (exp-run)
Reviewed by: kib, mjg (earlier version)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D23837
rm64/trap.c
b4fc99e12caf06ae17d192f12c597ccb686b7300 25-Mar-2020 cem <cem@FreeBSD.org> Expand generic subword atomic primitives

The goal of this change is to make the atomic_load_acq_{8,16},
atomic_testandset{,_acq}_long, and atomic_testandclear_long primitives
available in MI-namespace.

The second goal is to get this draft out of my local tree, as anything that
requires a full tinderbox is a big burden out of tree. MD specifics can be
refined individually afterwards.

The generic implementations may not be ideal for your architecture; feel
free to implement better versions. If no subword_atomic definitions are
needed, the include can be removed from your arch's machine/atomic.h.
Generic definitions are guarded by defined macros of the same name. To
avoid picking up conflicting generic definitions, some macro defines are
added to various MD machine/atomic.h to register an existing implementation.

Include _atomic_subword.h in arm and arm64 machine/atomic.h.

For some odd reason, KCSAN only generates some versions of primitives.
Generate the _acq variants of atomic_load.*_8, atomic_load.*_16, and
atomic_testandset.*_long. There are other questionably disabled primitives,
but I didn't run into them, so I left them alone. KCSAN is only built for
amd64 in tinderbox for now.

Add atomic_subword implementations of atomic_load_acq_{8,16} implemented
using masking and atomic_load_acq_32.

Add generic atomic_subword implementations of atomic_testandset_long(),
atomic_testandclear_long(), and atomic_testandset_acq_long(), using
atomic_fcmpset_long() and atomic_fcmpset_acq_long().

On x86, add atomic_testandset_acq_long as an alias for
atomic_testandset_long.

Reviewed by: kevans, rlibby (previous versions both)
Differential Revision: https://reviews.freebsd.org/D22963
nclude/atomic.h
b18019e1704ddec73ee5470c5e5e5a04cccdd925 24-Mar-2020 markj <markj@FreeBSD.org> Remove the secondary_stacks array in arm64 and riscv kernels.

Instead, dynamically allocate a page for the boot stack of each AP when
starting them up, like we do on x86. This shrinks the bss by
MAXCPU*KSTACK_PAGES pages, which corresponds to 4MB on arm64 and 256KB
on riscv.

Duplicate the logic used on x86 to free the bootstacks, by using a
sysinit to wait for each AP to switch to a thread before freeing its
stack.

While here, mark some static MD variables as such.

Reviewed by: kib
MFC after: 1 month
Sponsored by: Juniper Networks, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D24158
rm64/locore.S
rm64/mp_machdep.c
04d52c33d18d92f41f968d8fba2616f9771310ad 18-Mar-2020 bdragon <bdragon@FreeBSD.org> [PowerPC][Book-E] Fix missing load base in elf_cpu_parse_dynamic().

When I implemented MD DYNAMIC parsing, I was originally passing a
linker_file_t so that the MD code could relocate pointers.

However, it turns out this isn't even filled in until later, so it was
always 0.

Just pass the load base (ef->address) directly, as that's really the only
thing we were interested in in the first place.

This fixes a crash on RB800 where it was trying to write to an unmapped
address when updating the GOT.

Reviewed by: jhibbits
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D24105
rm64/elf_machdep.c
11fcae0558b640a8916d95659b8bbd76e19f9c3c 09-Mar-2020 mmel <mmel@FreeBSD.org> Add the missing brackets to the logical expression.

Reported by: clang10 via dim
MFC with: r355755
ockchip/rk_pcie.c
cf07ed23e4b46907ab2f7ef0f0b7c829a100af6d 06-Mar-2020 andrew <andrew@FreeBSD.org> Add more are64 special register fields

Obtained from: https://github.com/FreeBSD-UPB/freebsd
nclude/armreg.h
3620fb0b18618866da7fc7f8ad351a92c183f789 06-Mar-2020 andrew <andrew@FreeBSD.org> Update the hypervisor registers

- Add more registers needed by bhyve [1]
- Move EL2 registers from armreg.h to hypervisor.h
- Add the register name to hypervisor.h

Obtained from: https://github.com/FreeBSD-UPB/freebsd [1]
nclude/armreg.h
nclude/hypervisor.h
1da2fbfb53944377439a142c280d79f5e22dbe28 05-Mar-2020 andrew <andrew@FreeBSD.org> Mark the arm64 machdep.h as kernel only

None of this is useful for userspace.

Sponsored by: Innovate UK
nclude/machdep.h
4a6509d155b097d94f504a604e1a46111fd41686 03-Mar-2020 andrew <andrew@FreeBSD.org> Fix the spelling of aliasing.

Sponsored by: Innovate UK
rm64/identcpu.c
nclude/cpufunc.h
dbed753b3fe09fc5b98bc19b445c6a1f523ea4a8 03-Mar-2020 andrew <andrew@FreeBSD.org> Move the arm64 cache identification to identcpu.c

This allows us to call it on a per-CPU basis and to warn if the details
are different across CPUs.

While here read the L1 I-Cache type and store this for use later by pmap.

Sponsored by: Innovate UK
rm64/identcpu.c
rm64/machdep.c
nclude/cpu.h
nclude/cpufunc.h
444460945e9b2b4739c9f9b4920e8844269c4a60 03-Mar-2020 andrew <andrew@FreeBSD.org> Fix the spelling of the VIPT cache type field

Sponsored by: Innovate UK
rm64/identcpu.c
nclude/armreg.h
2b5b74c14ddd16bea5bc0d0c23077fe6b7a70ca5 03-Mar-2020 andrew <andrew@FreeBSD.org> Store the boot exception level on arm64 so it can be queried later

A hypervisor, e.g. bhyve, will need to know what exception levelthe kernel
was in when it started booting. If it was EL2 we can then enable said
hypervisor.

Store the boot exception level and allow the kernel to later query it.

Obtained from: https://github.com/FreeBSD-UPB/freebsd (earlier version)
Sponsored by: Innovate UK
rm64/genassym.c
rm64/locore.S
rm64/machdep.c
nclude/machdep.h
5faa1fbd6fe5b8694153d0e536cd993a36e2009a 02-Mar-2020 andrew <andrew@FreeBSD.org> Add a space missed in r358545

Sponsored by: Innovate UK
rm64/genassym.c
5af3998139aad951f607f69aeade92737e09e83b 02-Mar-2020 andrew <andrew@FreeBSD.org> Generate the offsets for struct arm64_bootparams and use it in locore.S

This removes one place with hard coded offsets in locore.S

Sponsored by: Innovate UK
rm64/genassym.c
rm64/locore.S
37df5e46b084696ed87e8002e60680a49e734858 29-Feb-2020 dim <dim@FreeBSD.org> Fix the following -Werror warning from clang 10.0.0:

sys/arm64/arm64/identcpu.c:1170:5: error: misleading indentation; statement is not part of the previous 'if' [-Werror,-Wmisleading-indentation]
break;
^
sys/arm64/arm64/identcpu.c:1168:4: note: previous statement is here
if (fv[j].desc[0] != '\0')
^

The break should be after the if statement, indented one level less.

MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D23871
rm64/identcpu.c
ccf948ce6c75db1bcc7754bf40606218d9510f17 26-Feb-2020 andrew <andrew@FreeBSD.org> Fix the cache type identification

DIC and IDC are supported when the field bits are set.

Sponsored by: Innovate UK
rm64/identcpu.c
ad355b0a9dbd6a8aabe7c081a731d24904a0f2c1 26-Feb-2020 kaktus <kaktus@FreeBSD.org> Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718
rm64/busdma_bounce.c
rm64/pmap.c
7eb60bc21c7828533af68925dc4ad8e2f83b1794 26-Feb-2020 andrew <andrew@FreeBSD.org> Teach the arm64 ident CPU code to print non-ID registers

Add support for non-ID registers when printing CPU information. This is
used with the cache type register to print details of the cache on boot.

Sponsored by: Innovate UK
rm64/identcpu.c
246dc0cc919f6c21c61b1e33f6807dcefcd4fa35 26-Feb-2020 andrew <andrew@FreeBSD.org> Generalise the arm64 ASID allocator.

The requirements of an Address Space ID allocator and a Virtual Machine ID
allocator are similar. Generalise the former code so it can be used with
the latter.

Reviewed by: alc (previous version)
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D23831
rm64/pmap.c
nclude/pmap.h
b28fc4c314c56108154f488b0a815acdba8c3cc9 26-Feb-2020 andrew <andrew@FreeBSD.org> Start to support multiple stages in the arm64 pmap.

On arm64 the stage 1 and stage 2 pte formats are similar enough we can
reuse the pmap code for both. As they are only similar and not identical
we need to know if we are managing stage 1 or stage 2 tables.

Add an enum to store this information and a check to make sure it is
set to stage 1 when we manage stage 1 pte fields.

Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D23830
rm64/pmap.c
nclude/pmap.h
6cb95687b2be0edd46ac42b3fa105f4b8fb0a868 26-Feb-2020 andrew <andrew@FreeBSD.org> Add more arm64 CTR_EL0 register fields

While here make the _SIZE macros return the size in bytes, not the log2
of the size

Sponsored by: Innovate UK
rm64/machdep.c
nclude/armreg.h
dcc60239b6e4c0ec988c2067802ad3fda77ee8e2 24-Feb-2020 andrew <andrew@FreeBSD.org> Split out the stage 1 pte bits and add the stage 2 bits

In preperation for adding bhyve support to arm64 we need to split the
stage 1 and stage 2 pte fields to allow future changes to create stage 2
page tables.

MFC after: 1 month
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D23669
rm64/efirt_machdep.c
rm64/locore.S
rm64/pmap.c
nclude/pte.h
ffaf9d2a0bff8dd8c81dcb67d9dcbdf8cc134ddd 24-Feb-2020 cognet <cognet@FreeBSD.org> debug_monitor: Avoid setting the PSR_D flag for 32bits binaries.

In dbg_monitor_exit(), avoid setting the PSR_D bit if the process is
a 32bits binary. PSR_D is an aarch64-only flags, and for aarch32 processes,
it means "run in big endian".
This should make COMPAT_FREEBSD32 run much better on arm64.
rm64/debug_monitor.c
8b4e3014226ce2cedf2ce81c1424d7ec71e9ceb6 24-Feb-2020 kaktus <kaktus@FreeBSD.org> Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (13 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Approved by: kib (mentor, blanket)
Differential Revision: https://reviews.freebsd.org/D23635
ockchip/rk_tsadc.c
b24845d0f3b1debb662e3ed28fbf7d8d3a519a6c 24-Feb-2020 manu <manu@FreeBSD.org> arm64: rockchip: rk808: Only init regulator not enabled

If a regulator is already enabled, do not set its value to the minimum
supported on the board.
This fixes booting on rock64 where we set some regulator to the minimal value
while the IPs needs more based on what the bootloader configured.

MFC after: 1 week
ockchip/rk805.c
3d77e3fa2f0de60dd4e31e941c15a1504f530536 24-Feb-2020 manu <manu@FreeBSD.org> arm64: rockchip: rk_i2c: Bump to DELAY to 1000

In polling mode with use DELAY to wait for interrupts. The value was
too low for RK3328.

MFC after: 1 week
ockchip/rk_i2c.c
e9f5c437df87778494ee4915632bdc7a0237a203 14-Feb-2020 mjg <mjg@FreeBSD.org> arm64: remove no longer needed atomic_load_ptr casts
rm64/gicv3_its.c
d3e265fd0bc0b455d7ac2d9b69d358b33563838c 11-Feb-2020 br <br@FreeBSD.org> Add PCI Express driver for the ARM Neoverse N1 System Development
Platform (N1SDP).

Neoverse N1 is a high-performance ARM microarchitecture designed
by the ARM Holdings for the server market.

The PCI part on N1SDP was shipped untested and suffers from some
integration issues.

For instance accessing to not existing BDFs causes System Error
(SError) exception. To mitigate this, the firmware scans the bus,
catches SErrors and creates a table with valid BDFs. That allows
us to filter-out accesses to invalid BDFs in this driver.

Also the root complex config space (BDF == 0) has an unusual
location in memory map, so remapping accesses to it is required.

Finally, the config space is restricted to 32-bit accesses only.

This was tested on the ARM boxes kindly provided by the ARM Ltd
to the DARPA CHERI Project.

In collaboration with: andrew
Reviewed by: andrew
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D23349
onf/GENERIC
821c573b6d6c76cfd0daf25e76b1c1f81ab7177f 08-Feb-2020 andrew <andrew@FreeBSD.org> Implement the Linux/arm64 VDSO gettimeofday and clock_gettime

These are used by Centos 7 and will be needed for the
emulators/linux_base-c7 port.

Sponsored by: DARPA, AFRL
inux/linux_locore.asm
80f1eda869e67c243fda9dfd0b0411d03aa69373 05-Feb-2020 markj <markj@FreeBSD.org> Define MAXCPU consistently between the kernel and KLDs.

This reverts r177661. The change is no longer very useful since
out-of-tree KLDs will be built to target SMP kernels anyway. Moveover
it breaks the KBI in !SMP builds since cpuset_t's layout depends on the
value of MAXCPU, and several kernel interfaces, notably
smp_rendezvous_cpus(), take a cpuset_t as a parameter.

PR: 243711
Reviewed by: jhb, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23512
nclude/param.h
77a92d8644e7b1b97832b4a058653519c0b56830 05-Feb-2020 emaste <emaste@FreeBSD.org> linuxulator: implement sendfile

Submitted by: Bora Özarslan <borako.ozarslan@gmail.com>
Submitted by: Yang Wang <2333@outlook.jp>
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19917
inux/linux_dummy.c
6371e0efd7d22bcd9666a69ed8163a0937abd6aa 04-Feb-2020 andrew <andrew@FreeBSD.org> Print useful debug data on unhandled kernel fault on arm64

When panicing because of an unhandled data abort from the kernel it is
useful to know the register state and faulting address to aid debugging.
Print these registers before calling panic.

Sponsored by: DARPA, AFRL
rm64/trap.c
73a1608d91be97516e39fd671b15ea6ec5fe6080 03-Feb-2020 markj <markj@FreeBSD.org> Dynamically select LSE-based atomic(9)s on arm64.

Once all CPUs are online, determine if they all support LSE atomics and
set lse_supported to indicate this. For now the atomic(9)
implementations are still always inlined, though it would be preferable
to create out-of-line functions to avoid text bloat. This was not done
here since big.little systems exist in which some CPUs implement LSE
while others do not, and ifunc resolution must occur well before this
scenario can be detected. It does seem unlikely that FreeBSD will
ever run on such platforms, however, so converting atomic(9) to use
ifuncs is probably a good next step.

Add a LSE_ATOMICS arm64 kernel configuration option to unconditionally
select LSE-based atomic(9) implementations when the target system is
known.

Reviewed by: andrew, kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation, Amazon (hardware)
Differential Revision: https://reviews.freebsd.org/D23325
rm64/identcpu.c
nclude/atomic.h
1f66354182b4edd933c34bba2192e7ef8eb6e0cb 03-Feb-2020 markj <markj@FreeBSD.org> Add LSE-based atomic(9) implementations.

These make use of the cas*, ld* and swp instructions added in ARMv8.1.
Testing shows them to be significantly more performant than LL/SC-based
implementations.

No functional change here since the wrappers still unconditionally
select the _llsc variants.

Reviewed by: andrew, kib
MFC after: 1 month
Submitted by: Ali Saidi <alisaidi@amazon.com> (original version)
Differential Revision: https://reviews.freebsd.org/D23324
nclude/atomic.h
4200d8d829b0dbc600e723fd61cfd72af3cc6cf9 03-Feb-2020 markj <markj@FreeBSD.org> Add wrappers for arm64 atomics.

Add a _llsc suffix for the existing LL/SC-based implementations and add
trivial wrappers. This is in preparation for supporting LSE-based
atomic(9) implementations.

No functional change intended.

Reviewed by: andrew, kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation, Amazon (hardware)
Differential Revision: https://reviews.freebsd.org/D23323
nclude/atomic.h
ef0260fb177b3aead8aebf9f5838a46e99003958 03-Feb-2020 markj <markj@FreeBSD.org> Provide a single implementation for each of the arm64 atomic(9) ops.

Parameterize the macros by type width as well as acq/rel semantics.
This makes modifying the implementations much less tedious and
error-prone and makes it easier to support alternate LSE-based
implementations. No functional change intended.

Reviewed by: andrew, kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation, Amazon (hardware)
Differential Revision: https://reviews.freebsd.org/D23322
nclude/atomic.h
4f89429eca71cd2487015a3458d4de37a32ada85 03-Feb-2020 andrew <andrew@FreeBSD.org> Remove the GICv3 ITS irq and replace it with an ID

In r357324 most of the use of gi_irq was moved to gi_lpi. Complete this
with the last few places we need the IRQ value and create gi_id for the
per-device value we need.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL
rm64/gicv3_its.c
5652acaf962d5657300e42d7e1e36b0ea88a537f 03-Feb-2020 andrew <andrew@FreeBSD.org> Use a unique name for the GICv3 ITS vmem

When there are multiple GICv3 ITS devices we don't know which vmem is for
which device. Use device_get_nameunit to get a per-device name.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL
rm64/gicv3_its.c
5154c79c1670a519d65efdbbf481fab89cda8bd0 03-Feb-2020 andrew <andrew@FreeBSD.org> Disable the use of the quantum cache in the GICv3 ITS

This uses UMA to allocate space. It causes issues when there are multiple
ITS devices in the system where interrupts are not allocated from a low
address on some interrupt controllers. Disabling the quantum cache fixes
this on the Neoverse N1 SDP.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL
rm64/gicv3_its.c
97176dfe26093b2c4c3ed523e979ce26edc96ce7 31-Jan-2020 markj <markj@FreeBSD.org> Reimplement stack capture of running threads on i386 and amd64.

After r355784 the td_oncpu field is no longer synchronized by the thread
lock, so the stack capture interrupt cannot be delievered precisely.
Fix this using a loop which drops the thread lock and restarts if the
wrong thread was sampled from the stack capture interrupt handler.

Change the implementation to use a regular interrupt instead of an NMI.
Now that we drop the thread lock, there is no advantage to the latter.

Simplify the KPIs. Remove stack_save_td_running() and add a return
value to stack_save_td(). On platforms that do not support stack
capture of running threads, stack_save_td() returns EOPNOTSUPP. If the
target thread is running in user mode, stack_save_td() returns EBUSY.

Reviewed by: kib
Reported by: mjg, pho
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23355
rm64/stack_machdep.c
832e6ede23587bc3cad0250821e86e4999165113 31-Jan-2020 andrew <andrew@FreeBSD.org> Call the MAPTI command earlier in the ITS driver

The GICv3 Software Overview suggests when allocating a new MSI/MSI-X
interrupt we need to call MAPD followed by MAPTI. Unfortunately the code
would place a MOVI command between these. This is invalid as it needs
values set by the MAPTI to be present.

Re-order so we allocate a temporary CPU for the interrupt, then use the
MAPTI command to assign the MSI to it.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL
rm64/gicv3_its.c
86e6ac26b3224ad9ad26aedc3d41b3ce1cc951b0 31-Jan-2020 andrew <andrew@FreeBSD.org> Only create one ITS configuration table

When there are multiple ITS devices in the system we would allocate a
configuration table for each, however only one table is needed as all the
ITS devices share this.

Allocate a table only when the global table is unset.

While here fix the type of this to be a pointer to a uint8_t array as the
entries are all 8 bits wide.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL
rm64/gicv3_its.c
00c33c4006eba8934d1c9f0901c5665ca9288f0b 31-Jan-2020 andrew <andrew@FreeBSD.org> Ignore the SMMUv3 and PMCG interrupt controller in the IORT tables

When mapping MSI/MSI-X interrupts throught he Arm IORT ACPI tables we may
need to ignore an interrupt controller even if it is within the bounds the
entry describes. When the SMMUv3 is not using GSIV (non-MSI/MSI-X)
interrupts we need to read the defined field. The Performance Monitoring
Counter Group always ignores the first table entry.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL
cpica/acpi_iort.c
53f175976e0fb7b22c593d2188bacc79bcd115d0 30-Jan-2020 andrew <andrew@FreeBSD.org> Shift the ITS processor ID after reading it.

When using the processor ID value we mask off the low and high bits that
should be zero. Unfortunatly we don't shift the ID value so it won't be
affected. Add the shift when reading the ID as this will need to align
with the address based target value.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL
rm64/gicv3_its.c
14c5035597f56a9b575a1d8c2ceb2e763544bb2d 29-Jan-2020 ganbold <ganbold@FreeBSD.org> Enable USB3 support for Rockchip RK3328 SoC.

Reviewed by: manu
ockchip/rk_dwc3.c
f8a8685f3987e38425e5df5fb29bbc931bb34eb2 29-Jan-2020 ganbold <ganbold@FreeBSD.org> Add USB3 related clock definitions for Rockchip RK3328 SoC.

Reviewed by: manu
ockchip/clk/rk3328_cru.c
9838560238c74a59f46715d9750655eb4e21a396 23-Jan-2020 markj <markj@FreeBSD.org> Print missing ID_AA64PFR{0,1}_EL1 register fields.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23213
rm64/identcpu.c
nclude/armreg.h
50a3e6955b8166d8f0f52c741c26d36099c26e70 23-Jan-2020 markj <markj@FreeBSD.org> arm64: Don't enable interrupts in init_secondary().

Doing so can cause deadlocks or panics during boot, if an interrupt
handler accesses uninitialized per-CPU scheduler structures. This seems
to occur frequently when running under QEMU or AWS. The idle threads
are set up to release a spinlock section and enable interrupts in
fork_exit(), so there is no need to enable interrupts earlier.

Reviewed by: kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23328
rm64/mp_machdep.c
ad161bbe96fef2239eed75fdc9f08a2f26e2a1f5 21-Jan-2020 markj <markj@FreeBSD.org> Add relocation handling required for -zifunc-noplt to work on arm64.

Static relocations for the immediate operand of a branch instruction
must be applied.

In a patch which implements LSE-based atomic(9) operations using ifuncs,
-zifunc-noplt reduces system CPU usage during a buildkernel by several
percent.

Also fix elf_reloc_internal() to return an error if symbol lookup fails.

Reviewed by: andrew
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17392
rm64/elf_machdep.c
ce2492e3e5c83ec3cb8b3a818d11bb0f154af181 19-Jan-2020 manu <manu@FreeBSD.org> rk805: Add a regnode_init method

This method will set the desired voltaged based on values in the DTS.
It will not enable the regulator, this is the job of either a consumer
or regnode_set_constraint SYSINIT if the regulator is boot_on or always_on.

Reviewed by: mmel
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D23216
ockchip/rk805.c
20614c0f40a14194eb6654b1fafd2f8fb311c71e 18-Jan-2020 kevans <kevans@FreeBSD.org> sysent targets: further cleanup and deduplication

r355473 vastly improved the readability and cleanliness of these Makefiles.
Every single one of them follows the same pattern and duplicates the exact
same logic.

Now that we have GENERATED/SRCS, split SRCS up into the two parameters we'll
use for ${MAKESYSCALLS} rather than assuming a specific ordering of SRCS and
include a common sysent.mk to handle the rest. This makes it less tedious to
make sweeping changes.

Some default values are provided for GENERATED/SYSENT_*; almost all of these
just use a 'syscalls.master' and 'syscalls.conf' in cwd, and they all use
effectively the same filenames with an arbitrary prefix. Most ABIs will be
able to get away with just setting GENERATED_PREFIX and including
^/sys/conf/sysent.mk, while others only need light additions. kern/Makefile
is the notable exception, as it doesn't take a SYSENT_CONF and the generated
files are spread out between ^/sys/kern and ^/sys/sys, but it otherwise fits
the pattern enough to use the common version.

Reviewed by: brooks, imp
Nice!: emaste
Differential Revision: https://reviews.freebsd.org/D23197
inux/Makefile
2225c7a2a293cfaa92b68eea3be395f7e2643904 17-Jan-2020 jhb <jhb@FreeBSD.org> Save and restore floating point registers in get/set_mcontext().

arm64 and riscv were only saving and restoring floating point
registers for sendsig() and sys_sigreturn(), but not for getcontext(),
setcontext(), and swapcontext().

While here, remove an always-false check for uap being NULL from
sys_sigreturn().

Reviewed by: br, mhorne
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23218
rm64/machdep.c
5438208aba2dcebf84e97179e1f13826fec1ad9f 16-Jan-2020 manu <manu@FreeBSD.org> arm64: rockchip: Add RK3399 PWM driver

Add a driver for the pwm controller in the RK3399 SoC

Submitted by: bdragon (original version)
Reviewed by: ganbold (previous version)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D19046
onf/GENERIC
ockchip/clk/rk3399_pmucru.c
ockchip/rk_pwm.c
ed6be8c4ecdf7b63fd103dcc9dc9f8c155a08d48 16-Jan-2020 manu <manu@FreeBSD.org> arm64: rockchip: Add new interface for rk_pinctrl

The gpio controller in rockchips SoC in a child of the pinctrl driver
and cannot control pullups and pulldowns.
Use the new fdt_pinctrl interface for accessing pin capabilities and
setting them.
We can now report that every pins is capable of being IN or OUT function
and PULLUP PULLDOWN.
If the pin isn't in gpio mode no changes will be allowed.

Reviewed by: ganbold (previous version)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22849
ockchip/rk_gpio.c
ockchip/rk_pinctrl.c
f813cffaf1153922051989adeefaeeaed66311ea 10-Jan-2020 kevans <kevans@FreeBSD.org> Set .ORDER for makesyscalls generated files

When either makesyscalls.lua or syscalls.master changes, all of the
${GENERATED} targets are now out-of-date. With make jobs > 1, this means we
will run the makesyscalls script in parallel for the same ABI, generating
the same set of output files.

Prior to r356603 , there is a large window for interlacing output for some
of the generated files that we were generating in-place rather than staging
in a temp dir. After that, we still should't need to run the script more
than once per-ABI as the first invocation should update all of them. Add
.ORDER to do so cleanly.

Reviewed by: brooks
Discussed with: sjg
Differential Revision: https://reviews.freebsd.org/D23099
inux/Makefile
f2c97d85285292fcb3cfb19da31a16129e388188 09-Jan-2020 andrew <andrew@FreeBSD.org> Add atomic_testandset/clear on arm64.

These will reportedly be used in future uma changes.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D23019
nclude/atomic.h
b63b91aad543c2ae28fe796110aad9db2689fc37 08-Jan-2020 manu <manu@FreeBSD.org> rk805: Add regnode_status method

This allow consumers to check if the regulator is enable or not.

Reviewed by: mmel
Differential Revision: https://reviews.freebsd.org/D23005
ockchip/rk805.c
4bbb0710badf20aa46a433936a4aaa936830d728 08-Jan-2020 manu <manu@FreeBSD.org> rk808: Add min/max for the switch regulators

The two switch regulator are always 3.0V.
Add a special case in get_voltage that if min=max we directly
return the value without calculating it.

Reviewed by: mmel
Differential Revision: https://reviews.freebsd.org/D23004
ockchip/rk805.c
338a93d2b7c3bd053cbe728cb5f6d3bc23f4e9d3 06-Jan-2020 andrew <andrew@FreeBSD.org> Add more Arm arm64 CPU identification values

- Add all the Cortex-A CPU ID register values I can find.
- Add the Neoverse-N1 ID regiser value [1]
- Sort macros by register value.

PR: 243065
Submitted by: Ali Saidi <alisaidi AT amazon.com> [1]
Sponsored by: DARPA, AFRL (other than [1])
rm64/identcpu.c
nclude/cpu.h
b05c0a53bd97ab8ab523446f8946e34294750547 06-Jan-2020 kaktus <kaktus@FreeBSD.org> sysctl: mark more nodes as MPSAFE

vm.kvm_size and vm.kvm_free are read only and marked as MPSAFE on i386
already. Mark them as that on amd64 and arm64 too to avoid locking Giant.

Reviewed by: kib (mentor)
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D23039
rm64/pmap.c
0adae291f5700a429504b40a2c6ceb226c081655 04-Jan-2020 alc <alc@FreeBSD.org> When a copy-on-write fault occurs, pmap_enter() is called on to replace the
mapping to the old read-only page with a mapping to the new read-write page.
To destroy the old mapping, pmap_enter() must destroy its page table and PV
entries and invalidate its TLB entry. This change simply invalidates that
TLB entry a little earlier, specifically, on amd64 and arm64, before the PV
list lock is held.

Reviewed by: kib, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D23027
rm64/pmap.c
9eaa615b301de05149c9363b895afb26315934f0 03-Jan-2020 andrew <andrew@FreeBSD.org> Add the 8 and 16 bit atomic load/store functions with a barrier on arm64.

Reviewed by: cem
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22966
nclude/atomic.h
972798ebfd1bd74199c171f283d86d1432197967 02-Jan-2020 ian <ian@FreeBSD.org> Add the xdma framework and pl330 dma drivers to arm and arm64 lint builds.
onf/NOTES
f587e7148a5b210125c69ae60b233e03c321281a 02-Jan-2020 manu <manu@FreeBSD.org> arm64: rockchip: Add a module for rk_spi

The spi node doesn't lives under a simple-bus compatible node so we need
OFWBUS_PNP_INFO instead of SIMPLEBUS_PNP_INFO.
ockchip/rk_spi.c
f870efbd57f42155d26b2bc7fb95846897449dca 31-Dec-2019 trasz <trasz@FreeBSD.org> Add basic getcpu(2) support to linuxulator. The purpose of this
syscall is to query the CPU number and the NUMA domain the calling
thread is currently running on. The third argument is ignored.
It doesn't do anything regarding scheduling - it's literally
just a way to query the current state, without any guarantees
you won't get rescheduled an opcode later.

This unbreaks Java from CentOS 8
(java-11-openjdk-11.0.5.10-0.el8_0.x86_64).

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22972
inux/linux_dummy.c
e10b156363bcec2ff5dc6da656a2e2ff59fc73d6 30-Dec-2019 alc <alc@FreeBSD.org> Determine whether the MMU hardware is capable of updating a page table
entry's access flag and dirty state, and enable this feature when it's
available.

Ensure that we don't overlook a dirty state update that is concurrent
with a call to pmap_enter(). (Previously, all dirty state updates would
have occurred with the containing pmap's lock held, so a page table entry's
dirty state could not have changed while pmap_enter() held that same lock.)

Reviewed by: andrew, markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22907
rm64/locore.S
rm64/pmap.c
nclude/armreg.h
121a4336abf995fd8768d9aa2594e5e07f380420 30-Dec-2019 kaktus <kaktus@FreeBSD.org> linux(4): implement copy_file_range(2)

copy_file_range(2) is implemented natively since r350315, make it available
for Linux binaries too.

Reviewed by: kib (mentor), trasz (previous version)
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D22959
inux/linux_dummy.c
4b3d6ef9bc03864312621102fed65161b024931d 29-Dec-2019 trasz <trasz@FreeBSD.org> Implement Linux syslog(2) syscall; just enough to make Linux dmesg(8)
utility work.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22465
inux/linux_dummy.c
fe6463862ccf9f935ac21e5f2fc8f701f661c20f 28-Dec-2019 manu <manu@FreeBSD.org> arm64: rockchip: Add driver for the io domain

This driver configure the registers in the GRF according to the value
of the regulators for the platform.
Some IP can run with either 3.0V or 1.8V, if we don't configure them
correctly according to the external voltage used they will not work.
It's only done at boot time for now and might be needed at runtime for
IP like sdmmc.

Reviewed by: mmel
Tested On: RockPro64, Firefly-RK3399 (gonzo), AIO-3288 (mmel)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22854
onf/GENERIC
ockchip/rk_iodomain.c
c0f6a2ec659599f8a8615b27fec36a9b52a72fe9 28-Dec-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk808: Add remaining regulators

The RK808 driver was missing the LDO and switch regulators.
Add support for them.

Reviewed by: mmel
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22852
ockchip/rk805.c
ockchip/rk805reg.h
8191fa8f428a0d79f6b1d1ce7b0c4bfe4b53bcd6 24-Dec-2019 bdragon <bdragon@FreeBSD.org> [PowerPC] Implement Secure-PLT jump table processing for ppc32.

Due to clang and LLD's tendency to use a PLT for builtins, and as they
don't have full support for EABI, we sometimes have to deal with a PLT in
.ko files in a clang-built kernel.

As such, augment the in-kernel linker to support jump table processing.

As there is no particular reason to support lazy binding in kernel modules,
only implement Secure-PLT immediate binding.

As part of these changes, add elf_cpu_parse_dynamic() to the MD API of the
in-kernel linker (except on platforms that use raw object files.)

The new function will allow MD code to act on MD tags in _DYNAMIC.

Use this new function in the PowerPC MD code to ensure BSS-PLT modules using
PLT will be rejected during insertion, and to poison the runtime resolver to
ensure we get a clear panic reason if a call is made to the resolver.

Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D22608
rm64/elf_machdep.c
15e06ff0d9961d44bf803186786853953c964715 24-Dec-2019 scottl <scottl@FreeBSD.org> Introduce the concept of busdma tag templates. A template can be allocated
off the stack, initialized to default values, and then filled in with
driver-specific values, all without having to worry about the numerous
other fields in the tag. The resulting template is then passed into
busdma and the normal opaque tag object created. See the man page for
details on how to initialize a template.

Templates do not support tag filters. Filters have been broken for many
years, and only existed for an ancient make/model of hardware that had a
quirky DMA engine. Instead of breaking the ABI/API and changing the
arugment signature of bus_dma_tag_create() to remove the filter arguments,
templates allow us to ignore them, and also significantly reduce the
complexity of creating and managing tags.

Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D22906
rm64/busdma_machdep.c
41b7f699dcaedbec92b71740e74175782c52c93b 20-Dec-2019 alc <alc@FreeBSD.org> Correct a mistakenly inverted condition in r355833.

Noticed by: kib
X-MFC with: r355833
rm64/pmap.c
29fb48ace4186a41c409fde52bcf4216e9e50b61 19-Dec-2019 andrew <andrew@FreeBSD.org> Stop speculation past an eret instruction

On arm64 the eret instruction is used to return from an exception handler.
Some implementations may speculate past this instruction into the next
function. As the user may control many registers in these functions add
a synchronisation barrier sequence after the eret instruction to stop these
CPUs from speculating out of the exception handler.

PR: 242676
Submitted by: Anthony Steinhauser <asteinhauser@google.com> (previous version)
MFC after: 1 week
rm64/exception.S
rm64/swtch.S
nclude/asm.h
e5b1f5ca560adbe27aacea5f4fa409e480fb0bd2 18-Dec-2019 alc <alc@FreeBSD.org> When pmap_enter_{l2,pde}() are called to create a kernel mapping, they are
incrementing (and decrementing) the ref_count on kernel page table pages.
They should not do this. Kernel page table pages are expected to have a
fixed ref_count. Address this problem by refactoring pmap_alloc{_l2,pde}()
and their callers. This also eliminates some duplicated code from the
callers.

Correctly implement PMAP_ENTER_NOREPLACE in pmap_enter_{l2,pde}() on kernel
mappings.

Reduce code duplication by defining a function, pmap_abort_ptp(), for
handling a common error case.

Handle a possible page table page leak in pmap_copy(). Suppose that we are
determining whether to copy a superpage mapping. If we abort because there
is already a mapping in the destination pmap at the current address, then
simply decrementing the page table page's ref_count is correct, because the
page table page must have a ref_count > 1. However, if we abort because we
failed to allocate a PV entry, this might be a just allocated page table
page that has a ref_count = 1, so we should call pmap_abort_ptp().

Simplify error handling in pmap_enter_quick_locked().

Reviewed by: kib, markj (an earlier)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22763
rm64/pmap.c
fbedd6e8d7db73108408289bd2cc114db78f8d1f 17-Dec-2019 ian <ian@FreeBSD.org> Remove unnecessary MODULE_DEPEND() from imx_i2c.c, and also from rk_i2c
where it got copied to.
ockchip/rk_i2c.c
63a4cb2ce5f278516ab125586ec317074c959e3b 17-Dec-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk_gpio: Fix pin number

The maxpin counter starts at 0, fix one by one error.
This is still not totally correct for some banks in some SoC that have
fewer pins but this will be dealt with in another commit.

MFC after: 3 days
ockchip/rk_gpio.c
d20f5688ee0421fb7800a5ed3aec7cee04d3f537 17-Dec-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk_pinctrl: Fix clear bits in SYSCON_MODIFY

r351187 change the SYSCON_WRITE to SYSCON_MODIFY but didn't changed the
mask variable that used to hold the bitmask in the upper 16 bits of the
register that control which bits are changed. So we ended up clearing
bit from the upper 16bits half which are always 0 after a read.
Use the correct bit mask for bits that we want to clear.

MFC after: 3 days
ockchip/rk_pinctrl.c
aefc8ff8af51cff738a975e85a36968bdf8aca1b 16-Dec-2019 jeff <jeff@FreeBSD.org> Repeat the spinlock_enter/exit pattern from amd64 on other architectures to
fix an assert violation introduced in r355784. Without this spinlock_exit()
may see owepreempt and switch before reducing the spinlock count. amd64
had been optimized to do a single critical enter/exit regardless of the
number of spinlocks which avoided the problem and this optimization had
not been applied elsewhere.

Reported by: emaste
Suggested by: rlibby
Discussed with: jhb, rlibby
Tested by: manu (arm64)
rm64/machdep.c
e41f6b35b706a65fc5243073451db8cc3214dbc0 16-Dec-2019 trasz <trasz@FreeBSD.org> Add compat.linux.emul_path, so it can be set to something other
than "/compat/linux". Useful when you have several compat directories
with different Linux versions and you don't want to clash with files
installed by linux-c7 packages.

Reviewed by: bcr (manpages)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22574
inux/linux_sysvec.c
21262f8f6ba292f349403299c224d181329ea53a 15-Dec-2019 alc <alc@FreeBSD.org> Apply a small optimization to pmap_remove_l3_range(). Specifically, hoist a
PHYS_TO_VM_PAGE() operation that always returns the same vm_page_t out of
the loop. (Since arm64 is configured as VM_PHYSSEG_SPARSE, the
implementation of PHYS_TO_VM_PAGE() is more costly than that of
VM_PHYSSEG_DENSE platforms, like amd64.)

MFC after: 1 week
rm64/pmap.c
8d617afff89bec1af683d2d5dad1d57fbb581525 14-Dec-2019 mmel <mmel@FreeBSD.org> Add driver for Rockchip PCIe root complex found in RK3399 SOC.
Unfortunately, there are some limitations:
- memory aperture of his controller is only 16MiB, so it is nearly
unusable for graphic cards
- every attempt to generate type 1 config cycle always causes trap.
These config cycles are disabled now and we don't support cards
with PCIe switch.
- in some cases, attempt to do config cycle to (probably) not-yet ready
card also causes trap. This cannot be detected at runtime, but it seems
like very rare issue.

MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D22724
ockchip/rk_pcie.c
ockchip/rk_pcie_phy.c
820308e362842a8bf5db2d437e6ee913c7919889 14-Dec-2019 trasz <trasz@FreeBSD.org> Add sync_file_range(2) implementation to linux(4); it's a thin wrapper
over the usual fsync(2).

This silences some warnings when running "apt-get upgrade".

Reviewed by: brooks, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22371
inux/linux_dummy.c
2d2dde30e5b9735fa9ade8f138bb5ccdd296c3a8 14-Dec-2019 trasz <trasz@FreeBSD.org> Regen after r355752.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22371
inux/linux_proto.h
inux/linux_sysent.c
inux/linux_systrace_args.c
90c1a7bcc7f0a36ae66bba1edbd1fd9df34bea37 14-Dec-2019 trasz <trasz@FreeBSD.org> Fix definitions for linuxulator's sync_file_range(2).

Reviewed by: brooks, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22371
inux/syscalls.master
34e92dc3f5835f726760e21289c2b2dec41ab25a 12-Dec-2019 andrew <andrew@FreeBSD.org> Add comments and macros to the tcr_el1 setting code to help understand it.

This code is non-obvious when reading for the first time. To help with
understanding of it add comments explaining what it's doing.

While here use macros from armreg.h rather than magic numbers.

Sponsored by: DARPA, AFRL
rm64/locore.S
nclude/armreg.h
408bbe634d4cd76948979ff299e4b051f6c6485e 12-Dec-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk_pinctrl: Fix parse_bias for RK3399

Only bank 0 and bank 2 are different than other rockchip SoC, fix this.
While here remove some debug printfs that where added in r355648

MFC after: 3 days
X-MFC-With: r355648
ockchip/rk_pinctrl.c
aa384bac5c479195505fda196667403a93c68535 12-Dec-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk_pinctrl: Add bias parsing based on the SoC type

Not all rockchip have the same value for pullup/pulldown so add a function
per SoC and call the right one to have the proper value.

MFC after: 3 days
ockchip/rk_pinctrl.c
07f91577ca82884755b2bd47413601110196311c 11-Dec-2019 manu <manu@FreeBSD.org> arm64: rk3328: Add the *clk_peri_niu clocks

Those clocks are always enable by default and are not really explained
in the TRM but the reason we had them is that they have the periph clock
as a parent and those parent should never be disable which can happen
if we disable all the childs. The current childs are the sd/emmc/sdio clocks
so the board will hang if we disable them.

MFC after: 1 month
ockchip/clk/rk3328_cru.c
22d489d15f404fb16c965665febf74115b481e29 11-Dec-2019 manu <manu@FreeBSD.org> arm64: Add explicit devices for dwmmc variant

We used to include the hisi version if soc_hisi_hi6220 was present,
include the altera version if dwmmc_altera was present and include
the rockchip version if soc_rockchip_rk3328 was present.
Now every version have it's own device directive.
The rockchip version isn't named dwmmc_rockchip because all other
rockchip driver are named rk_XXX.

MFC after: 1 month
onf/GENERIC
e0995c18f647f9b974fb815174e7699766f460fb 10-Dec-2019 markj <markj@FreeBSD.org> Introduce vm_page_astate.

This is a 32-bit structure embedded in each vm_page, consisting mostly
of page queue state. The use of a structure makes it easy to store a
snapshot of a page's queue state in a stack variable and use cmpset
loops to update that state without requiring the page lock.

This change merely adds the structure and updates references to atomic
state fields. No functional change intended.

Reviewed by: alc, jeff, kib
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D22650
rm64/pmap.c
nclude/pmap.h
4459aedbdc7ef2215f488ce43f8119620605025d 09-Dec-2019 jhb <jhb@FreeBSD.org> Copy out aux args after the argument and environment vectors.

Partially revert r354741 and r354754 and go back to allocating a
fixed-size chunk of stack space for the auxiliary vector. Keep
sv_copyout_auxargs but change it to accept the address at the end of
the environment vector as an input stack address and no longer
allocate room on the stack. It is now called at the end of
copyout_strings after the argv and environment vectors have been
copied out.

This should fix a regression in r354754 that broke the stack alignment
for newer Linux amd64 binaries (and probably broke Linux arm64 as
well).

Reviewed by: kib
Tested on: amd64 (native, linux64 (only linux-base-c7), and i386)
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D22695
inux/linux_sysvec.c
e8e1c994a8eb08ae92d766f26de49589b4da27f8 09-Dec-2019 manu <manu@FreeBSD.org> arm64: Use the kenv provided by loader

Otherwise we have an empty kenv a likely cannot boot.

Submitted by: kevans
MFC after: 1 month
X-MFC-With: 355487
rm64/machdep_boot.c
1f44036356a26cb340a6a0a4b195d1bde9c62921 07-Dec-2019 mmel <mmel@FreeBSD.org> Add support for booting kernel directly from U-Boot using booti command.

In some cases, like is locked bootstrap or device's inability to boot from
removable media, we cannot use standard boot sequence and is necessary to
boot kernel directly from U-Boot.

Discussed with: jhibbits
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D13861
rm64/locore.S
rm64/machdep.c
rm64/machdep_boot.c
nclude/machdep.h
dfa2e15cbee94059564bd5ac5d6225fe20a0fb01 06-Dec-2019 brooks <brooks@FreeBSD.org> sysent: Reduce duplication and improve readability.

Use the power of variable to avoid spelling out source and generated
files too many times. The previous Makefiles were hard to read, hard to
edit, and badly formatted.

Reviewed by: kevans, emaste
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22714
inux/Makefile
17acaa87d380e51e9cbe930a5dee13e084f30a79 06-Dec-2019 loos <loos@FreeBSD.org> Add the SPI driver for the Marvell Armada 37x0 SoC.

Interrupt based driver, implements SPI mode and clock configuration.

Tested on espressobin and SG-3200.

Sponsored by: Rubicon Communications, LLC (Netgate)
onf/GENERIC
5ec50a41dd9c455ed0ee037b357a5d896418c929 05-Dec-2019 alc <alc@FreeBSD.org> On a context switch, handle the possibility that the old thread was
preempted after an "ic" or "tlbi" instruction but before it performed a
"dsb" instruction. The "ic" and "tlbi" instructions have unusual
synchronization requirements. If the old thread migrates to a new
processor, its completion of a "dsb" instruction on that new processor does
not guarantee that the "ic" or "tlbi" instructions performed on the old
processor have completed.

This issue is not restricted to the kernel. Since locore.S sets the UCI bit
in SCTLR, user-space programs can perform "ic ivau" instructions (as well as
some forms of the "dc" instruction).

Reviewed by: andrew, kib, markj, mmel
X-MFC with: r355145
Differential Revision: https://reviews.freebsd.org/D22622
rm64/pmap.c
1635b069c01a6ce812988517c8da5a6b476d6a04 05-Dec-2019 loos <loos@FreeBSD.org> Add the I2C driver for the Armada 37x0.

This controller is a bit tricky as the STOP condition must be indicated in
the last tranferred byte, some devices will not like the repeated start
behavior of this controller. A proper fix to this issue is in the works.

This driver works in polling mode, can be used early in the boot (required
in some cases).

Tested on espressobin/SG-1100 and the SG-3200.

Obtained from: pfSense
Sponsored by: Rubicon Communications, LLC (Netgate)
onf/GENERIC
a476ba06d5016dd96ede66d0a2bb7c2a73d19810 04-Dec-2019 imp <imp@FreeBSD.org> Regularize my copyright notice

o Remove All Rights Reserved from my notices
o imp@FreeBSD.org everywhere
o regularize punctiation, eliminate date ranges
o Make sure that it's clear that I don't claim All Rights reserved by listing
All Rights Reserved on same line as other copyright holders (but not
me). Other such holders are also listed last where it's clear.
nclude/_bus.h
575f514c41f620e7d64c51188c72a90f9099d983 04-Dec-2019 lwhsu <lwhsu@FreeBSD.org> Fix arm64 build after r355373

Sponsored by: The FreeBSD Foundation
inux/linux_sysvec.c
0d8d23a6a3f467016a24a8eea6ca95087ba82581 03-Dec-2019 jhb <jhb@FreeBSD.org> Use uintptr_t instead of register_t * for the stack base.

- Use ustringp for the location of the argv and environment strings
and allow destp to travel further down the stack for the stackgap
and auxv regions.
- Update the Linux copyout_strings variants to move destp down the
stack as was done for the native ABIs in r263349.
- Stop allocating a space for a stack gap in the Linux ABIs. This
used to hold translated system call arguments, but hasn't been used
since r159992.

Reviewed by: kib
Tested on: md64 (amd64, i386, linux64), i386 (i386, linux)
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D22501
rm64/elf32_machdep.c
rm64/machdep.c
loudabi32/cloudabi32_sysvec.c
loudabi64/cloudabi64_sysvec.c
inux/linux_sysvec.c
d355b145cd980f7acb1e064c72129553f568dade 03-Dec-2019 manu <manu@FreeBSD.org> Remove "all rights reserved" from copyright for the file I own.
Some of the files have both me and Jared McNeill and he gave me
permission to remove it from his files too.
ockchip/clk/rk3328_cru.c
ockchip/clk/rk_clk_armclk.c
ockchip/clk/rk_clk_armclk.h
ockchip/clk/rk_clk_composite.c
ockchip/clk/rk_clk_composite.h
ockchip/clk/rk_clk_gate.h
ockchip/clk/rk_clk_pll.c
ockchip/clk/rk_clk_pll.h
ockchip/clk/rk_cru.c
ockchip/clk/rk_cru.h
ockchip/if_dwc_rk.c
ockchip/rk805.c
ockchip/rk805reg.h
ockchip/rk_gpio.c
ockchip/rk_grf.c
ockchip/rk_pinctrl.c
5422fbcf56d63c01f8fc4257b9b3d7d777e7b6f1 03-Dec-2019 manu <manu@FreeBSD.org> arm64: rockchip: rl3399: Remove the ability to put the PLL in normal mode at boot

RK3399 PLLs have three modes :
- Normal, where they behave normally and their freq is calculated based on
the registers values.
- Slow, where the PLL freq is 24Mhz (well, the external oscillator).
- Deep Slow, used for suspend where the freq is 32Khz.

We used to put every CPU related PLL in normal mode but it can cause problem
if the firmware didn't setup the clocks register correctly.
And even if it did but left the pll in slow or deep slow mode that might be
because the PMIC suppling voltage for the CPU haven't been configured yet
and we cannot do that at this point.
So remove the ability to set PLLs to normal mode at boot to avoid any problems.
ockchip/clk/rk3399_cru.c
ockchip/clk/rk_clk_pll.c
ockchip/clk/rk_clk_pll.h
f611a81b2f8fc95b8c3000be6309528dd983f058 02-Dec-2019 jhibbits <jhibbits@FreeBSD.org> revert r354714 "Boot arm64 kernel using booti command from U-boot."

After discussing with mmel@, it was clear this is insufficient to address
all the needs. mmel@ will commit his original patch, from
https://reviews.freebsd.org/D13861, and the additions needed from r354714
will be made afterward.

Requested by: mmel
Sponsored by: Juniper Networks, Inc.
rm64/locore.S
rm64/machdep.c
rm64/machdep_boot.c
onf/GENERIC
nclude/machdep.h
8e4a0f45834bd03049e92c3548804eed378957b0 29-Nov-2019 andrew <andrew@FreeBSD.org> Use the VM_MEMATTR macros to describe the MAIR offsets.

Remove the duplicate macros that defined a subset of the VM_MEMATTR values.
While here use VM_MEMATTR macros when filling in the MAIR register.

Reviewed by: alc, markj
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22241
rm64/locore.S
rm64/pmap.c
fe9f77299e2d3453885389122e64d14705b02203 28-Nov-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk3328: Add TSADC clocks

Add the clocks so we can use the rk_tsadc driver to monitor
the cpu temperature.
ockchip/clk/rk3328_cru.c
b307a2a06ebd0c0b33ba645d60ef7ee16436c857 28-Nov-2019 manu <manu@FreeBSD.org> arm64: rockchip: tsadc: Do not free the sysctl context is it wasn't created

MFC after: 3 weeks
X-MFC-With: r355173
ockchip/rk_tsadc.c
0f42a93892ed771a206fd3d0e95754d770208e60 28-Nov-2019 mmel <mmel@FreeBSD.org> Add driver for temperature sensors found in RK32898, RK3328 and RK3399 SoC's.

MFC after: 3 weeks
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D22442
ockchip/rk_tsadc.c
ockchip/rk_tsadc_if.m
48b79da9f8fa83e4084c35092104dbc026fed579 27-Nov-2019 alc <alc@FreeBSD.org> There is no reason why we need to pin the underlying thread to its current
processor in pmap_invalidate_{all,page,range}(). These functions are using
an instruction that broadcasts the TLB invalidation to every processor, so
even if a thread migrates in the middle of one of these functions every
processor will still perform the required TLB invalidations.

Reviewed by: andrew, markj
MFC after: 10 days
Differential Revision: https://reviews.freebsd.org/D22502
rm64/pmap.c
06f07aa5cc34a5ad4e9248133e5aa9d60a242d89 26-Nov-2019 mmel <mmel@FreeBSD.org> Finish implementation of RK3299 clocks.
- implement of all but mmc clocks. MMC clocks will be added later by own commit.
- use 'link' clock type for external clocks.
- use macros for initialization of structure's named members.

MFC after: 3 weeks
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D22441
ockchip/clk/rk3399_cru.c
ockchip/clk/rk3399_cru_dt.h
ockchip/clk/rk_cru.h
54618cdc3b58b9a582d804e05d48dd0c384bdccc 23-Nov-2019 kevans <kevans@FreeBSD.org> arm64/NOTES: add SOC_BRCM_BCM2838

This should have been done back when it was added, but it was not. It only
really adds an extra entry for memory mapping bits in bcm2835_vcbus.c, so
nothing too extensive yet.
onf/NOTES
e95c204297607d3e81aa2d8d964fc84a0015b333 21-Nov-2019 andrew <andrew@FreeBSD.org> Add kcsan_md_unsupported from NetBSD.

It's used to ignore virtual addresses that may have a different physical
address depending on the CPU.

Sponsored by: DARPA, AFRL
nclude/csan.h
6e5970c8f44eec5f975888cbfafd1ee5fb5bf4b3 21-Nov-2019 andrew <andrew@FreeBSD.org> Port the NetBSD KCSAN runtime to FreeBSD.

Update the NetBSD Kernel Concurrency Sanitizer (KCSAN) runtime to work in
the FreeBSD kernel. It is a useful tool for finding data races between
threads executing on different CPUs.

This can be enabled by enabling KCSAN in the kernel config, or by using the
GENERIC-KCSAN amd64 kernel. It works on amd64 and arm64, however the later
needs a compiler change to allow -fsanitize=thread that KCSAN uses.

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22315
rm64/bus_machdep.c
rm64/copystr.c
rm64/machdep.c
rm64/mp_machdep.c
onf/GENERIC
nclude/atomic.h
nclude/bus.h
nclude/csan.h
2b8635a91f7b9702fe8ae4b32848dac9c63f0b84 20-Nov-2019 alc <alc@FreeBSD.org> Until every possible root cause for an "invalid ASID" assertion failure is
resolved, assign every pmap a valid ASID when it is first initialized.
rm64/pmap.c
2cac79878c1edf70b54d57646431f7470a40691e 19-Nov-2019 alc <alc@FreeBSD.org> Achieve two goals at once: (1) Avoid an unnecessary broadcast TLB
invalidation in reclaim_pv_chunk(). (2) Prevent an "invalid ASID" assertion
failure in reclaim_pv_chunk(). The detailed explanation for this change is
provided by r354792.

X-MFC with: r354792
rm64/pmap.c
90f5f26ff0564ccfea92dbd730fc0b63afa6bd6a 19-Nov-2019 andrew <andrew@FreeBSD.org> Return 0 from ptrace_set_pc as it now completes successfully.

Sponsored by: DARPA, AFRL
rm64/machdep.c
8327e3d8431ab761b1ba634f6bbff97137d6d1e7 19-Nov-2019 andrew <andrew@FreeBSD.org> Allow ptrace to set the probram counter on arm64.

Sponsored by: DARPA, AFRL
rm64/machdep.c
0fe49e2eeda2d87ef0cb78ea7adba3f69fcc0b26 19-Nov-2019 andrew <andrew@FreeBSD.org> Fix the definition of bus_space_read_stream_8 on arm64.

This is currently unused, however will be when the Kernel Concurrency
Sanitizer (KCSAN) is imported from NetBSD.

Sponsored by: DARPA, AFRL
nclude/bus.h
60027726b9e566af064789b976c245b18a5b6f07 18-Nov-2019 kevans <kevans@FreeBSD.org> Convert in-tree sysent targets to use new makesyscalls.lua

flua is bootstrapped as part of the build for those on older
versions/revisions that don't yet have flua installed. Once upgraded past
r354833, "make sysent" will again naturally work as expected.

Reviewed by: brooks
Differential Revision: https://reviews.freebsd.org/D21894
inux/Makefile
81f62ee15e1ce883c7d2ffc22b18422c08483012 18-Nov-2019 jhb <jhb@FreeBSD.org> Check for errors from copyout() and suword*() in sv_copyout_args/strings.

Reviewed by: brooks, kib
Tested on: amd64 (amd64, i386, linux64), i386 (i386, linux)
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D22401
inux/linux_sysvec.c
8d166fea2aabaa764e5f8076d499f5f4711cf35b 18-Nov-2019 markj <markj@FreeBSD.org> Set MALLOC_DEBUG_MAXZONES=1 in GENERIC-NODEBUG configurations.

The purpose of this option is to make it easier to track down memory
corruption bugs by reducing the number of malloc(9) types that might
have recently been associated with a given chunk of memory. However, it
increases fragmentation and is disabled in release kernels.

MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
onf/GENERIC-NODEBUG
609735ee02f4cfd4aaca3578d8f2561f32291cbf 18-Nov-2019 markj <markj@FreeBSD.org> Implement vm.pmap.kernel_maps for arm64.

Reviewed by: alc
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22142
rm64/pmap.c
efc9eaa3cb7822632409aee772f18080b69c0f41 18-Nov-2019 markj <markj@FreeBSD.org> Let arm64 pmap_qenter() and pmap_kenter() unconditionally set NX.

As on amd64, there is no need for mappings created by these functions to
be executable.

Reviewed by: alc, andrew
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22141
rm64/pmap.c
f087196e158e1f454306fa8f548aefd7b8073cbc 17-Nov-2019 alc <alc@FreeBSD.org> Achieve two goals at once: (1) Avoid an unnecessary broadcast TLB
invalidation in pmap_remove_all(). (2) Prevent an "invalid ASID" assertion
failure in pmap_remove_all().

The architecture definition specifies that the TLB will not cache mappings
that don't have the "AF" bit set, so pmap_remove_all() needn't issue a TLB
invalidation for mappings that don't have the "AF" bit set.

We allocate ASIDs lazily. Specifically, we don't allocate an ASID for a
pmap until we are activating it. Now, consider what happens on a fork().
Before we activate the child's pmap, we use pmap_copy() to copy mappings
from the parent's pmap to the child's. These new mappings have their "AF"
bits cleared. Suppose that the page daemon decides to reclaim a page that
underlies one of these new mappings. Previously, the pmap_invalidate_page()
performed by pmap_remove_all() on a mapping in the child's pmap would fail
an assertion because that pmap hasn't yet been assigned an ASID. However,
we don't need to issue a TLB invalidation for such mappings because they
can't possibly be cached in the TLB.

Reported by: bob prohaska <fbsd@www.zefox.net>
Reviewed by: markj
MFC after: 1 week
X-MFC-before: r354286
Differential Revision: https://reviews.freebsd.org/D22388
rm64/pmap.c
ad77a7b3cc4cc606899edc75f831fcc0220f54c8 15-Nov-2019 jhb <jhb@FreeBSD.org> Use a sv_copyout_auxargs hook in the Linux ELF ABIs.

Reviewed by: emaste
Tested on: amd64 (linux64 only), i386
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D22356
inux/linux_sysvec.c
3f50cb74914f1f9f9cf2930d182207512a7033fc 15-Nov-2019 jhb <jhb@FreeBSD.org> Add a sv_copyout_auxargs() hook in sysentvec.

Change the FreeBSD ELF ABIs to use this new hook to copyout ELF auxv
instead of doing it in the sv_fixup hook. In particular, this new
hook allows the stack space to be allocated at the same time the auxv
values are copied out to userland. This allows us to avoid wasting
space for unused auxv entries as well as not having to recalculate
where the auxv vector is by walking back up over the argv and
environment vectors.

Reviewed by: brooks, emaste
Tested on: amd64 (amd64 and i386 binaries), i386, mips, mips64
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D22355
rm64/elf32_machdep.c
rm64/elf_machdep.c
399f6e87ffb7be9caf9aa5fb99c39df0efeb33aa 14-Nov-2019 jhibbits <jhibbits@FreeBSD.org> Boot arm64 kernel using booti command from U-boot.

Summary:
Boot arm64 kernel using booti command from U-boot. booti can relocate initrd
image into higher ram addresses, therefore align the initrd load address to 1GiB
and create VA = PA map for it. Create L2 pagetable entries to copy the initrd
image into KVA.
(parts of the code in https://reviews.freebsd.org/D13861 was referred and used
as appropriate)

Submitted by: Siddharth Tuli <siddharthtuli_gmail.com>
Reviewed by: manu
Sponsored by: Juniper Networks, Inc
Differential Revision: https://reviews.freebsd.org/D22255
rm64/locore.S
rm64/machdep.c
rm64/machdep_boot.c
onf/GENERIC
nclude/machdep.h
b1193a1f2cf1b52b99f559cb12a631f4b5c15790 14-Nov-2019 kevans <kevans@FreeBSD.org> arm64: busdma_bounce: fix BUS_DMA_ALLOCNOW for non-paged aligned sizes

For any size that isn't page-aligned, we end up not pre-allocating enough
for a single mapping because we truncate the size instead of rounding up to
make sure the last bit is accounted for, leaving us one page shy of what we
need to fulfill a request.

Differential Revision: https://reviews.freebsd.org/D22288
rm64/busdma_bounce.c
b4dc5956c19c34863aa775faacbd516f1dc65c6e 10-Nov-2019 alc <alc@FreeBSD.org> Eliminate a redundant pmap_load() from pmap_remove_pages().

There is no reason why the pmap_invalidate_all() in pmap_remove_pages()
must be performed before the final PV list lock release. Move it past
the lock release.

Eliminate a stale comment from pmap_page_test_mappings(). We implemented
a modified bit in r350004.

MFC after: 1 week
rm64/pmap.c
127577020f2bf81b01ef13f29d2d1c5771ae7ea7 10-Nov-2019 kevans <kevans@FreeBSD.org> arm64: add SOC_BRCM_BCM2838, build it in GENERIC

BCM2838/BCM2711 is the Raspberry Pi 4, which we will soon be able to boot
on once some ports bits are worked out.
onf/GENERIC
fd01ead7e42b74378bc9cc45be579faa3f5978ae 08-Nov-2019 mmel <mmel@FreeBSD.org> Remove explicit declaration of rk_clk_fract_set_freq() function
forgotten in r354556.

MFC after: 3 weeks
MFC with: r354556
Noticed by: manu
ockchip/clk/rk_clk_fract.c
6f228baa3477cbd72f800078c92f5a1efb427549 08-Nov-2019 mmel <mmel@FreeBSD.org> Tidy up Rockchip composite clock.
- add support for log2 based dividers
- use proper write mask when writing to divider register

MFC after: 3 weeks
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D22283
ockchip/clk/rk_clk_composite.c
ockchip/clk/rk_clk_composite.h
5bdc25e51aabdc36b9658766de016af070f9c192 08-Nov-2019 mmel <mmel@FreeBSD.org> Enhance Rockchip clocks implementation.
- add support for fractional dividers
- allow to declare fixed and linked clock

MFC after: 3 weeks
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D22282
ockchip/clk/rk_clk_fract.c
ockchip/clk/rk_clk_fract.h
ockchip/clk/rk_cru.c
ockchip/clk/rk_cru.h
144319cf6095a919656a80faac4f8e5450b46e24 08-Nov-2019 mmel <mmel@FreeBSD.org> Cleanup Rockchip clocks implementation.
- style
- unify dprinf defines
- make dprinf's 32-bit compatible
Not a functional change.

MFC after: 3 weeks
Reviewed by: manu, imp
Differential Revision: https://reviews.freebsd.org/D22281
ockchip/clk/rk_clk_armclk.c
ockchip/clk/rk_clk_composite.c
ockchip/clk/rk_clk_gate.c
ockchip/clk/rk_clk_mux.c
ockchip/clk/rk_clk_pll.c
ockchip/clk/rk_cru.c
c598107ad1d7e4cc7e7288902090aab5f125d340 07-Nov-2019 andrew <andrew@FreeBSD.org> Add more 8 and 16 bit variants of the the atomic(9) functions on arm64.

These are direct copies of the 32 bit functions, adjusted ad needed.
While here fix atomic_fcmpset_16 to use the valid load and store exclusive
instructions.

Sponsored by: DARPA, AFRL
nclude/atomic.h
ff2ae9c1e6d5ab6f0ab887e6ec7da975449ce9ea 06-Nov-2019 manu <manu@FreeBSD.org> regulator: Add regulator_check_voltage function

This function will call the regnode_check_voltage method for a given regulator
and check if the desired voltage in reachable by it.
Also adds a default method that check the std_param and which should be enough
for most regulators and add it as the method for axp* rk805 and fixed regulators.

Reviewed by: mmel
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22260
ockchip/rk805.c
c63bb46164024f99925c98b7e8cd8f0034a8eb88 04-Nov-2019 emaste <emaste@FreeBSD.org> arm64 linuxulator: default to RW stack (no X)

This matches Linux's default arm64 data / stack permissions.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation
inux/linux_sysvec.c
3de5e49155f9706ffba0cdfb30d79f2cec0f25af 03-Nov-2019 andrew <andrew@FreeBSD.org> Move the struct debug_monitor_state out of _KERNEL.

Some userland libraries incude machine/pcb.h and this needs the full
definition of struct debug_monitor_state. To allow this to work move
stuct debug_monitor_state out of the _KERNEL guard.

Sponsored by: DARPA, AFRL
nclude/debug_monitor.h
2a4c328a15cf201b7e90d698835fe69a6e8a1fdf 03-Nov-2019 alc <alc@FreeBSD.org> Utilize ASIDs to reduce both the direct and indirect costs of context
switching. The indirect costs being unnecessary TLB misses that are
incurred when ASIDs are not used. In fact, currently, when we perform a
context switch on one processor, we issue a broadcast TLB invalidation that
flushes the TLB contents on every processor.

Mark all user-space ("ttbr0") page table entries with the non-global flag so
that they are cached in the TLB under their ASID.

Correct an error in pmap_pinit0(). The pointer to the root of the page
table was being initialized to the root of the kernel-space page table
rather than a user-space page table. However, the root of the page table
that was being cached in process 0's md_l0addr field correctly pointed to a
user-space page table. As long as ASIDs weren't being used, this was
harmless, except that it led to some unnecessary page table switches in
pmap_switch(). Specifically, other kernel processes besides process 0 would
have their md_l0addr field set to the root of the kernel-space page table,
and so pmap_switch() would actually change page tables when switching
between process 0 and other kernel processes.

Implement a workaround for Cavium erratum 27456 affecting ThunderX machines.
(I would like to thank andrew@ for providing the code to detect the affected
machines.)

Address integer overflow in the definition of TCR_ASID_16.

Setup TCR according to the PARange and ASIDBits fields from
ID_AA64MMFR0_EL1. Previously, TCR_ASID_16 was unconditionally set.

Modify build_l1_block_pagetable so that lower attributes, such as ATTR_nG,
can be specified as a parameter.

Eliminate some unused code.

Earlier versions were tested to varying degrees by: andrew, emaste, markj

MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D21922
rm64/cpu_errata.c
rm64/cpufunc_asm.S
rm64/efirt_machdep.c
rm64/genassym.c
rm64/locore.S
rm64/machdep.c
rm64/mp_machdep.c
rm64/pmap.c
rm64/vm_machdep.c
nclude/armreg.h
nclude/cpufunc.h
nclude/pcpu.h
nclude/pmap.h
nclude/proc.h
a7095edbfd8a817cfaa5c4991dcee9823542b220 03-Nov-2019 andrew <andrew@FreeBSD.org> Add support for setting hardware breakpoints from ptrace on arm64.

Implement get/fill_dbregs on arm64. This is used by ptrace with the
PT_GETDBREGS and PT_SETDBREGS requests. It allows userspace to set hardware
breakpoints.

The struct dbreg is based on Linux to ease adding hardware breakpoint
support to debuggers.

Reviewed by: jhb
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22195
rm64/debug_monitor.c
rm64/exception.S
rm64/identcpu.c
rm64/machdep.c
rm64/trap.c
nclude/armreg.h
nclude/pcb.h
nclude/reg.h
f83862b781a53e4ef11668b7d34e56ea24e38962 31-Oct-2019 mw <mw@FreeBSD.org> Fix pmap_change_attr() on arm64 to allow KV addresses

Altough in the comment above the pmap_change_attr() it was mentioned
that VA could be in KV or DMAP memory space. However,
pmap_change_attr_locked() was accepting only the values inside the DMAP
memory range.

To fix that, the condition check was changed so also the va inside the
KV memory range would be accepted.

The sample use case that wasn't supported is the PCI Device that has the
BAR which should me mapped with the Write Combine attribute - for
example BAR2 of the ENA network controller on the A1 instances on AWS.

Tested on A1 AWS instance and changed ENA BAR2 mapped resource to be
write-combined memory region.

Differential Revision: https://reviews.freebsd.org/D22055
MFC after: 2 weeks
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
rm64/pmap.c
bea520b3e4d8113a351597495798764a88102f06 31-Oct-2019 markj <markj@FreeBSD.org> Fix a typo in r353895.

Reported by: andrew
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
rm64/machdep.c
adeaebe505b91aea5b84dbcb50e891045263485b 30-Oct-2019 andrew <andrew@FreeBSD.org> Set the userspace execute never bit on kernel mappings.

Arm64 allows us to create execute only mappings. To make sure userspace is
unable to accidentally execute kernel code set the user execute never
bit in the kernel page tables.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/locore.S
rm64/pmap.c
5943536ffeae252bd278790437e6596d0157fe6e 30-Oct-2019 jhibbits <jhibbits@FreeBSD.org> ARM64: Treat alignment faults as bus errors

Summary:
ARM64 currently treats all data abort exceptions as page faults. This
can cause infinite loops on non-page fault faults, such as alignment faults.

Since kernel-side alignment faults should be avoided, this adds support directly
to the el0 fault handler, instead of the data_abort() handler.

Test Plan: Tested on rpi3, with a misaligned ldm test.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D22133
rm64/trap.c
50cea160e65f7482b078aaa3efc773a0e52529f9 30-Oct-2019 andrew <andrew@FreeBSD.org> Allow exceptions to be masked when in userspace

We may want to mask exceptions when in userspace. This was previously
impossible as threads are created with all exceptions unmasked and
signals expected userspace to mask any. Fix these by copying the
mask state on thread creation and allow exceptions to be masked on
signal return, as long as they don't change.

Sponsored by: DARPA, AFRL
rm64/machdep.c
rm64/vm_machdep.c
nclude/armreg.h
a8fc8da0637d43f8b27cd615f5c3270219f6c083 30-Oct-2019 andrew <andrew@FreeBSD.org> Allow the userspace ID register fields to be read from the kernel

To allow consistent values to be used in both the kernel and userspace
create a function for these to be read from the kernel. They use a newly
created macro with the name of the ID register to read. For now there is
redundant information in the user_regs array as it still holds the CRm and
Op2 values, however this will be fixed in a later change.

This will be used by ptrace to allow hardware breakpoints in userspace.

Sponsored by: DARPA, AFRL
rm64/identcpu.c
nclude/armreg.h
nclude/undefined.h
74f88ec2ba5ba2b10ce7952f1ef97ac540a52633 30-Oct-2019 andrew <andrew@FreeBSD.org> Use a lowercase name for arm64 special registers so they don't conflict
with macros of the same name.

Sponsored by: DARPA, AFRL
rm64/debug_monitor.c
rm64/freebsd32_machdep.c
rm64/gic_v3_reg.h
rm64/machdep.c
rm64/trap.c
4b3c23bef9daf8a4aa919b70ecdb3d3ab4ed9fc8 30-Oct-2019 andrew <andrew@FreeBSD.org> Move the MRS instruction decode macros to armreg.h

These instructions are used to access the registers described in armreg.h,
and will be used in a future change to create a per-register identification
macro.

Sponsored by: DARPA, AFRL
nclude/armreg.h
nclude/undefined.h
626419b403883acddb640b0dd1a6491e608410de 30-Oct-2019 andrew <andrew@FreeBSD.org> Update the debug monitor handling to work after userspace has started

The debug monitor register state is now stored in a struct and updated
when required. Currently there is only a kernel state, however a
per-process state will be added in a future change.

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22128
rm64/db_trace.c
rm64/debug_monitor.c
rm64/exception.S
rm64/mp_machdep.c
nclude/debug_monitor.h
nclude/pcpu.h
a59d8dfbb2752adf82a317aa64feea46abcace48 30-Oct-2019 andrew <andrew@FreeBSD.org> Use an array of handlers in the data and instruction aborts

Previously we would call data_abort on all data and instruction aborts
however this is incorrect for most abort types. Move to use an array
of function pointers to allow for more handlers to be easily added.

Reviewed by: jhibbits
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22170
rm64/trap.c
19bdf709addf3d141bbeed4f1b7d86c0db4b8b05 30-Oct-2019 andrew <andrew@FreeBSD.org> Add two files missed in r354170

Sponsored by: DARPA, AFRL
rm64/identcpu.c
rm64/machdep.c
d4cdc59b4194a31f316ce9dc5d67bd8e0b4c77f3 30-Oct-2019 andrew <andrew@FreeBSD.org> Rename the macros to extract a single arm64 ID field.

Because of the previous naming scheme the old ID_AA64PFR0_EL1 macro
collided with a potential macro for the register of the same name. To fix
this collision rename these macros.

Sponsored by: DARPA, AFRL
rm64/elf32_machdep.c
nclude/armreg.h
44dd93b9cb28d88453a20fb806313d6bde603d4c 29-Oct-2019 manu <manu@FreeBSD.org> arm64: rockchip: typec_phy: Rename timeout to retry

Declare retry in the function scope.
Rename it to retry as there is a timeout function which was
causing to code to compile.

Reported by: jhibbits
MFC after: 1 month
X-MFC-WITH: r354089
ockchip/rk_typec_phy.c
7c8bef6beab6ff37d47625517aa3752b15e56188 25-Oct-2019 gonzo <gonzo@FreeBSD.org> arm64: rk3399: add SPI driver and include it in GENERIC config

SPI driver for Rockchip's RK3399 SoC. Implements PIO mode, CS selection,
SPI mode and frequency configuration.

Reviewed by: manu
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22148
onf/GENERIC
ockchip/rk_spi.c
a642819db84efe6e9d2d0ca6ca2acac64ce0b939 25-Oct-2019 gonzo <gonzo@FreeBSD.org> arm64: rk3399: Add clock and gate for SPI clocks

MFC after: 1 month
ockchip/clk/rk3399_cru.c
2fc0413f87ad1d5926d40b21b4310de95a5860a0 25-Oct-2019 peterj <peterj@FreeBSD.org> Fix use of uninitialised variable.

The RK805 regs array was being allocated before it's required size was
known, causing the driver to use memory it didn't own. That memory
was subsequently allocated and used elsewhere causing later fatal data
aborts in rk805_map().

Whilst I'm here, add a sanity check to catch unsupported PMICs (this
shouldn't ever get hit because the probe should have failed).

Reviewed by: manu
MFC after: 1 week
Sponsored by: Google
ockchip/rk805.c
b2e908a72541782369ede4da4cb427095bdd0d63 25-Oct-2019 manu <manu@FreeBSD.org> arm64: rockchip: Add RK3399 TypeC phy driver

This is a driver for the USB3 PHY present in the RK3399.
While the phy support DP (Display Port) the driver doesn't has we have
no driver to test this with for now.
All the lane and pll configuration is just magic values from rockchip.
While the manual have some info on those registers it's really hard to
understand how to calculate those values (if there is a way).

MFC after: 1 month
onf/GENERIC
ockchip/rk_typec_phy.c
77f7cf2489cf5d2b087c6388aa67a41fd7b921ee 25-Oct-2019 manu <manu@FreeBSD.org> arm64: rockchip: Add rk_dwc3 driver

This is a simplebus like driver that attaches the dwc3 child node and
enable the clocks needed for the module.

MFC after: 1 month
onf/GENERIC
ockchip/rk_dwc3.c
2ce78127ee6ed9e31a6a92cbcd2e96e600cb2f79 25-Oct-2019 manu <manu@FreeBSD.org> arm64: rk3399: Add clock and gate for usb3 clocks

MFC after: 1 month
ockchip/clk/rk3399_cru.c
efe7a1e01f59079007f4c42d4b7f1ce513bf67f6 25-Oct-2019 andrew <andrew@FreeBSD.org> Remove the arm4 ID register masks, they are not needed after r353641.

Sponsored by: DARPA, AFRL
nclude/armreg.h
7a2f3d8b4102bcb0007053ea83b8b9d2d383bcb7 25-Oct-2019 andrew <andrew@FreeBSD.org> Make special register names lowercase so they don't conflict with future
ID register macros.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL
rm64/identcpu.c
d0f324ec6a41d9ecaf43348b4fbd4235b7b58311 23-Oct-2019 emaste <emaste@FreeBSD.org> arm64: enable options NUMA in GENERIC

As with amd64 NUMA is required for reasonable operation on big-iron
arm64 systems and is expected to have no significant impact on small
systems. Enable it now for wider testing in advance of FreeBSD 13.0.

You can use the 'vm.ndomains' sysctl to see if multiple domains are in
use - for example (from Cavium/Marvell ThunderX2):

# sysctl vm.ndomains
vm.ndomains: 2

No objection: manu
Sponsored by: The FreeBSD Foundation
onf/GENERIC
a0c18fcde6182d7f5c3291e4d214e430917be89d 23-Oct-2019 andrew <andrew@FreeBSD.org> Stop enabling interrupts when reentering kdb on arm64

When we raise a data abort from the kernel we need to enable interrupts,
however we shouldn't be doing this when in the kernel debugger. In this
case interrupts can lead to a further panic as they don't expect to be
run from such a context.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL
rm64/trap.c
9582ed1083377d2d9984e9158c6fde1c51f9e441 22-Oct-2019 markj <markj@FreeBSD.org> Apply r353893 to arm64.

Reported by: Jenkins (hardware CI lab)
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
rm64/locore.S
rm64/machdep.c
189f4bb0a262a54e6b88a95a9c22de1321f118f6 20-Oct-2019 mmel <mmel@FreeBSD.org> Update Armada 8k drivers to cover newly imported DT and latest changes
in simple multifunction driver.
- follow interrupt changes in DT. Split old ICU driver to function oriented
parts and add drivers for newly defined parts (system error interrupts).
- Many drivers are children of simple multifunction driver. But after r349596
simple MF driver doesn't longer exports memory resources, and all children
must use syscon interface to access their registers. Adapt affected
drivers to this fact.

MFC after: 3 weeks
onf/GENERIC
ad16acfadb9196e6d3f626287de2461710b871bb 18-Oct-2019 yuripv <yuripv@FreeBSD.org> linux: futex_mtx should follow futex_list

Move futex_mtx to linux_common.ko for amd64 and aarch64 along
with respective list/mutex init/destroy.

PR: 240989
Reported by: Alex S <iwtcex@gmail.com>
inux/linux_sysvec.c
f3a0ee41db2b870d4a81d36ad4357bce1b1b840c 17-Oct-2019 cem <cem@FreeBSD.org> Split out a more generic debugnet(4) from netdump(4)

Debugnet is a simplistic and specialized panic- or debug-time reliable
datagram transport. It can drive a single connection at a time and is
currently unidirectional (debug/panic machine transmit to remote server
only).

It is mostly a verbatim code lift from netdump(4). Netdump(4) remains
the only consumer (until the rest of this patch series lands).

The INET-specific logic has been extracted somewhat more thoroughly than
previously in netdump(4), into debugnet_inet.c. UDP-layer logic and up, as
much as possible as is protocol-independent, remains in debugnet.c. The
separation is not perfect and future improvement is welcome. Supporting
INET6 is a long-term goal.

Much of the diff is "gratuitous" renaming from 'netdump_' or 'nd_' to
'debugnet_' or 'dn_' -- sorry. I thought keeping the netdump name on the
generic module would be more confusing than the refactoring.

The only functional change here is the mbuf allocation / tracking. Instead
of initiating solely on netdump-configured interface(s) at dumpon(8)
configuration time, we watch for any debugnet-enabled NIC for link
activation and query it for mbuf parameters at that time. If they exceed
the existing high-water mark allocation, we re-allocate and track the new
high-water mark. Otherwise, we leave the pre-panic mbuf allocation alone.
In a future patch in this series, this will allow initiating netdump from
panic ddb(4) without pre-panic configuration.

No other functional change intended.

Reviewed by: markj (earlier version)
Some discussion with: emaste, jhb
Objection from: marius
Differential Revision: https://reviews.freebsd.org/D21421
onf/GENERIC
84cd531f967717b727ba20cc2543e71d1de993ff 16-Oct-2019 markj <markj@FreeBSD.org> Remove page locking from pmap_mincore().

After r352110 the page lock no longer protects a page's identity, so
there is no purpose in locking the page in pmap_mincore(). Instead,
if vm.mincore_mapped is set to the non-default value of 0, re-lookup
the page after acquiring its object lock, which holds the page's
identity stable.

The change removes the last callers of vm_page_pa_tryrelock(), so
remove it.

Reviewed by: kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21823
rm64/pmap.c
53cf0bb146ffcf4ce8fc8cde7c591fd1a63f929a 16-Oct-2019 andrew <andrew@FreeBSD.org> Use tables to store the information to decode the arm64 ID registers.

Arm updates these with each new architecture revision. To help keep them
updated use a collection of tables to hold the needed information to
decode these registers.

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D22020
rm64/identcpu.c
50eb2e428851c1068a16e3eef88091e946470ef4 15-Oct-2019 jeff <jeff@FreeBSD.org> (6/6) Convert pmap to expect busy in write related operations now that all
callers hold it.

This simplifies pmap code and removes a dependency on the object lock.

Reviewed by: kib, markj
Tested by: pho
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D21596
rm64/pmap.c
0a6e7a4266bb7087807243dc8a4e55991f4cb8be 15-Oct-2019 jeff <jeff@FreeBSD.org> (3/6) Add a shared object busy synchronization mechanism that blocks new page
busy acquires while held.

This allows code that would need to acquire and release a very large number
of page busy locks to use the old mechanism where busy is only checked and
not held. This comes at the cost of false positives but never false
negatives which the single consumer, vm_fault_soft_fast(), handles.

Reviewed by: kib
Tested by: pho
Sponsored by: Netflix, Intel
Differential Revision: https://reviews.freebsd.org/D21592
rm64/pmap.c
f6952dddfabe61a5af54de91d4f76466c2f73bbd 14-Oct-2019 manu <manu@FreeBSD.org> arm64: Add Synopsys DWC3 driver

This add a driver for the Synopsys DWC3 driver found on multiple SoCs.
It only supports host mode for now.

MFC after: 1 month
onf/GENERIC
9116e61067a9b8e6fdde1345411e916c758d0584 14-Oct-2019 manu <manu@FreeBSD.org> arm64: allwinner: Add aw_dwc3 driver

This is a simplebus like driver that just deal with clocks and resets
and attach the dwc3 child node.

MFC after: 1 month
onf/GENERIC
a6c0ec79f38eda67ef4ed7ea98a4f2dd5b20ca19 14-Oct-2019 manu <manu@FreeBSD.org> arm64: allwinner: Add Allwinner H6 Support

This adds support for H6 SoC.
Add a CCU driver for H6 that support all PLLs and most of the clocks
that we are intersted in for now (i2c, mmc, usb, etc ...)

MFC after: 1 month
onf/GENERIC
12778f78cc1cbdf2f35b99645224a38a536ed8f3 14-Oct-2019 andrew <andrew@FreeBSD.org> Sort the id_aa64*_fields arrays to be in alphanumerical order.

Sponsored by: DARPA, AFRL
rm64/identcpu.c
197e87f234e10bef1c9d7daef5052f79f596ec64 11-Oct-2019 jhibbits <jhibbits@FreeBSD.org> gcore: Add aarch64 32-bit core support

Summary: Add trivial 32-bit arm cores on aarch64 support for gcore. This
doesn't handle fpregs.

Reviewed by: #arm, andrew
Sponsored by: Juniper Networks, Inc
Differential Revision: https://reviews.freebsd.org/D21947
nclude/elf.h
fbed214991013cea5f2812f0e4ae713f2ca6c25e 07-Oct-2019 alc <alc@FreeBSD.org> Eliminate an unused declaration. The variable in question is only defined
and used on sparc64.

MFC after: 1 week
nclude/vmparam.h
d81aec09def159c13a223b63b7b19f3ce0ed8f0c 07-Oct-2019 alc <alc@FreeBSD.org> Eliminate a redundant bzero(). The l0 page table page was already zeroed
by efi_1t1_page().

MFC after: 1 week
rm64/efirt_machdep.c
efd86141b4907260833144013e8297ab63271905 05-Oct-2019 manu <manu@FreeBSD.org> arm64: rockchip: usb2phy: Add set/get mode

We only support host mode so those functions are just added so
we won't panic when generic-{e,o}hci will set the phy to host mode.

MFC after: 1 month
X-MFC-With: r353062
ockchip/rk_usb2phy.c
c2cd689979d2b34f4fb3a76446b96e75ef6643f0 04-Oct-2019 alc <alc@FreeBSD.org> The implementation of arm64_tlb_flushID_SE() was removed from cpufunc_asm.S
in r313347. Eliminate its declaration from this file.

MFC after: 1 week
nclude/cpufunc.h
1fc6d3f31ec26d4fd5a5b7936156056fd655a6e0 03-Oct-2019 jhb <jhb@FreeBSD.org> Remove aw_ehci from NOTES to fix LINT kernel builds after r353063.

Reported by: Jenkins
onf/NOTES
2eef25cd6c253156cc8389961c4249f676c8ac63 03-Oct-2019 manu <manu@FreeBSD.org> allwinner: Remove a10_ehci driver

We have generic-ehci since r353062 so use it.

MFC after: 1 month
X-MFC-With: r353062
onf/GENERIC
c976945339915d2b101cfbd1a24670ecd3e4211d 01-Oct-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk805: Switch to iicdev_{readfrom,writeto}

This simpify the code a bit.
ockchip/rk805.c
b5c0c1bef331f5b612f23160a65ce157a0731972 01-Oct-2019 alc <alc@FreeBSD.org> In short, pmap_enter_quick_locked("user space", ..., VM_PROT_READ) doesn't
work. More precisely, it doesn't set ATTR_AP(ATTR_AP_USER) in the page
table entry, so any attempt to read from the mapped page by user space
generates a page fault. This problem has gone unnoticed because the page
fault handler, vm_fault(), will ultimately call pmap_enter(), which
replaces the non-working page table entry with one that has
ATTR_AP(ATTR_AP_USER) set.

This change reduces the number of page faults during a "buildworld" by
about 19.4%.

Reviewed by: andrew, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21841
rm64/pmap.c
57d7210f04ea131371b7c57fac8325cc0d7f327c 30-Sep-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk_clk_pll: Check mode on recalc

If the pll is in slow or deep slow mode return the correct frequency.
ockchip/clk/rk_clk_pll.c
ad9caa60bb45cbf8d620f5501be7d291cf265e0a 30-Sep-2019 manu <manu@FreeBSD.org> arm64: rockchip: correct reset value

If bit is 0 the reset is not asserted.
Also register our self as a reset provider, this was commented
in r352850

Reported by: mmel
ockchip/clk/rk_cru.c
c1ed6afbc4835ac43353394b5b24b852eea91ba8 28-Sep-2019 manu <manu@FreeBSD.org> arm64: rockchip: Add usb2phy driver

This driver is for the usb phy present on rockchip SoC.
It only support RK3399 and host mode for now.
The driver expose the usb clock needed by the usb controller.
onf/GENERIC
ockchip/rk_usb2phy.c
156f6841477feea66f7b9e470ef5c996e93eefa4 28-Sep-2019 manu <manu@FreeBSD.org> arm64: rockchip: Fix map_gpio

The map_gpio function wasn't correct, the first element is the pin
and not the phandle.
ockchip/rk_gpio.c
d3d27ce52f93645fd46a30785dc11ad35e74162e 28-Sep-2019 manu <manu@FreeBSD.org> arm64: rockchip: Implement resets

Module resets where not implemented when rockchip clocks were commited.
Implement them.
Since all resets registers are contiguous a driver only need to give
the start offset and the number of resets. This avoid to have to declare
every resets.
ockchip/clk/rk3328_cru.c
ockchip/clk/rk3399_cru.c
ockchip/clk/rk3399_pmucru.c
ockchip/clk/rk_cru.c
ockchip/clk/rk_cru.h
6e398455139d30ea0d2b01093d54630c03b1f87d 28-Sep-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk3399: Add usb2 clocks
ockchip/clk/rk3399_cru.c
fe710b3242abb89deab33c466c1911d45dbde795 28-Sep-2019 alc <alc@FreeBSD.org> Eliminate redundant calls to critical_enter() and critical_exit() from
pmap_update_entry(). It suffices that interrupts are blocked.

Reviewed by: andrew, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21753
rm64/pmap.c
d130e737a23b27d7025a6419a1a1ac1d8a836502 27-Sep-2019 markj <markj@FreeBSD.org> Implement pmap_page_is_mapped() correctly on arm64 and riscv.

We must also check for large mappings. pmap_page_is_mapped() is
mostly used in assertions, so the problem was not very noticeable.

Reviewed by: alc
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21824
rm64/pmap.c
nclude/pmap.h
957270782d62c4a6129d9bb270bbcdf59ef67777 27-Sep-2019 kib <kib@FreeBSD.org> Improve MD page fault handlers.

Centralize calculation of signal and ucode delivered on unhandled page
fault in new function vm_fault_trap(). MD trap_pfault() now almost
always uses the signal numbers and error codes calculated in
consistent MI way.

This introduces the protection fault compatibility sysctls to all
non-x86 architectures which did not have that bug, but apparently they
were already much more wrong in selecting delivered signals on
protection violations.

Change the delivered signal for accesses to mapped area after the
backing object was truncated. According to POSIX description for
mmap(2):
The system shall always zero-fill any partial page at the end of an
object. Further, the system shall never write out any modified
portions of the last page of an object which are beyond its
end. References within the address range starting at pa and
continuing for len bytes to whole pages following the end of an
object shall result in delivery of a SIGBUS signal.

An implementation may generate SIGBUS signals when a reference
would cause an error in the mapped object, such as out-of-space
condition.
Adjust according to the description, keeping the existing
compatibility code for SIGSEGV/SIGBUS on protection failures.

For situations where kernel cannot handle page fault due to resource
limit enforcement, SIGBUS with a new error code BUS_OBJERR is
delivered. Also, provide a new error code SEGV_PKUERR for SIGSEGV on
amd64 due to protection key access violation.

vm_fault_hold() is renamed to vm_fault(). Fixed some nits in
trap_pfault()s like mis-interpreting Mach errors as errnos. Removed
unneeded truncations of the fault addresses reported by hardware.

PR: 211924
Reviewed by: alc
Discussed with: jilles, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D21566
rm64/trap.c
199eacdc019891902a20afc188b8cadf038d7882 25-Sep-2019 kevans <kevans@FreeBSD.org> sysent: regenerate after r352693
inux/linux_proto.h
fbe7e9c7e4b8943ee859d74685d8b06032a41682 25-Sep-2019 markj <markj@FreeBSD.org> Complete the removal of the "wire_count" field from struct vm_page.

Convert all remaining references to that field to "ref_count" and update
comments accordingly. No functional change intended.

Reviewed by: alc, kib
Sponsored by: Intel, Netflix
Differential Revision: https://reviews.freebsd.org/D21768
rm64/efirt_machdep.c
rm64/pmap.c
1629aaf0f7ed551d8d7d0831a8d43b15907c6412 21-Sep-2019 alc <alc@FreeBSD.org> In case a translation fault on the kernel address space occurs from
within a critical section, we must perform a lock-free check on the
faulting address.

Reported by: andrew
Reviewed by: andrew, markj
X-MFC with: r350579
Differential Revision: https://reviews.freebsd.org/D21685
rm64/pmap.c
3616760326ec36864a35663facb1e5b9b6d28d42 16-Sep-2019 markj <markj@FreeBSD.org> Revert r352406, which contained changes I didn't intend to commit.
rm64/pmap.c
543f9366b9e57d326639d4c1650ae10b2d9ff916 16-Sep-2019 markj <markj@FreeBSD.org> Fix a couple of nits in r352110.

- Remove a dead variable from the amd64 pmap_extract_and_hold().
- Fix grammar in the vm_page_wire man page.

Reported by: alc
Reviewed by: alc, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21639
rm64/pmap.c
7ebcf07f42679d3df511f9d1952e4ba3450c58e6 13-Sep-2019 br <br@FreeBSD.org> Add support for Intel Stratix 10 platform.

Intel Stratix 10 SoC includes a quad-core arm64 cluster and FPGA fabric.

This adds support for reconfiguring FPGA.

Accessing FPGA core of this SoC require the level of privilege EL3,
while kernel runs in EL1 (lower) level of privilege.

This provides an Intel service layer interface that uses SMCCC to pass
queries to the secure-monitor (EL3).

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D21454
onf/GENERIC
ntel/firmware.c
ntel/intel-smc.h
ntel/stratix10-soc-fpga-mgr.c
ntel/stratix10-svc.c
ntel/stratix10-svc.h
673399d81f41e3fe89b757e46a3bfc5be766d8b9 11-Sep-2019 emaste <emaste@FreeBSD.org> Update comments and ordering in linux*_dummy.c

- sort alphabetically
- getcpu arrived in Linux 2.6.19
- fanotify_* arrived in 2.6.36
inux/linux_dummy.c
22e36b3874d94f9920070bba2e82df1cddcf9f80 11-Sep-2019 emaste <emaste@FreeBSD.org> linuxulator: add stub arm64 linux_genassym.c

This will be fleshed out in the future but allows us to build the arm64
linuxulator using the same infrastructure as x86.
inux/linux_genassym.c
bc0ee42672012ab7ff3c104e9e522942290c750a 11-Sep-2019 emaste <emaste@FreeBSD.org> linuxulator: memfd_create first appeared in Linux 3.17

Reference: http://man7.org/linux/man-pages/man2/memfd_create.2.html
inux/linux_dummy.c
142a3d6513bdfeaa7e9c8a10a176c477a6dc4c7a 11-Sep-2019 emaste <emaste@FreeBSD.org> linuxulator: seccomp syscall first appeared in Linux 3.17

Reference: http://man7.org/linux/man-pages/man2/seccomp.2.html
inux/linux_dummy.c
c1fe73ee39192b29843ecfb1e8c14d6b100f8fcb 11-Sep-2019 emaste <emaste@FreeBSD.org> linux: add trivial renameat2 implementation

Just return EINVAL if flags != 0. The Linux man page documents one
case of EINVAL as "The filesystem does not support one of the flags in
flags."

After r351723 userland binaries will try using new system calls.

Reported by: mjg
Reviewed by: mjg, trasz
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21590
inux/linux_dummy.c
89195b1af4ad8d91b87e51b802e110b8289fc1da 11-Sep-2019 emaste <emaste@FreeBSD.org> regen linuxulator sysent after r352208
inux/linux_proto.h
inux/linux_systrace_args.c
d50f11cc69cde4498c6d24103a1bd9f6f09ec635 11-Sep-2019 emaste <emaste@FreeBSD.org> make linux_renameat2 args consistent with linux_renameat

Use 'dfd' consistently for a directory fd.
inux/syscalls.master
ccbfa8304f694595b78758646a54dbeee04f4be7 09-Sep-2019 markj <markj@FreeBSD.org> Change synchonization rules for vm_page reference counting.

There are several mechanisms by which a vm_page reference is held,
preventing the page from being freed back to the page allocator. In
particular, holding the page's object lock is sufficient to prevent the
page from being freed; holding the busy lock or a wiring is sufficent as
well. These references are protected by the page lock, which must
therefore be acquired for many per-page operations. This results in
false sharing since the page locks are external to the vm_page
structures themselves and each lock protects multiple structures.

Transition to using an atomically updated per-page reference counter.
The object's reference is counted using a flag bit in the counter. A
second flag bit is used to atomically block new references via
pmap_extract_and_hold() while removing managed mappings of a page.
Thus, the reference count of a page is guaranteed not to increase if the
page is unbusied, unmapped, and the object's write lock is held. As
a consequence of this, the page lock no longer protects a page's
identity; operations which move pages between objects are now
synchronized solely by the objects' locks.

The vm_page_wire() and vm_page_unwire() KPIs are changed. The former
requires that either the object lock or the busy lock is held. The
latter no longer has a return value and may free the page if it releases
the last reference to that page. vm_page_unwire_noq() behaves the same
as before; the caller is responsible for checking its return value and
freeing or enqueuing the page as appropriate. vm_page_wire_mapped() is
introduced for use in pmap_extract_and_hold(). It fails if the page is
concurrently being unmapped, typically triggering a fallback to the
fault handler. vm_page_wire() no longer requires the page lock and
vm_page_unwire() now internally acquires the page lock when releasing
the last wiring of a page (since the page lock still protects a page's
queue state). In particular, synchronization details are no longer
leaked into the caller.

The change excises the page lock from several frequently executed code
paths. In particular, vm_object_terminate() no longer bounces between
page locks as it releases an object's pages, and direct I/O and
sendfile(SF_NOCACHE) completions no longer require the page lock. In
these latter cases we now get linear scalability in the common scenario
where different threads are operating on different files.

__FreeBSD_version is bumped. The DRM ports have been updated to
accomodate the KPI changes.

Reviewed by: jeff (earlier version)
Tested by: gallatin (earlier version), pho
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20486
rm64/efirt_machdep.c
rm64/pmap.c
44aa9c9fa3812265e16345987de0f6d41c3ab377 04-Sep-2019 br <br@FreeBSD.org> Include dwgpio to the build.

Sponsored by: DARPA, AFRL
onf/GENERIC
d33fdaeb7e71302043934dec84478ec0faf75659 29-Aug-2019 kib <kib@FreeBSD.org> Centralize __pcpu definitions.

Many extern struct pcpu <something>__pcpu declarations were
copied/pasted in sources. The issue is that the definition is MD, but
it cannot be provided by machine/pcpu.h due to actual struct pcpu
defined in sys/pcpu.h later than the inclusion of machine/pcpu.h.
This forced the copying when other code needed direct access to
__pcpu. There is no way around it, due to machine/pcpu.h supplying
part of struct pcpu fields.

To work around the problem, add a new machine/pcpu_aux.h header, which
should fill any needed MD definitions after struct pcpu definition is
completed. This allows to remove copies of __pcpu spread around the
source. Also on x86 it makes it possible to remove work arounds like
OFFSETOF_CURTHREAD or clang specific warnings supressions.

Reported and tested by: lwhsu, bcran
Reviewed by: imp, markj (previous version)
Discussed with: jhb
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D21418
rm64/mp_machdep.c
nclude/counter.h
nclude/pcpu_aux.h
c8ec5294dee10fa368d885f6eb78da326130e72e 27-Aug-2019 manu <manu@FreeBSD.org> arm64: rk3399: pinctrl: Add gpio banks and fix iomux

Since r351187 the pinctrl driver need to know the gpio bank as it
directly attach the gpio driver to handle some setup that might
be present in the dts, add the gpio banks table for rk3399.
While here fix some IOMUX definition that prevented to boot
on RK3399 as pinctrl wasn't configured correctly.

Submitted by: mmel (original version)
MFC after: 2 weeks
MFC With: r351187
ockchip/rk_pinctrl.c
19ba879ed0eaeb5ae5cccb63a13bef5f85edd45a 27-Aug-2019 manu <manu@FreeBSD.org> arm64: rk3328: pinctrl: Add gpio banks and fix iomux

Since r351187 the pinctrl driver need to know the gpio bank as it
directly attach the gpio driver to handle some setup that might
be present in the dts, add the gpio banks table for rk3328.
While here fix some IOMUX definition that prevented to boot
on RK3328 as pinctrl wasn't configured correctly.

Submitted by: mmel (original version)
MFC after: 2 weeks
MFC With: r351187
ockchip/rk_pinctrl.c
773a8248ccf0487a6fafc83cc4c9a23578dd3396 27-Aug-2019 mmel <mmel@FreeBSD.org> Add support for RK3288 into existing RockChip drivers.
This patch ensures only minimal level of compatibility necessary to boot
on RK3288 based boards. GPIO and pinctrl interaction, missing in current
implementation, will be improved by own patch in the near future.

MFC after: 2 weeks
MFC with: r351452
ockchip/if_dwc_rk.c
ockchip/rk_gpio.c
ockchip/rk_grf.c
31a2cbb52a7e30fd38955b55edefb2ca1ceab051 26-Aug-2019 tuexen <tuexen@FreeBSD.org> Identify eMAG CPU used in Ampere Computing systems.

Reviewed by: emaste@
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21314
rm64/identcpu.c
nclude/cpu.h
16f50c36fb4b80cf36263729a47a9d4e96f89ea7 18-Aug-2019 mmel <mmel@FreeBSD.org> Improve rk_pinctrl driver:
- add support for 'output-low', 'output-high', 'output-low' and
'output-enable' properties. These are use in RK3288 DT files
- add support for RK3288
- to reduce overall file size, use local macros for initialization
of pinctrl description structures.

MFC after: 2 weeks
ockchip/rk_pinctrl.c
59a1d217f46e9a51ebee35db1bf9ee36ba1646e1 18-Aug-2019 mmel <mmel@FreeBSD.org> Improve rk_i2c driver:
- Properly handle IIC_M_NOSTOP and IIC_M_NOSTART flags.
- add polling mode, so driver can be used even if interrupts are not
enabled (this is necessary for proper support of PMICs).
- add support for RK3288

MFC after: 2 weeks
ockchip/rk_i2c.c
ffea06a831a81ddf2247d0db5365d8de1febc851 16-Aug-2019 manu <manu@FreeBSD.org> arm64: Add EspressoBin DTB to the build

This will compile the espressobin dts to a dtb file and this will be
install in /boot/dtb/marvell/ during installkernel.

MFC after: 1 week
onf/GENERIC
271bf7a7890b9d72d6a8f6973011c340f20d93c1 16-Aug-2019 emaste <emaste@FreeBSD.org> aarch64: make pmap_change_attr public like on other platforms

Submitted by: Greg V <greg@unrelenting.technology>
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D20787
rm64/pmap.c
nclude/pmap.h
685a292036f635fca8e4316dfd50f75ad0a96b9f 16-Aug-2019 jeff <jeff@FreeBSD.org> Move phys_avail definition into MI code. It is consumed in the MI layer and
doing so adds more flexibility with less redundant code.

Reviewed by: jhb, markj, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21250
nclude/pmap.h
6a103b76a4c245d807917b720f15600bc6e05205 12-Aug-2019 imp <imp@FreeBSD.org> ukbdmap.h rule was identical on all platforms, so move them into sys/conf/files.

This allows us to remove 'nodevice ukbd' from the arm64 NOTES file.
onf/NOTES
75e675195a3ca7ec7da625dbe849ebed8b9443c6 12-Aug-2019 imp <imp@FreeBSD.org> Floppy driver really only works on x86

Move the floppy driver to the x86 specific notes file.

Reviewed by: jhb, manu, jhibbits, emaste
Differential Revision: https://reviews.freebsd.org/D21208
onf/NOTES
c0702cb45a02db4c04aecbb7a10ff452499d6e4e 12-Aug-2019 imp <imp@FreeBSD.org> Move sc out of the global file

x86 needs sc, as does sparc64. powerpc doesn't use it by default, but some old
powermac notebooks do not work with vt yet for reasons unknonw. Even so, I've
removed it from powerpc LINT. It's not in daily use there, and the intent is to
100% switch to vt now that it works for that platform to limit support burden.

All the other architectures omit some or all of the screen savers from their
lint config. Move them to the x86 NOTES files and remove the exclusions. This
reduces slightly the number of savers sparc64 compiles, but since they are in
GENERIC, the overage is adequate and if someone reaelly wants to sort them out
in sparc64 they can sweat the details and the testing.

Reviewed by: jhb (earlier version), manu (earlier version), jhibbits
Differential Revision: https://reviews.freebsd.org/D21233
onf/NOTES
02386873bc187d9d95d4e0ca14e54dd10865419a 08-Aug-2019 imp <imp@FreeBSD.org> First pass at a LINT for arm64.

Create a rough and ready NOTES file from GENERIC, remove the duplication from
sys/conf/NOTES and add relevant no* directives to make this compile.

Reviewed by: jhb, manu (earlier versions that differed only in comments)
Differential Revision: https://reviews.freebsd.org/D21184
onf/Makefile
onf/NOTES
33b1fa4d646ae06caa6e5464cada0c83957d684a 08-Aug-2019 imp <imp@FreeBSD.org> Make arm64 32-bit mode compile with COMPAT_43

The COMPAT_43 option isn't quite like the other compat options, and arm64 makes
attempts to support it in 64-bit mode. In 32-bit compat mode, however, two
syscall implementations that COMPAT_FREEBSD32 assumes will be there are
missing. Provide implementations for these: ofreebsd32_sigreturn (which we'll
never encounter, so implement it as nosys as is done in kern_sig.c) and
ofreebsd32_getpagesize, where we'll always return 4096 since that's the only
PAGE_SIZE we support, similar to how the ia32 implementation does things.

Reviewed by: manu@
Differential Revision: https://reviews.freebsd.org/D21192
rm64/freebsd32_machdep.c
6632123aeacc2a3a0f1db1dfd656f7d0fbeba109 08-Aug-2019 alc <alc@FreeBSD.org> Ordinarily, during a superpage promotion or demotion within a pmap, the
pmap's lock ensures that other operations on the pmap don't observe the
old mapping being broken before the new mapping is established. However,
pmap_kextract() doesn't acquire the kernel pmap's lock, so it may observe
the broken mapping. And, if it does, it returns an incorrect result.

This revision implements a lock-free solution to this problem in
pmap_update_entry() and pmap_kextract() because pmap_kextract() can't
acquire the kernel pmap's lock.

Reported by: andrew, greg_unrelenting.technology
Reviewed by: andrew, markj
Tested by: greg_unrelenting.technology
X-MFC with: r350579
Differential Revision: https://reviews.freebsd.org/D21169
rm64/pmap.c
nclude/pte.h
0b7fe37f61e031ba122f9e355024dfb2173b026f 05-Aug-2019 alc <alc@FreeBSD.org> Enable superpage promotion within the kernel pmap.

Reviewed by: markj
X-MFC after: r350004
Differential Revision: https://reviews.freebsd.org/D21149
rm64/pmap.c
e5a877a07103eefb60e5adc8d8f4cbae1a816d2c 02-Aug-2019 alc <alc@FreeBSD.org> Because of AArch64's weak memory consistency model, we need to include a
memory barrier between the stores for initializing a page table page and
the store for adding that page to the page table. Otherwise, a page table
walk by another processor's MMU could see the page table page before it
sees the initialized entries.

Simplify pmap_growkernel(). In particular, eliminate an unnecessary TLB
invalidation.

Reviewed by: andrew, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21126
rm64/pmap.c
db972bc38eef577dd43f13c5b39c0ce517469d6d 01-Aug-2019 markj <markj@FreeBSD.org> Use ATTR_DBM even when hardware dirty bit management is not enabled.

The ARMv8 reference manual only states that the bit is reserved in
this case; following Linux's example, use it instead of a
software-defined bit for the purpose of indicating that a managed
mapping is writable.

Reviewed by: alc, andrew
MFC after: r350004
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21121
rm64/pmap.c
nclude/pte.h
58ae190bd3c0e785e5895867257c9926915fb818 31-Jul-2019 alc <alc@FreeBSD.org> In pmap_advise(), when we encounter a superpage mapping, we first demote the
mapping and then destroy one of the 4 KB page mappings so that there is a
potential trigger for repromotion. Currently, we destroy the first 4 KB
page mapping that falls within the (current) superpage mapping or the
virtual address range [sva, eva). However, I have found empirically that
destroying the last 4 KB mapping produces slightly better results,
specifically, more promotions and fewer failed promotion attempts.
Accordingly, this revision changes pmap_advise() to destroy the last 4 KB
page mapping. It also replaces some nearby uses of boolean_t with bool.

Reviewed by: kib, markj
Differential Revision: https://reviews.freebsd.org/D21115
rm64/pmap.c
0c969d5f368b82971209557edd5ecef6864bae84 30-Jul-2019 emaste <emaste@FreeBSD.org> linuxulator: rename linux_locore.s to .asm

It is assembled using "${CC} -x assembler-with-cpp", which by convention
(bsd.suffixes.mk) uses the .asm extension.

This is a portion of the review referenced below (D18344). That review
also renamed linux_support.s to .S, but that is a functional change
(using the compiler's integrated assembler instead of as) and will be
revisited separately.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18344
inux/linux_locore.asm
inux/linux_locore.s
4eedc99e904c8fee0846310a7e9818ede195431f 30-Jul-2019 br <br@FreeBSD.org> Add support for the SD/MMC controller found in Terasic DE10-Pro
(an Intel Stratix 10 GX/SX FPGA Development Kit).

Set the bus speed manually due to lack of clock management support.

Sponsored by: DARPA, AFRL
onf/GENERIC
97200858d08f38f17836fdc9ff84fc78cdadb04b 29-Jul-2019 markj <markj@FreeBSD.org> Have arm64's pmap_fault() handle WnR faults on dirty PTEs.

If we take a WnR permission fault on a managed, writeable and dirty
PTE, simply return success without calling the main fault handler. This
situation can occur if multiple threads simultaneously access a clean
writeable mapping and trigger WnR faults; losers of the race to mark the
PTE dirty would end up calling the main fault handler, which had no work
to do.

Reported by: alc
Reviewed by: alc
MFC with: r350004
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21097
rm64/pmap.c
468883bcb0745130eeff7adfdd966b94126325ca 29-Jul-2019 markj <markj@FreeBSD.org> Remove an unneeded trunc_page() in pmap_fault().

Reported by: alc
MFC with: r350004
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
a1a1b094141faff327e301c27e636361fc5f660f 29-Jul-2019 br <br@FreeBSD.org> Add glue driver for Altera SOCFPGA Ethernet MAC (EMAC) found in
Terasic DE10-Pro (an Intel Stratix 10 GX/SX FPGA Development Kit).

The Altera EMAC is an instance of Synopsys DesignWare Gigabit MAC.

This driver sets correct clock range for MDIO interface on Intel Stratix 10
platform.

This is required due to lack of support for clock manager device for
this platform that could tell us the clock frequency value for ethernet
clock domain.

Sponsored by: DARPA, AFRL
onf/GENERIC
467944ce4546b5d2cd88b15125501d03597920bd 26-Jul-2019 alc <alc@FreeBSD.org> Implement pmap_advise(). (Without a working pmap_advise() implementation
madvise(MADV_DONTNEED) and madvise(MADV_FREE) are NOPs.)

Reviewed by: markj
X-MFC after: r350004
Differential Revision: https://reviews.freebsd.org/D21062
rm64/pmap.c
41dcc1dad149d003a42793019e23153fd2f537a3 25-Jul-2019 alc <alc@FreeBSD.org> Simplify the handling of superpages in pmap_clear_modify(). Specifically,
if a demotion succeeds, then all of the 4KB page mappings within the
superpage-sized region must be valid, so there is no point in testing the
validity of the 4KB page mapping that is going to be write protected.

Deindent the nearby code.

Reviewed by: kib, markj
Tested by: pho (amd64, i386)
X-MFC after: r350004 (this change depends on arm64 dirty bit emulation)
Differential Revision: https://reviews.freebsd.org/D21027
rm64/pmap.c
a68b656d72d93e25061ffa910ba918d60d9f6a0b 23-Jul-2019 andrew <andrew@FreeBSD.org> As with r350241 use the new UL macro on the main register mask.

MFC after: 1 week
Sponsored by: DARPA, AFRL
nclude/armreg.h
e52d3fea4b3b0fa390dd839502bf7f59515e887b 23-Jul-2019 andrew <andrew@FreeBSD.org> Ensure the arm64 ID register fields are 64 bit types.

Previously only some of the ID register fields were 64 bit. To allow
for a script to generate these mark them all 64 bit. To allow for their
use in assembly we need to use the UINT64_C macro via a new UL macro
to stop the lines from being too long.

MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D20977
nclude/armreg.h
cfe8b5460293132cfeaccb511f249f6add94af41 21-Jul-2019 alc <alc@FreeBSD.org> With the introduction of software dirty bit emulation for managed mappings,
we should test ATTR_SW_DBM, not ATTR_AP_RW, to determine whether to set
PGA_WRITEABLE. In effect, we are currently setting PGA_WRITEABLE based on
whether the dirty bit is preset, not whether the mapping is writeable.
Correct this mistake.

Reviewed by: markj
X-MFC with: r350004
Differential Revision: https://reviews.freebsd.org/D21013
rm64/pmap.c
19eb42c206110ac62e7f2be4723da6fc7b79fd7f 21-Jul-2019 alc <alc@FreeBSD.org> Introduce pmap_store(), and use it to replace pmap_load_store() in places
where the page table entry was previously invalid. (Note that I did not
replace pmap_load_store() when it was followed by a TLB invalidation, even
if we are not using the return value from pmap_load_store().)

Correct an error in pmap_enter(). A test for determining when to set
PGA_WRITEABLE was always true, even if the mapping was read only.

In pmap_enter_l2(), when replacing an empty kernel page table page by a
superpage mapping, clear the old l2 entry and issue a TLB invalidation. My
reading of the ARM architecture manual leads me to believe that the TLB
could hold an intermediate entry referencing the empty kernel page table
page even though it contains no valid mappings.

Replace a couple direct uses of atomic_clear_64() by the new
pmap_clear_bits().

In a couple comments, replace the term "paging-structure caches", which is
an Intel-specific term for the caches that hold intermediate entries in the
page table, with wording that is more consistent with the ARM architecture
manual.

Reviewed by: markj
X-MFC after: r350004
Differential Revision: https://reviews.freebsd.org/D20998
rm64/pmap.c
8d6dc914b6ab537b676321643a239e8d1ca24ce7 20-Jul-2019 manu <manu@FreeBSD.org> arm64: Implement HWCAP

Add HWCAP support for arm64.
defines are the same as in Linux and a userland program can use
elf_aux_info to get the data.
We only save the common denominator for all cores in case the
big and little cluster have different support (this is known to
exists even if we don't support those SoCs in FreeBSD)

Differential Revision: https://reviews.freebsd.org/D17137
rm64/elf_machdep.c
rm64/identcpu.c
nclude/cpu.h
nclude/elf.h
3da75da272098d724b4144e197c2406c1a1493ac 20-Jul-2019 ganbold <ganbold@FreeBSD.org> Add driver for Rockchip RK3399 eMMC PHY.
Tested on NanoPC-T4 board.

Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D20840
onf/GENERIC
ockchip/rk3399_emmcphy.c
f90826df9f018c95bd2e5e09f5365a921e9bc989 18-Jul-2019 andrew <andrew@FreeBSD.org> Rename arm64 macros in preperation for a script to generate them.

I have a script to generate most of the ID_AA64* macros from the Arm
XML source [1]. In preperation for using this we need to clean up the
macros to be in line with what the script will generate. This is the
first step, rename the macros to follow the names in said XML.

[1] https://developer.arm.com/architectures/cpu-architecture/a-profile/exploration-tools

MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D20976
rm64/identcpu.c
nclude/armreg.h
07347a7f8f1310278cde38bdc6107278d1257334 16-Jul-2019 markj <markj@FreeBSD.org> Always use the software DBM bit for now.

r350004 added most of the machinery needed to support hardware DBM
management, but it did not intend to actually enable use of the hardware
DBM bit.

Reviewed by: andrew
MFC with: r350004
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
079d038a05bc2081beddbc5b73cff8baec1d0e31 16-Jul-2019 markj <markj@FreeBSD.org> Fix the arm64 page table entry attribute mask.

It did not include the DBM or contiguous bits.

Reported by: andrew
Reviewed by: andrew
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
nclude/pte.h
5f4e2bae6a5a83d57b4bf6854b2850ba7e1c0da4 16-Jul-2019 markj <markj@FreeBSD.org> Propagate attribute changes during demotion.

After r349117 and r349122, some mapping attribute changes do not trigger
superpage demotion. However, pmap_demote_l2() was not updated to ensure
that the replacement L3 entries carry any attribute changes that
occurred since promotion.

Reported and tested by: manu
Reviewed by: alc
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D20965
rm64/pmap.c
895d57ec602c370d067c302e0e3edf3059f08892 15-Jul-2019 jhb <jhb@FreeBSD.org> Don't pass error from syscallenter() to syscallret().

syscallret() doesn't use error anymore. Fix a few other places to permit
removing the return value from syscallenter() entirely.
- Remove a duplicated assertion from arm's syscall().
- Use td_errno for amd64_syscall_ret_flush_l1d.

Reviewed by: kib
MFC after: 1 month
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D2090
rm64/trap.c
cdc0469c0e7a5058598baaa649f0367fe28e3fee 15-Jul-2019 markj <markj@FreeBSD.org> Implement software access and dirty bit management for arm64.

Previously the arm64 pmap did no reference or modification tracking;
all mappings were treated as referenced and all read-write mappings
were treated as dirty. This change implements software management
of these attributes.

Dirty bit management is implemented to emulate ARMv8.1's optional
hardware dirty bit modifier management, following a suggestion from alc.
In particular, a mapping with ATTR_SW_DBM set is logically writeable and
is dirty if the ATTR_AP_RW_BIT bit is clear. Mappings with
ATTR_AP_RW_BIT set are write-protected, and a write access will trigger
a permission fault. pmap_fault() handles permission faults for such
mappings and marks the page dirty by clearing ATTR_AP_RW_BIT, thus
mapping the page read-write.

Reviewed by: alc
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D20907
rm64/pmap.c
rm64/trap.c
nclude/pte.h
0b570c6d9c18d2528a71596f150b26afc89c1644 13-Jul-2019 alc <alc@FreeBSD.org> Revert r349442, which was a workaround for bus errors caused by an errant
TLB entry. Specifically, at the start of pmap_enter_quick_locked(), we
would sometimes have a TLB entry for an invalid PTE, and we would need to
issue a TLB invalidation before exiting pmap_enter_quick_locked(). However,
we should never have a TLB entry for an invalid PTE. r349905 has addressed
the root cause of the problem, and so we no longer need this workaround.

X-MFC after: r349905
rm64/pmap.c
ea314818c6b9e079e07ec395c7b878adb8c7ac11 12-Jul-2019 kib <kib@FreeBSD.org> Provide protection against starvation of the ll/sc loops when accessing userpace.

Casueword(9) on ll/sc architectures must be prepared for userspace
constantly modifying the same cache line as containing the CAS word,
and not loop infinitely. Otherwise, rogue userspace livelocks the
kernel.

To fix the issue, change casueword(9) interface to return new value 1
indicating that either comparision or store failed, instead of relying
on the oldval == *oldvalp comparison. The primitive no longer retries
the operation if it failed spuriously. Modify callers of
casueword(9), all in kern_umtx.c, to handle retries, and react to
stops and requests to terminate between retries.

On x86, despite cmpxchg should not return spurious failures, we can
take advantage of the new interface and just return PSL.ZF.

Reviewed by: andrew (arm64, previous version), markj
Tested by: pho
Reported by: https://xenbits.xen.org/xsa/advisory-295.txt
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D20772
rm64/support.S
e7f77dfbc0a07e4e47dce5339ec65bfaad78234a 12-Jul-2019 markj <markj@FreeBSD.org> Apply some light cleanup to uses of pmap_pte_dirty().

- Check for ATTR_SW_MANAGED before anything else.
- Use pmap_pte_dirty() in pmap_remove_pages().

No functional change intended.

Reviewed by: alc
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
8ba13f1606397781d4a9367ffc9ea55fbb7e0ffb 11-Jul-2019 markj <markj@FreeBSD.org> Fix some ISS bit definitions for data aborts.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation
nclude/armreg.h
f3e8efe2d48958d1c844c2d1bc4ac22b04404ebc 11-Jul-2019 alc <alc@FreeBSD.org> According to Section D5.10.3 "Maintenance requirements on changing System
register values" of the architecture manual, an isb instruction should be
executed after updating ttbr0_el1 and before invalidating the TLB. The
lack of this instruction in pmap_activate() appears to be the reason why
andrew@ and I have observed an unexpected TLB entry for an invalid PTE on
entry to pmap_enter_quick_locked(). Thus, we should now be able to revert
the workaround committed in r349442.

Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20904
rm64/efirt_machdep.c
rm64/pmap.c
98074aa6ddbdb4c9e7f10f7ba3fae4125b53f5d3 10-Jul-2019 markj <markj@FreeBSD.org> Rename pmap_page_dirty() to pmap_pte_dirty().

This is a precursor to implementing dirty bit management.

Discussed with: alc
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
6af2f0abb46cca406e84fa1d1dae1f020222bb0e 09-Jul-2019 alc <alc@FreeBSD.org> Introduce pmap_clear(), which zeroes a page table entry, and use it, instead
of pmap_load_clear(), in places where we don't care about the page table
entry's prior contents.

Eliminate an unnecessary pmap_load() from pmap_remove_all(). Instead, use
the value returned by the pmap_load_clear() on the very next line. (In the
future, when we support "hardware dirty bit management", using the value
from the pmap_load() rather than the pmap_load_clear() would have actually
been an error because the dirty bit could potentially change between the
pmap_load() and the pmap_load_clear().)

A KASSERT() in pmap_enter(), which originated in the amd64 pmap, was meant
to check the value returned by the pmap_load_clear() on the previous line.
However, we were ignoring the value returned by the pmap_load_clear(), and
so the KASSERT() was not serving its intended purpose. Use the value
returned by the pmap_load_clear() in the KASSERT().

MFC after: 2 weeks
rm64/pmap.c
039f74039e57a5d79cda830161af71a1e9c4050e 08-Jul-2019 markj <markj@FreeBSD.org> Merge the vm_page hold and wire mechanisms.

The hold_count and wire_count fields of struct vm_page are separate
reference counters with similar semantics. The remaining essential
differences are that holds are not counted as a reference with respect
to LRU, and holds have an implicit free-on-last unhold semantic whereas
vm_page_unwire() callers must explicitly determine whether to free the
page once the last reference to the page is released.

This change removes the KPIs which directly manipulate hold_count.
Functions such as vm_fault_quick_hold_pages() now return wired pages
instead. Since r328977 the overhead of maintaining LRU for wired pages
is lower, and in many cases vm_fault_quick_hold_pages() callers would
swap holds for wirings on the returned pages anyway, so with this change
we remove a number of page lock acquisitions.

No functional change is intended. __FreeBSD_version is bumped.

Reviewed by: alc, kib
Discussed with: jeff
Discussed with: jhb, np (cxgbe)
Tested by: pho (previous version)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D19247
rm64/pmap.c
6b5e2b622397bd64d57f356d28062aa7e16656de 07-Jul-2019 alc <alc@FreeBSD.org> Three changes to pmap_enter():

1. Use _pmap_alloc_l3() instead of pmap_alloc_l3() in order to handle the
possibility that a superpage mapping for "va" was created while we slept.
(This is derived from the amd64 version.)

2. Eliminate code for allocating kernel page table pages. Kernel page
table pages are preallocated by pmap_growkernel().

3. Eliminate duplicated unlock operations when KERN_RESOURCE_SHORTAGE is
returned.

MFC after: 2 weeks
rm64/pmap.c
1b381cfc3ca55ac35dcb6e7eee70efd8b292ec42 05-Jul-2019 alc <alc@FreeBSD.org> Restructure cache_handle_range to avoid repeated barriers. Specifically,
restructure cache_handle_range so that all of the data cache operations are
performed before any instruction cache operations. Then, we only need one
barrier between the data and instruction cache operations and one barrier
after the instruction cache operations.

On an Amazon EC2 a1.2xlarge instance, this simple change reduces the time
for a "make -j8 buildworld" by 9%.

Reviewed by: andrew
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20848
rm64/cpufunc_asm.S
ee002f95a338d1ff367a1e3dccc6368038486ad5 03-Jul-2019 ganbold <ganbold@FreeBSD.org> Subclass Rockchip's General Register Files driver from Simple MFD driver.
ockchip/rk_grf.c
b233a7ed74744f095dacfa4c29f5a9f30489b87f 02-Jul-2019 alc <alc@FreeBSD.org> Implement pmap_copy(). (This includes the changes applied to the amd64
pmap_copy() in r349585.)

Reviewed by: kib, markj
Differential Revision: https://reviews.freebsd.org/D20790
rm64/pmap.c
0ee80ab0e2ff5b640bd9c05976f5db1853e5eb5e 01-Jul-2019 manu <manu@FreeBSD.org> arm64: efi: Map memory IO region as device

Reviewed by: andrew
Sponsored by: Ampere Computing, LLC
rm64/efirt_machdep.c
a33a56c7b548f3c176d0f213dd8dd69ee64233f0 30-Jun-2019 np <np@FreeBSD.org> Display the approximate space needed when a minidump fails due to lack
of space.

Reviewed by: kib@
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D20801
rm64/minidump_machdep.c
f18057177b20b404b392e839aaff14c466c90099 26-Jun-2019 cognet <cognet@FreeBSD.org> In get_fpcontext32() and set_fpcontext32(), we can't just use memcpy() to
copy the VFP registers.
arvm7 VFP uses 32 64bits fp registers (but those could be used in pairs to
make 16 128bits registers), while aarch64 uses 32 128bits fp registers, so
we have to copy the value of each register.
rm64/freebsd32_machdep.c
8646c1fa6bac4bbdcfd6faa04a6d276b2fab8113 26-Jun-2019 alc <alc@FreeBSD.org> Revert one of the changes from r349323. Specifically, undo the change
that replaced a pmap_invalidate_page() with a dsb(ishst) in
pmap_enter_quick_locked(). Even though this change is in principle
correct, I am seeing occasional, spurious bus errors that are only
reproducible without this pmap_invalidate_page(). (None of adding an
isb, "upgrading" the dsb to wait on loads as well as stores, or
disabling superpage mappings eliminates the bus errors.) Add an XXX
comment explaining why the pmap_invalidate_page() is being performed.

Discussed with: andrew, markj
rm64/pmap.c
dff664a8556490c7bc01931399e1f17b202da2d7 26-Jun-2019 cognet <cognet@FreeBSD.org> Fix debugging of 32bits arm binaries on arm64.

In set_regs32()/fill_regs32(), we have to get/set SP and LR from/to
tf_x[13] and tf_x[14].
set_regs() and fill_regs() may be called for a 32bits process, if the process
is ptrace'd from a 64bits debugger. So, in set_regs() and fill_regs(), get
or set PC and SPSR from where the debugger expects it, from tf_x[15] and
tf_x[16].
rm64/machdep.c
d4ca47be394b58863863386ae3acd30e25079071 24-Jun-2019 jchandra <jchandra@FreeBSD.org> arm64 acpi_iort: add some error handling

Print warnings for some bad kernel configurations (like NUMA disabled
with multiple domains). Check and report some firmware errors (like
incorrect proximity domain entries).

Differential Revision: https://reviews.freebsd.org/D20416
cpica/acpi_iort.c
4dbc8bbbad301794941fb0e40419df2e1d7b8015 24-Jun-2019 jchandra <jchandra@FreeBSD.org> arm64 gicv3_its: enable all ITS blocks for a CPU

We now support multiple ITS blocks raising interrupts to a CPU.
Add all available CPUs to the ITS when no NUMA information is
available.

This reverts the check added in r340602, at that tim we did not
suppport multiple ITS blocks for a CPU.

Differential Revision: https://reviews.freebsd.org/D20417
rm64/gicv3_its.c
6dd56b2c134f6eccbb45d0e3c916bd8a81d9aef2 24-Jun-2019 jchandra <jchandra@FreeBSD.org> arm64 gic: Drop unused GICV3_IVAR_REDIST_VADDR

Now that GICV3_IVAR_REDIST is available, GICV3_IVAR_REDIST_VADDR
is unused and can be removed. Drop the define and add a comment.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D20454
rm64/gic_v3.c
rm64/gic_v3_var.h
7cd42a206e85cf22882b3fec3479511171ac737a 23-Jun-2019 alc <alc@FreeBSD.org> pmap_enter_quick_locked() never replaces a valid mapping, so it need not
perform a TLB invalidation. A barrier suffices. (See r343876.)

Add a comment to pmap_enter_quick_locked() in order to highlight the
fact that it does not replace valid mappings.

Correct a typo in one of pmap_enter()'s comments.

MFC after: 1 week
rm64/pmap.c
f228ac131cb51b24b9caad31053ffc0acb2db67d 22-Jun-2019 alc <alc@FreeBSD.org> Introduce pmap_remove_l3_range() and use it in two places:
(1) pmap_remove(), where it eliminates redundant TLB invalidations by
pmap_remove() and pmap_remove_l3(), and (2) pmap_enter_l2(), where it may
optimize the TLB invalidations by batching them.

Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D12725
rm64/pmap.c
aae886faded9a19b91af118f441c582361e5e0b7 21-Jun-2019 cem <cem@FreeBSD.org> sys: Remove DEV_RANDOM device option

Remove 'device random' from kernel configurations that reference it (most).
Replace perhaps mistaken 'nodevice random' in two MIPS configs with 'options
RANDOM_LOADABLE' instead. Document removal in UPDATING; update NOTES and
random.4.

Reviewed by: delphij, markm (previous version)
Approved by: secteam(delphij)
Differential Revision: https://reviews.freebsd.org/D19918
onf/GENERIC
4bc7fb5303438337b36b08cb1503e899d39df210 19-Jun-2019 alc <alc@FreeBSD.org> Correct an error in r349122. pmap_unwire() should update the pmap's wired
count, not its resident count.

X-MFC with: r349122
rm64/pmap.c
6c4c499cffcba5ffed2e6459f725638800cf81a3 17-Jun-2019 alc <alc@FreeBSD.org> Eliminate a redundant call to pmap_invalidate_page() from
pmap_ts_referenced().

MFC after: 14 days
Differential Revision: https://reviews.freebsd.org/D12725
rm64/pmap.c
dd738ba8796da1396781867713057b5451adb790 16-Jun-2019 alc <alc@FreeBSD.org> Three changes to arm64's pmap_unwire():

Implement wiring changes on superpage mappings. Previously, a superpage
mapping was unconditionally demoted by pmap_unwire(), even if the wiring
change applied to the entire superpage mapping.

Rewrite a comment to use the arm64 names for bits in a page table entry.
Previously, the bits were referred to by their x86 names.

Use atomic_"op"_64() instead of atomic_"op"_long() to update a page table
entry in order to match the prevailing style in this file.

MFC after: 10 days
rm64/pmap.c
e9b66164d9fb172b094a5e01aec50af4a03ebf99 16-Jun-2019 alc <alc@FreeBSD.org> Three enhancements to arm64's pmap_protect():

Implement protection changes on superpage mappings. Previously, a superpage
mapping was unconditionally demoted by pmap_protect(), even if the
protection change applied to the entire superpage mapping.

Precompute the bit mask describing the protection changes rather than
recomputing it for every page table entry that is changed.

Skip page table entries that already have the requested protection changes
in place.

Reviewed by: andrew, kib
MFC after: 10 days
Differential Revision: https://reviews.freebsd.org/D20657
rm64/pmap.c
1c176f7393d5a24636f878840553e1b962d8eaeb 15-Jun-2019 alc <alc@FreeBSD.org> Previously, when pmap_remove_pages() destroyed a dirty superpage mapping,
it only called vm_page_dirty() on the first of the superpage's constituent
4KB pages. This revision corrects that error, calling vm_page_dirty() on
all of superpage's constituent 4KB pages.

MFC after: 3 days
rm64/pmap.c
a54ff7ccd8f2924041dbc747fbca97820e4eaac1 15-Jun-2019 julian <julian@FreeBSD.org> Lightly hide the 'var' inside the macros to read the arm special registers.
I just happenned to have 3rd party code using 'var' as the output variable
which drew my attention to this. variables defined inside macros should be
prefixed to avoid getting shadowed varable wanrings from clang.
nclude/armreg.h
5d89b521c5fbb54138833d9e75b347f0a3533926 14-Jun-2019 alc <alc@FreeBSD.org> Batch the TLB invalidations that are performed by pmap_protect() rather
than performing them one at a time.

MFC after: 10 days
rm64/pmap.c
4fd6fe044c7407d68435d36c51e2413ba39d6a3a 14-Jun-2019 alc <alc@FreeBSD.org> Change the arm64 pmap so that updates to the global count of wired pages are
not performed directly by the pmap. Instead, they are performed by
vm_page_free_pages_toq(). (This is the same approach that we use on x86.)

Reviewed by: kib, markj
MFC after: 10 days
Differential Revision: https://reviews.freebsd.org/D20627
rm64/pmap.c
18116a4713e80e4563f6c20565acd8e759f0596c 12-Jun-2019 alc <alc@FreeBSD.org> Change pmap_demote_l2_locked() so that it removes the superpage mapping on a
demotion failure. Otherwise, some callers to pmap_demote_l2_locked(), such
as pmap_protect(), may leave an incorrect mapping in place on a demotion
failure.

Change pmap_demote_l2_locked() so that it handles addresses that are not
superpage aligned. Some callers to pmap_demote_l2_locked(), such as
pmap_protect(), may not pass a superpage aligned address.

Change pmap_enter_l2() so that it correctly calls vm_page_free_pages_toq().
The arm64 pmap is updating the count of wired pages when freeing page table
pages, so pmap_enter_l2() should pass false to vm_page_free_pages_toq().

Optimize TLB invalidation in pmap_remove_l2().

Reviewed by: kib, markj (an earlier version)
Discussed with: andrew
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D20585
rm64/pmap.c
8797c3d838465ecc5452526de8eab4b26a54a605 10-Jun-2019 bz <bz@FreeBSD.org> A bit of code hygiene (no functional changes).

Hide unused code under #ifdef notyet (in one case the only caller is under
that same ifdef), or if it is arm (not arm64) specific code under the
__arm__ ifdef to not yield -Wunused-function warnings during the arm64
kernel compile.

MFC after: 2 weeks
ockchip/if_dwc_rk.c
8b044e9dba967e2ae97fdeb86392e9150e994089 10-Jun-2019 loos <loos@FreeBSD.org> Add the GPIO driver for the North/South bridge in Marvell Armada 37x0.

The A3700 has a different GPIO controller and thus, do not use the old (and
shared) code for Marvell.

The pinctrl driver, also part of the controller, is not supported yet (but
the implementation should be straightforward).

Sponsored by: Rubicon Communications, LLC (Netgate)
onf/GENERIC
0b597963c6e9528645adb972875d1348d1a4cce2 09-Jun-2019 alc <alc@FreeBSD.org> Implement an alternative solution to the amd64 and i386 pmap problem that we
previously addressed in r348246.

This pmap problem also exists on arm64 and riscv. However, the original
solution developed for amd64 and i386 cannot be used on arm64 and riscv. In
particular, arm64 and riscv do not define a PG_PROMOTED flag in their level
2 PTEs. (A PG_PROMOTED flag makes no sense on arm64, where unlike x86 or
riscv we are required to break the old 4KB mappings before making the 2MB
mapping; and on riscv there are no unused bits in the PTE to define a
PG_PROMOTED flag.)

This commit implements an alternative solution that can be used on all four
architectures. Moreover, this solution has two other advantages. First, on
older AMD processors that required the Erratum 383 workaround, it is less
costly. Specifically, it avoids unnecessary calls to pmap_fill_ptp() on a
superpage demotion. Second, it enables the elimination of some calls to
pagezero() in pmap_kernel_remove_{l2,pde}().

In addition, remove a related stale comment from pmap_enter_{l2,pde}().

Reviewed by: kib, markj (an earlier version)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20538
rm64/pmap.c
e0b11c4354f5486711802df9cd15e74d7686db5c 03-Jun-2019 tychon <tychon@FreeBSD.org> very large dma mappings can cause integer overflow

Reviewed by: kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D20505
rm64/busdma_bounce.c
786a38578631be34dbc23e792c51aaff8fce6532 30-May-2019 brooks <brooks@FreeBSD.org> makesyscalls.sh: always use absolute path for syscalls.conf

syscalls.conf is included using "." which per the Open Group:

If file does not contain a <slash>, the shell shall use the search
path specified by PATH to find the directory containing file.

POSIX shells don't fall back to the current working directory.

Submitted by: Nathaniel Wesley Filardo <nwf20@cl.cam.ac.uk>
Reviewed by: bdrewery
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D20476
inux/Makefile
ce38ee5193ad12770d1b408f9fd3bdbd24dd6edc 30-May-2019 jchandra <jchandra@FreeBSD.org> arm64 gicv3_its: Fix a typo

Fix 'Cavium' spelling in errata description.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D20418
rm64/gicv3_its.c
03c5d90f7c16375f06d80eff126bd847f155a04b 30-May-2019 jchandra <jchandra@FreeBSD.org> gicv3_its: do LPI init only once per CPU

The initialization required for LPIs (setting up pending tables etc.)
has to be done just once per CPU, even in the case where there are
multiple ITS blocks associated with the CPU.

Add a flag lpi_enabled in the per-cpu distributor info for this and
use it to ensure that we call its_init_cpu_lpi() just once.

This enables us to support platforms where multiple GIC ITS blocks
can generate LPIs to a CPU.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D19844
rm64/gic_v3.c
rm64/gic_v3_var.h
rm64/gicv3_its.c
21280ab5e71deb90c041b0953732506260aa58ac 30-May-2019 jchandra <jchandra@FreeBSD.org> gicv3_its: refactor LPI init into a new function

Move the per-cpu LPI intialization to a separate function. This is
in preparation for a commit that does LPI init only once for a CPU,
even when there are multiple ITS blocks associated with the CPU.

No functional changes in this commit.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D19843
rm64/gicv3_its.c
281e5a5253c88b03a01c73660ee6ef0fb6051d4f 30-May-2019 jchandra <jchandra@FreeBSD.org> gic_v3: consolidate per-cpu redistributor information

Update 'struct gic_redists' to consolidate all per-cpu redistributor
information into a new 'struct redist_pcpu'. Provide a new interface
(GICV3_IVAR_REDIST) for the GIC driver, which can be used to retrieve
the per-cpu data.

This per-cpu redistributor struct will be later used to improve the
GIC ITS setup.

While there, remove some unused fields in gic_v3_var.h interface.
No functional changes.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D19842
rm64/gic_v3.c
rm64/gic_v3_var.h
rm64/gicv3_its.c
fdfda4909e403b42cf25bb5534a515abe40a5a9c 26-May-2019 jchandra <jchandra@FreeBSD.org> arm64 nexus: remove incorrect warning

acpi_config_intr() will be called when an arm64 system booted with ACPI.
We do the interrupt mapping for ACPI interrupts in nexus_acpi_map_intr()
on arm64, so acpi_config_intr() has to just return success without
printing this error message.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D19432
rm64/nexus.c
3038f1af7b5d53f0c56c2b7edc86ad5edf2dcd23 21-May-2019 cem <cem@FreeBSD.org> Include ktr.h in more compilation units

Similar to r348026, exhaustive search for uses of CTRn() and cross reference
ktr.h includes. Where it was obvious that an OS compat header of some kind
included ktr.h indirectly, .c files were left alone. Some of these files
clearly got ktr.h via header pollution in some scenarios, or tinderbox would
not be passing prior to this revision, but go ahead and explicitly include it
in files using it anyway.

Like r348026, these CUs did not show up in tinderbox as missing the include.

Reported by: peterj (arm64/mp_machdep.c)
X-MFC-With: r347984
Sponsored by: Dell EMC Isilon
rm64/mp_machdep.c
rm64/trap.c
250e158ddf52459661439141407bca505d199c19 20-May-2019 cem <cem@FreeBSD.org> Extract eventfilter declarations to sys/_eventfilter.h

This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.

EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).

As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions. The remainder of the patch addresses
adding appropriate includes to fix those files.

LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).

No functional change (intended). Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed. __FreeBSD_version has been bumped.
rm64/gicv3_its.c
rm64/machdep.c
oresight/coresight.c
ockchip/clk/rk_cru.c
3cfe83e1e000b8eeb0bb94d62be1eb5ac7985ece 17-May-2019 dougm <dougm@FreeBSD.org> Implement the ffs and fls functions, and their longer counterparts, in
cpufunc, in terms of __builtin_ffs and the like, for arm64
architectures, and use those, rather than the simple libkern
implementations, in building arm64 kernels.

Tested by: greg_unrelenting.technology (earlier version)
Reviewed by: alc
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D20250
nclude/cpufunc.h
653b035c893fedb7d7531c34b61bab4f7037bd26 16-May-2019 kib <kib@FreeBSD.org> Remove resolver_qual from DEFINE_IFUNC/DEFINE_UIFUNC macros.

In all practical situations, the resolver visibility is static.

Requested by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Approved by: so (emaste)
Differential revision: https://reviews.freebsd.org/D20281
nclude/ifunc.h
b267afaa6b3bc38dd70df8cbcf0fdb6e28d4d085 16-May-2019 tychon <tychon@FreeBSD.org> Allow loading the same DMA address multiple times without any prior
unload for the LinuxKPI.

Reviewed by: kib, zeising
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D20181
rm64/busdma_bounce.c
nclude/bus_dma.h
nclude/bus_dma_impl.h
cc98d74ced94bcda8a5a29d613ab36a9caec6ba3 16-May-2019 jhibbits <jhibbits@FreeBSD.org> arm64: Add the rename interpreter path for compat32 ld-elf

Let arm64 and arm32 dynamic binaries coexist. Match all other compat32
archs.

Reviewed by: manu
Sponsored by: Juniper Networks, Inc
rm64/elf32_machdep.c
142b7761cc8697546dc852e4d22e380463a1a584 16-May-2019 kib <kib@FreeBSD.org> arm64: bzero buffer for ucontext in freebsd32_swapcontext().

This change is the same as r340994 for amd64.

PR: 237922
Submitted by: Young <yangx92@hotmail.com>
MFC after: 3 days
rm64/freebsd32_machdep.c
f7e99603a761273593b99380eb79e865cd274413 13-May-2019 dchagin <dchagin@FreeBSD.org> Linuxulator depends on a fundamental kernel settings such as SMP. Many
of them listed in opt_global.h which is not generated while building
modules outside of a kernel and such modules never match real cofigured
kernel.

So, we should prevent our users from building obviously defective modules.

Therefore, remove the root cause of the building of modules outside of a
kernel - the possibility of building modules with DEBUG or KTR flags.
And remove all of DEBUG printfs as it is incomplete and in threaded
programms not informative, also a half of system call does not have DEBUG
printf. For debuging Linux programms we have dtrace, ktr and ktrace ability.

PR: 222861
Reviewed by: trasz
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D20178
inux/linux.h
inux/linux_sysvec.c
2c4607867d661fceb056907b0230a53d656c26c0 10-May-2019 manu <manu@FreeBSD.org> arm64: rockchip: Don't always put PLL to normal mode

We used to put every PLL in normal mode (meaning that the output would
be the result of the PLL configuration) instead of slow mode (the output
is equal to the external oscillator frequency, 24-26Mhz) but this doesn't
work for most of the PLLs as when we put them into normal mode the registers
configuring the output frequency haven't been set.
Add a normal_mode member in clk_pll_def/clk_pll_sc struct and if it's true
we then set the PLL to normal mode.
For now only set it to the LPLL and BPLL (Little cluster PLL and Big cluster
PLL respectively).

Reviewed by: ganbold
Differential Revision: https://reviews.freebsd.org/D20174
ockchip/clk/rk3399_cru.c
ockchip/clk/rk_clk_pll.c
ockchip/clk/rk_clk_pll.h
d0514c054297cad36305407bbcf5281cf3157ca8 09-May-2019 gallatin <gallatin@FreeBSD.org> Remove IPSEC from GENERIC due to performance issues

Having IPSEC compiled into the kernel imposes a non-trivial
performance penalty on multi-threaded workloads due to IPSEC
refcounting. In my benchmarks of multi-threaded UDP
transmit (connected sockets), I've seen a roughly 20% performance
penalty when the IPSEC option is included in the kernel (16.8Mpps
vs 13.8Mpps with 32 senders on a 14 core / 28 HTT Xeon
2697v3)). This is largely due to key_addref() incrementing and
decrementing an atomic reference count on the default
policy. This cause all CPUs to stall on the same cacheline, as it
bounces between different CPUs.

Given that relatively few users use ipsec, and that it can be
loaded as a module, it seems reasonable to ask those users to
load the ipsec module so as to avoid imposing this penalty on the
GENERIC kernel. Its my hope that this will make FreeBSD look
better in "out of the box" benchmark comparisons with other
operating systems.

Many thanks to ae for fixing auto-loading of ipsec.ko when
ifconfig tries to configure ipsec, and to cy for volunteering
to ensure the the racoon ports will load the ipsec.ko module

Reviewed by: cem, cy, delphij, gnn, jhb, jpaetzel
Differential Revision: https://reviews.freebsd.org/D20163
onf/GENERIC
9028c47957cb9b2c4c6bfc070ac3fb65432af43e 08-May-2019 manu <manu@FreeBSD.org> Add support for USB 3.0 XHCI via ACPI

Ampere eMAG systems have XHCI just described in ACPI, not on PCI.

Submitted by: Greg V <greg@unrelenting.technology>
Reviewed by: andrew
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D19986
onf/GENERIC
0f415eea65ecbddff79f47e67d6bd3bf6346fc6b 08-May-2019 kevans <kevans@FreeBSD.org> tun/tap: merge and rename to `tuntap`

tun(4) and tap(4) share the same general management interface and have a lot
in common. Bugs exist in tap(4) that have been fixed in tun(4), and
vice-versa. Let's reduce the maintenance requirements by merging them
together and using flags to differentiate between the three interface types
(tun, tap, vmnet).

This fixes a couple of tap(4)/vmnet(4) issues right out of the gate:
- tap devices may no longer be destroyed while they're open [0]
- VIMAGE issues already addressed in tun by kp

[0] emaste had removed an easy-panic-button in r240938 due to devdrn
blocking. A naive glance over this leads me to believe that this isn't quite
complete -- destroy_devl will only block while executing d_* functions, but
doesn't block the device from being destroyed while a process has it open.
The latter is the intent of the condvar in tun, so this is "fixed" (for
certain definitions of the word -- it wasn't really broken in tap, it just
wasn't quite ideal).

ifconfig(8) also grew the ability to map an interface name to a kld, so
that `ifconfig {tun,tap}0` can continue to autoload the correct module, and
`ifconfig vmnet0 create` will now autoload the correct module. This is a
low overhead addition.

(MFC commentary)

This may get MFC'd if many bugs in tun(4)/tap(4) are discovered after this,
and how critical they are. Changes after this are likely easily MFC'd
without taking this merge, but the merge will be easier.

I have no plans to do this MFC as of now.

Reviewed by: bcr (manpages), tuexen (testing, syzkaller/packetdrill)
Input also from: melifaro
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D20044
onf/GENERIC
206ba42431462826952c1e457d0b3a3405931a13 07-May-2019 emaste <emaste@FreeBSD.org> make sysent after r347228

Regenerate to add @generated tag in generated files.
inux/linux_proto.h
inux/linux_syscall.h
inux/linux_syscalls.c
inux/linux_sysent.c
inux/linux_systrace_args.c
ef0f4aa09f3be011b38929bd60a6b5b41cc315ea 06-May-2019 emaste <emaste@FreeBSD.org> Reformat arm64 linux syscalls.master per current style

Equivalent to r339958 for sys/kern/syscalls.master.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D14858
inux/syscalls.master
58510286a032c0adf4bb7c6918d290d829a2b347 04-May-2019 kib <kib@FreeBSD.org> arm64: Properly restore PAN when done with userspace access in casueword.

Approved by: andrew
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
rm64/support.S
e4bfebd8e9e6fd32965c88f8d70ac6f37a421b27 04-May-2019 ganbold <ganbold@FreeBSD.org> Add emmc clock definitions for Rockchip RK3399 SoC.
ockchip/clk/rk3399_cru.c
90cdaa0665b339584f85be1ce3b5a4ae4db3a185 03-May-2019 dchagin <dchagin@FreeBSD.org> In order to reduce duplication between MD parts of the Linuxulator
move bits that are MI out into the headers in compat/linux.
For that remove bogus _packed attribute from struct l_sockaddr
and use MI types for struct members.

And continue to move into the linux_common module a code that is
intended for both Linuxulator modules (both instruction set - 32 & 64 bit)
or for external modules like linsysfs or linprocfs.

To avoid header pollution introduce new sys/compat/linux_common.h header.

Reviewed by: emaste
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D20137
inux/linux.h
09c46e6d56e68d1f089fb428d04fba33c7a5bb89 02-May-2019 cem <cem@FreeBSD.org> Add a COMPAT_FREEBSD12 kernel option.

Use it wherever COMPAT_FREEBSD11 is currently specified, like r309749.

Reviewed by: imp, jhb, markj
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D20120
onf/GENERIC
1f1b2ec3c5cca379778b2a17ccccdf976af74904 02-May-2019 manu <manu@FreeBSD.org> arm64: Add support for NanoPI NEO2

Add overlay files and activate devicetree file for NanoPi NEO2 featuring
Allwinner H5 ARM64 core.
To enable sound, dma and codec drivers are enabled for build.

Submitted by: Manuel Stühn (freebsdnewbie@freenet.de)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D20129
onf/GENERIC
b278d30ff05ca3dcc9cb7ad785a0e00ab7aaf570 01-May-2019 andrew <andrew@FreeBSD.org> Restore x18 in efi_arch_leave.

Some UEFI implementations trash this register and, as we use it as a
platform register, the kernel doesn't save it before calling into the UEFI
runtime services. As we have a copy in tpidr_el1 restore from there when
exiting the EFI environment.

PR: 237234, 237055
Reviewed by: manu
Tested On: Ampere eMAG
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Sponsored by: Ampere Computing (hardware)
Differential Revision: https://reviews.freebsd.org/D20127
rm64/efirt_machdep.c
dbc342b6b49fd2ae01ca5afc10d4332e9fdbb2d0 01-May-2019 ganbold <ganbold@FreeBSD.org> Add a hw.model sysctl oid for arm64 which reports the CPU model similar to armv6/7.

Reviewed by: andrew, manu
Differential Revision: https://reviews.freebsd.org/D20123
rm64/identcpu.c
a7e393156a4e5a931c5abeaabf62ef2cbdb118ed 20-Apr-2019 ganbold <ganbold@FreeBSD.org> Add SY8106A Buck Regulator and Allwinner CIR devices to GENERIC arm64 kernel.
onf/GENERIC
c30ced85d126d9898bc266339fa9a620e063e52d 29-Mar-2019 jkim <jkim@FreeBSD.org> Merge ACPICA 20190329.
cpica/acpi_machdep.c
0c946c6523f348463eec486d615dca999801373c 25-Mar-2019 andrew <andrew@FreeBSD.org> Sort printing of the ID registers on arm64 to be identical to the
documentation. This will simplify checking new fields when they are added.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL
rm64/identcpu.c
d8bc0028e763fa169529c43adb234fdf3c69dabc 23-Mar-2019 mw <mw@FreeBSD.org> Enable etherswitchcfg and e6000sw driver in arm64 build

After latest binding update, this patch enables usage of
the switch on Armada 3720 EspressoBin, so compile it
by default with arm64 GENERIC.

A patch was extracted from https://reviews.freebsd.org/D19036

Submitted by: Bert JW Regeer <xistence@0x58.com>
Reviewed by: manu
onf/GENERIC
be4e05cbf35fb3007a65f7924d976b6c0f70e9ef 22-Mar-2019 cperciva <cperciva@FreeBSD.org> Add nvme support to the arm64 GENERIC kernel.

Submitted by: Greg V
Differential Revision: https://reviews.freebsd.org/D19657
onf/GENERIC
6bc222605a21640fd15291ae9d91df83e882ad6b 21-Mar-2019 mw <mw@FreeBSD.org> Add bus_release_resource() method to nexus on arm64

The nexus module was missing method for releasing bus resources. As a
result, it couldn't be released and the bus_release_resource() call would
return ENXIO.

Next call to bus_alloc_resource() for the same resource was returning
error, because it wasn't released previously and it was still busy.

The implementation of the nexus_release_resource() is the same as for
arm architecture.

Submitted by: Michal Krawczyk <mk@semihalf.com>
Reported-by: Greg V <greg@unrelenting.technology>
Tested-by: cperciva, Greg V <greg@unrelenting.technology>
Obtained from: Semihalf
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Differential revision: https://reviews.freebsd.org/D19641
rm64/nexus.c
71140c5be468621cc47b728e33e8b875b0c0b608 16-Mar-2019 kib <kib@FreeBSD.org> amd64 KPTI: add control from procctl(2).

Add the infrastructure to allow MD procctl(2) commands, and use it to
introduce amd64 PTI control and reporting. PTI mode cannot be
modified for existing pmap, the knob controls PTI of the new vmspace
created on exec.

Requested by: jhb
Reviewed by: jhb, markj (previous version)
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D19514
rm64/vm_machdep.c
nclude/procctl.h
9b5673f116be94488c7a589131864a3354001542 16-Mar-2019 kib <kib@FreeBSD.org> amd64: Add md process flags and first P_MD_PTI flag.

PTI mode for the process pmap on exec is activated iff P_MD_PTI is set.

On exec, the existing vmspace can be reused only if pti mode of the
pmap matches the P_MD_PTI flag of the process. Add MD
cpu_exec_vmspace_reuse() callback for exec_new_vmspace() which can
vetoed reuse of the existing vmspace.

MFC note: md_flags change struct proc KBI.

Reviewed by: jhb, markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D19514
rm64/vm_machdep.c
1a536e4f2450047129536463dda63fcd52ff7319 01-Mar-2019 trasz <trasz@FreeBSD.org> Remove sv_pagesize, originally introduced with r100384.

In all of the architectures we have today, we always use PAGE_SIZE.
While in theory one could define different things, none of the
current architectures do, even the ones that have transitioned from
32-bit to 64-bit like i386 and arm. Some ancient mips binaries on
other systems used 8k instead of 4k, but we don't support running
those and likely never will due to their age and obscurity.

Reviewed by: imp (who also contributed the commit message)
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D19280
rm64/elf32_machdep.c
rm64/elf_machdep.c
loudabi32/cloudabi32_sysvec.c
loudabi64/cloudabi64_sysvec.c
inux/linux_sysvec.c
68a8109f870d476d2864b0d023ff0c9bae2b2acf 01-Mar-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk3399_pll: Fix the recalc function

The plls frequency are now correctly calculated in fractional mode
and integer mode.
While here add some debug printfs (disabled by default)
Tested with powerd on the little cluster on a RockPro64.

MFC after: 1 week
ockchip/clk/rk_clk_pll.c
21d07a1dd3883e98dbdee3bb9e0f5a809bce2a86 28-Feb-2019 andrew <andrew@FreeBSD.org> Add the hw.ncpu tunable to arm64.

This allows us to limit the number of CPUs to use, e.g. to debug problems
seen when enabling multiple clusters.

Reviewed by: manu
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D19404
rm64/mp_machdep.c
ab251adb8bb875cdde6094772c07219f5205a051 27-Feb-2019 manu <manu@FreeBSD.org> xhci_mv: Move the driver to generic_xhci

Marvell XHCI is in fact generic-xhci, so move the driver and
add the compatible string.
While here, get and enable the phy if the dtb provide one.
The xhci bindings state that phys should be in a 'phys' property but
Marvell DTS uses 'usb-phy', only add support for 'usb-phy' for now.

Sponsored-by: Rubicon Communications, LCC ("Netgate")
onf/GENERIC
101bf68bc2d41801ed8382b7791218ff131560f2 27-Feb-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk3399_clk: Add sd clock definitions

MFC after: 1 week
ockchip/clk/rk3399_cru.c
9a5226a11b8a8e5b62ec170d7c462f4c8ce36ff6 27-Feb-2019 manu <manu@FreeBSD.org> arm64: rockchip: clk_pll: Multiple improvement

Remove the mode_val from the clock definition as it's a bit unreadable.
Use mode_shift to represent which bit control the mode in the register.
Simplify some case where we can avoid a register read before changing it.
Set the PLL back to normal mode after the PLL have stabilized.

Discussed with: mmel
MFC after: 1 week
ockchip/clk/rk3328_cru.c
ockchip/clk/rk_clk_pll.c
ockchip/clk/rk_clk_pll.h
4d0588ce6ed1da557fc430469a417a54a7647307 26-Feb-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk3399_pll: Fix copy paste

RK3399 PLLs don't have mode_reg, use the correct register.

MFC after: 1 week
ockchip/clk/rk_clk_pll.c
2aca62577482b67eec9d8d837d578b36545e6e3c 26-Feb-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk3399_pll: Switch to slow mode when changing the freq

Like r344578 but for RK3399.
This solve some hangs when switching between frequency.

MFC after: 1 week
ockchip/clk/rk_clk_pll.c
f5b0c5881353b2f8e201c18e25ae2b0a346876fe 26-Feb-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk_pinctrl: Fix two banks in RK3328

The last two banks don't have 3 bits for the pin function but only 2.
This fixes eMMC on the Rock64.

MFC after: 1 week
ockchip/rk_pinctrl.c
8822e00f7dd83f4e9982358db50353a10413ad82 26-Feb-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk805: Map the regulator

No map function was provided before so every regulator lookup resolved
the regulator with id 1, as it uses the default mapper, which is wrong.
Correctly map the regulators.
While here remove some debug printfs and make them disable by default.

MFC after: 1 week
ockchip/rk805.c
ockchip/rk805reg.h
836eb970450e67724f998bf60d68823efc69fbde 26-Feb-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk805: Add LDO regulators

Add the 3 LDO regulator found in the RK805 Power Management IC.

MFC after: 1 week
ockchip/rk805.c
ed8d62c03871114d692cf8e407ac54a3cf52a945 26-Feb-2019 manu <manu@FreeBSD.org> arm64: rockchip: rk3328_pll: Multiple improvement

RockChip clocks register have a write mask in the upper 16 bits, if a 1
is present the corresponding bit in the lower 16 ones is set.
Use this instead of always setting the mask to 0xFFFF0000.
This avoids a read of the register.
While here, when switching PLL frequency, first switch it to slow mode.
When set to slow mode the PLL clock will be the external oscillator.
Changing the PLL parameters while its output is used can cause hang (sometimes).

MFC after: 1 week
ockchip/clk/rk_clk_pll.c
ockchip/clk/rk_clk_pll.h
5761a7675b8332330696608297ed79162bfe0d43 26-Feb-2019 manu <manu@FreeBSD.org> arm64: rockchip: clk: ARM CLK improvement

RockChip clocks register have a write mask in the upper 16 bits, if a 1
is present the corresponding bit in the lower 16 ones is set.
Use this instead of always setting the mask to 0xFFFF0000.
This avoids a read of the register.
While here set the parent after changing its freqeuncy, this reduce the time
between changing the parent and changing the divider for the arm clock.

MFC after: 1 week
ockchip/clk/rk_clk_armclk.c
ockchip/clk/rk_clk_armclk.h
c78c3e1e8a538966bed610a2113c81aa4251d104 26-Feb-2019 manu <manu@FreeBSD.org> arm64: rockchip: clk: rk_clk_composite: Properly use the mask bits

RockChip clocks register have a write mask in the upper 16 bits, if a 1
is present the corresponding bit in the lower 16 ones is set.
Use this instead of always setting the mask to 0xFFFF0000.
This avoids a read of the register.
While here add some debug printf useful for debuging clock problems

MFC after: 1 week
ockchip/clk/rk_clk_composite.c
ockchip/clk/rk_clk_composite.h
10e5aeb6eb5aee6f5e68c10e693364fbe36efb48 25-Feb-2019 manu <manu@FreeBSD.org> arm64: rockchip: clk: Set the write mask when setting the clock mux

RockChip clocks have a write mask in the upper 16bits of the mux register
which wasn't set in the set_mux function.
Also the wrong parent was tested instead of the real current one, when
switch parent, test with the current one before.

Pointy Hat: manu
MFC after: 1 week
ockchip/clk/rk_clk_composite.c
4adce57d6f1aea90a8de719a50a814ae9e9e0d49 20-Feb-2019 kib <kib@FreeBSD.org> Add kernel support for Intel userspace protection keys feature on
Skylake Xeons.

See SDM rev. 68 Vol 3 4.6.2 Protection Keys and the description of the
RDPKRU and WRPKRU instructions.

Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D18893
nclude/pmap.h
0af6ee1daf60159b89727191fe9eff7c3c0f1b0e 14-Feb-2019 kib <kib@FreeBSD.org> Enable enabling ASLR on non-x86 architectures.

Discussed with: emaste
Sponsored by: The FreeBSD Foundation
rm64/elf_machdep.c
329de1d73b0aeaee6d10480e016a940d90062644 10-Feb-2019 manu <manu@FreeBSD.org> arm64: Fix compile when removing SOC_ROCKCHIP_* options

Make every rockchip file depend on the multiple soc_rockchip options
While here make rk_i2c and rk_gpio depend on their device options.

Reported by: sbruno
onf/GENERIC
78a5bf893531c62a03fb2336faa4c92d482c7108 10-Feb-2019 cem <cem@FreeBSD.org> Revert r343713 temporarily

The COVERAGE option breaks xtoolchain-gcc GENERIC kernel early boot
extremely badly and hasn't been fixed for the ~week since it was committed.
Please enable for GENERIC only when it doesn't do that.

Related fallout reported by: lwhsu, tuexen (pr 235611)
onf/GENERIC
493354932e6be7eefcfbe2807457dc1a2cc5f530 07-Feb-2019 andrew <andrew@FreeBSD.org> Add missing data barriers after storeing a new valid pagetable entry.

When moving from an invalid to a valid entry we don't need to invalidate
the tlb, however we do need to ensure the store is ordered before later
memory accesses. This is because this later access may be to a virtual
address within the newly mapped region.

Add the needed barriers to places where we don't later invalidate the
tlb. When we do invalidate the tlb there will be a barrier to correctly
order this.

This fixes a panic on boot on ThunderX2 when INVARIANTS is turned off:
panic: vm_fault_hold: fault on nofault entry, addr: 0xffff000040c11000

Reported by: jchandra
Tested by: jchandra
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D19097
rm64/pmap.c
2b51ea18e4a97c985180fb6d3e1e0911bad7a718 07-Feb-2019 andrew <andrew@FreeBSD.org> Add a missing data barrier to the start of arm64_tlb_flushID.

We need to ensure the page table store has happened before the tlbi.

Reported by: jchandra
Tested by: jchandra
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D19097
rm64/cpufunc_asm.S
5053141bcbea708eda40c20cc60a120f3a4c87f0 07-Feb-2019 jchandra <jchandra@FreeBSD.org> arm64 gicv3: add IORT and NUMA support

acpi_iort.c has added support to query GIC proximity and MSI XREF
ID for GIC ITS blocks. Use this when GIC ITS blocks are initialized
from ACPI.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D18003
rm64/gic_v3_acpi.c
rm64/gic_v3_var.h
rm64/gicv3_its.c
47b24738b81eb479e169fe248afda102731a5207 07-Feb-2019 jchandra <jchandra@FreeBSD.org> arm64 acpi: Add support for IORT table

Add new file arm64/acpica/acpi_iort.c to support the "IO Remapping
Table" (IORT). The table is specified in ARM document "ARM DEN 0049D"
titled "IO Remapping Table Platform Design Document". The IORT table
has information on the associations between PCI root complexes, SMMU
blocks and GIC ITS blocks in the system.

The changes are to parse and save the information in the IORT table.
The API to use this information is added to sys/dev/acpica/acpivar.h.

The acpi_iort.c also has code to check the GIC ITS nodes seen in the
IORT table with corresponding entries in MADT table (for validity)
and with entries in SRAT table (for proximity information).

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D18002
cpica/acpi_iort.c
22d4af1621c5c582195aef3f7d6e624ddcb235b8 03-Feb-2019 andrew <andrew@FreeBSD.org> Enable COVERAGE and KCOV by default on arm64 and amd64.

This allows userspace to trace the kernel using the coverage sanitizer
found in clang. It will also allow other coverage tools to be built as
modules and attach into the same framework.

Sponsored by: DARPA, AFRL
onf/GENERIC
1b7795b92a0803745f14befdfa3787a355e7d2d9 31-Jan-2019 kib <kib@FreeBSD.org> Make iflib a loadable module.

iflib is already a module, but it is unconditionally compiled into the
kernel. There are drivers which do not need iflib(4), and there are
situations where somebody might not want iflib in kernel because of
using the corresponding driver as module.

Reviewed by: marius
Discussed with: erj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D19041
onf/GENERIC
ef2c4f184e41e284f13b0e06b1d3c5d2143b9dbf 29-Jan-2019 obrien <obrien@FreeBSD.org> Follow arm[32] and sparc64 KAPI and provide the FreeBSD standard spelling
across all architectures for this header.

Reviewed by: stevek
Obtained from: Juniper Networks
nclude/sigframe.h
be20ad6345abc7ba2c84e91b37e87376ce36f05b 29-Jan-2019 andrew <andrew@FreeBSD.org> Extract the coverage sanitizer KPI to a new file.

This will allow multiple consumers of the coverage data to be compiled
into the kernel together. The only requirement is only one can be
registered at a given point in time, however it is expected they will
only register when the coverage data is needed.

A new kernel conflig option COVERAGE is added. This will allow kcov to
become a module that can be loaded as needed, or compiled into the
kernel.

While here clean up the #include style a little.

Reviewed by: kib
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D18955
onf/GENERIC
onf/GENERIC-NODEBUG
7b97f0ab8331e82bf133476900bdd8bbe66d847b 16-Jan-2019 gonzo <gonzo@FreeBSD.org> [rpi] Reorganize spigen(4) overlays for Raspberry Pi

- Remove CS=2 entry from spigen-rpi2 since it didn't work
- Add spigen-rpi3 overlay for Raspberry Pi 3
- Enable rpi overlay modules for GENERIC kernel on aarch64

PR: 233489
Submitted by: bobf@mrp3.com
Reviewed by: db
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D16088
onf/GENERIC
10d58df29fb3adde1c813d4ab09e39a674e83a70 15-Jan-2019 andrew <andrew@FreeBSD.org> Ensure the I-Cache is correctly handled in arm64_icache_sync_range

The cache_handle_range macro to handle the arm64 instruction and data
cache operations would return when it was complete. This causes problems
for arm64_icache_sync_range and arm64_icache_sync_range_checked as they
assume they can execute the i-cache handling instruction after it has been
called.

Fix this by making this assumption correct.

While here add missing instruction barriers and adjust the style to
match the rest of the assembly.

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D18838
rm64/cpufunc_asm.S
5de4964761f17dbc0cffdc210e29a67a6cfa11df 13-Jan-2019 cognet <cognet@FreeBSD.org> Don't forget to add the needed #includes.

Pointy hat to: cognet
rm64/cpufunc_asm.S
f68e64c61223b429e018adb7fb33cfc0a0443052 13-Jan-2019 cognet <cognet@FreeBSD.org> Introduce cpu_icache_sync_range_checked(), that does the same thing as
cpu_icache_sync_range(), except that it sets pcb_onfault to catch any page
fault, as doing cache maintenance operations for non-mapped generates a
data abort, and use it in freebsd32_sysarch(), so that a userland program
attempting to sync the icache with unmapped addresses doesn't crash the
kernel.

Spotted out by: andrew
rm64/cpufunc_asm.S
rm64/freebsd32_machdep.c
nclude/cpufunc.h
38d007eb07876353a9b3006284c96fc81059172c 13-Jan-2019 cognet <cognet@FreeBSD.org> Impleent COMPAT_FREEBSD32 for arm64.
This is based on early work by andrew@.
rm64/elf32_machdep.c
rm64/freebsd32_machdep.c
rm64/identcpu.c
rm64/locore.S
rm64/machdep.c
rm64/undefined.c
rm64/vm_machdep.c
onf/GENERIC
nclude/armreg.h
nclude/elf.h
nclude/frame.h
nclude/reg.h
nclude/ucontext.h
nclude/vfp.h
5e0e456d9f080474dddb07ba390b47412a24cf0c 12-Jan-2019 andrew <andrew@FreeBSD.org> Add support for the Clang Coverage Sanitizer in the kernel (KCOV).

When building with KCOV enabled the compiler will insert function calls
to probes allowing us to trace the execution of the kernel from userspace.
These probes are on function entry (trace-pc) and on comparison operations
(trace-cmp).

Userspace can enable the use of these probes on a single kernel thread with
an ioctl interface. It can allocate space for the probe with KIOSETBUFSIZE,
then mmap the allocated buffer and enable tracing with KIOENABLE, with the
trace mode being passed in as the int argument. When complete KIODISABLE
is used to disable tracing.

The first item in the buffer is the number of trace event that have
happened. Userspace can write 0 to this to reset the tracing, and is
expected to do so on first use.

The format of the buffer depends on the trace mode. When in PC tracing just
the return address of the probe is stored. Under comparison tracing the
comparison type, the two arguments, and the return address are traced. The
former method uses on entry per trace event, while the later uses 4. As
such they are incompatible so only a single mode may be enabled.

KCOV is expected to help fuzzing the kernel, and while in development has
already found a number of issues. It is required for the syzkaller system
call fuzzer [1]. Other kernel fuzzers could also make use of it, either
with the current interface, or by extending it with new modes.

A man page is currently being worked on and is expected to be committed
soon, however having the code in the kernel now is useful for other
developers to use.

[1] https://github.com/google/syzkaller

Submitted by: Mitchell Horne <mhorne063@gmail.com> (Earlier version)
Reviewed by: kib
Testing by: tuexen
Sponsored by: DARPA, AFRL
Sponsored by: The FreeBSD Foundation (Mitchell Horne)
Differential Revision: https://reviews.freebsd.org/D14599
onf/GENERIC
58ce42c5d771494d37d3868d9e2e785e274d61d6 11-Jan-2019 andrew <andrew@FreeBSD.org> Fix the location of td->td_frame at the top of the kernel stack.

In cpu_thread_alloc we would allocate space for the trap frame at the top of
the kernel stack. This is just below the pcb, however due to a missing cast
the pointer arithmetic would use the pcb size, not the trapframe size. As
the pcb is larger than the trapframe this is safe, however later in cpu_fork
we include the case leading to the two disagreeing on the location.

Fix by using the same arithmetic in both locations.

Found by: An early KASAN patch
Sponsored by: DARPA, AFRL
rm64/vm_machdep.c
8aa0301c6331cba4673a2ae470b7f4625eaf434a 11-Jan-2019 fsu <fsu@FreeBSD.org> Fix errno values returned from DUMMY_XATTR linuxulator calls

Reported by: weiss@uni-mainz.de
Reviewed by: markj
MFC after: 1 day
Differential Revision: https://reviews.freebsd.org/D18812
inux/linux_dummy.c
2aef96ef0a665d80502d6a212a2c09b44d4de7a4 10-Jan-2019 andrew <andrew@FreeBSD.org> Fix a comment, pushed onto is two words.

While here make the comments sentences.

Sponsored by: DARPA, AFRL
nclude/frame.h
4dd54d8bd81fd722e043435cf9a9ed8b37c52583 27-Dec-2018 andrew <andrew@FreeBSD.org> Pass VM_PROT_EXECUTE to vm_fault for instruction faults.

We need to tell vm_fault the reason for the fault was because we tried to
execute from the memory location. Without this it may return with success
as we only request read-only memory, then we return to the same location
and try to execute from the same memory address. This leads to an infinite
loop raising the same fault and returning to the same invalid location.

MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D18511
rm64/trap.c
c39e5a04869dd7380aba830f3e95d7d999d87c5a 19-Dec-2018 mjg <mjg@FreeBSD.org> Remove iBCS2, part2: general kernel

Reviewed by: kib (previous version)
Sponsored by: The FreeBSD Foundation
rm64/elf_machdep.c
rm64/trap.c
inux/linux_sysvec.c
a3af78d3ad78d0ea966445626117ae25628ee9a0 12-Dec-2018 manu <manu@FreeBSD.org> mv_thermal: Add thermal driver for AP806 and CP110 thermal sensor

Sponsored by: Rubicon Communications, LLC ("Netgate")
onf/GENERIC
46170dc3e9ee64e903ffbf224b58bf23f0f0b8d6 12-Dec-2018 manu <manu@FreeBSD.org> arm64: Add mv_cp110_icu and mv_cp110_gicp

icu is a interrupt concentrator in the CP110 block and gicp
is a gic extension to allow interrupts in the CP block to be turned
into GIC SPI interrupts

Sponsored by: Rubicon Communications, LLC ("Netgate")
onf/GENERIC
c6fea0d9a5cc728586fa78edcaf196af4f26a2e3 12-Dec-2018 manu <manu@FreeBSD.org> arm64: mv_gpio: Add Marvell 8K support

While here put the interrupts setup in it's own function

Sponsored by: Rubicon Communications, LCC ("Netgate")
onf/GENERIC
7abba6b3dd6826a77cffb5d1ef0b1e12a2a14004 12-Dec-2018 manu <manu@FreeBSD.org> arm64: mvebu_pinctrl: Add driver for Marvell Pinmux Controller

Add a driver compatible with Marvell mvebu-pinctrl and add ap806-pinctrl
support.

Sponsored by: Rubicon Communications, LCC ("Netgate")
onf/GENERIC
a3e7e990781e5fba79f63b78c1e0b9d13c872cc1 12-Dec-2018 manu <manu@FreeBSD.org> arm64: Add new SoC type MARVELL_8K

Sponsored by: Rubicon Communications, LLC ("Netgate")
onf/GENERIC
f7c033a4e84739fae5fc7b888944af6914c66575 12-Dec-2018 manu <manu@FreeBSD.org> arm64: allwinner: Add pwm driver

Add a pwm driver for Allwinner PWM
Add pwm and aw_pwm to the GENERIC kernel
onf/GENERIC
3e3733a17799e133ba2b542639bd347198f77860 11-Dec-2018 andrew <andrew@FreeBSD.org> Only read the ACPI proximity tabled on arm64 when we are booting from
ACPI.

Sponsored by: DARPA, AFRL
cpica/acpi_machdep.c
5a913206e062653f12d3e0f6a24e991a6d7c9b15 08-Dec-2018 jchandra <jchandra@FreeBSD.org> arm64: add ACPI based NUMA support

Use the newly defined SRAT/SLIT parsing APIs in arm64 to support
ACPI based NUMA.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D17943
cpica/acpi_machdep.c
rm64/mp_machdep.c
4fd47b16f422ddfd9ee311ed6eaf57345d709601 02-Dec-2018 andreast <andreast@FreeBSD.org> Build the dtb for the rock64 board.

Reviewed by: manu@
onf/GENERIC
0fec4403f4e778c2747490878c119df68efc3da1 01-Dec-2018 manu <manu@FreeBSD.org> Add Silergy SYR827 PMIC driver

SYR827 is a PMIC that can output a voltage from 0.7125V to 1.5V in 12.5mV steps
It's controlled via I2C.

MFC after: 1 month
onf/GENERIC
cdbfc50a64919e3d5f0e76d4593ad3f1cf454980 01-Dec-2018 manu <manu@FreeBSD.org> arm64: rockchip: rk805: Add basic support for RK808 PMIC

RK808 PMIC is the companion chip for RK3399 SoC.
Add basic regulator support in RK805 since they are similar.

MFC after: 1 month
ockchip/rk805.c
ockchip/rk805reg.h
1e3a6a295fa78316faded74e5fe3e7f5cb629778 01-Dec-2018 manu <manu@FreeBSD.org> arm64: rockchip: rk_i2c: Use correct clock

While here add RK3399 support and call clk_set_assigned to set the correct
clock set in the DTS.

MFC after: 1 month
ockchip/rk_i2c.c
eb7da2ff3a6e32c2c52f8fc50bdf776564a9660c 01-Dec-2018 manu <manu@FreeBSD.org> arm64/rockchip: add RK3399 support

Add CRU (Clock and Reset Unit) driver for RK3399.
Add support in rk_pinctrl driver.

Submitted by: Greg V <greg@unrelenting.technology> (Original version)
Differential Revision: https://reviews.freebsd.org/D16732

MFC after: 1 month
onf/GENERIC
ockchip/clk/rk3399_cru.c
ockchip/clk/rk3399_pmucru.c
ockchip/if_dwc_rk.c
ockchip/rk_grf.c
ockchip/rk_pinctrl.c
61544c68b3e80fca624f9c82b78d89d1612cb101 01-Dec-2018 manu <manu@FreeBSD.org> arm64: rockchip: Add RK3399_CLK_PLL

PLLs on the RK3399 are different than the ones on the RK3328.
Add a new type and some dedicated recalc and set_freq functions.
Rename the RK3328 dedicated rk_clk_pll function with rk3328_ prefix.

MFC after: 1 month
ockchip/clk/rk3328_cru.c
ockchip/clk/rk_clk_pll.c
ockchip/clk/rk_clk_pll.h
ockchip/clk/rk_cru.c
ockchip/clk/rk_cru.h
0e4f4afae7b6d1fe77ec414775e04d89db2d370f 30-Nov-2018 vangyzen <vangyzen@FreeBSD.org> Fix reporting of SS_ONSTACK

Fix reporting of SS_ONSTACK in nested signal delivery when sigaltstack()
is used on some architectures.

Add a unit test for this. I tested the test by introducing the bug
on amd64. I did not test it on other architectures.

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D18347
rm64/machdep.c
c0ab163d39f1eaae0be486bac0094ba207f6d451 29-Nov-2018 manu <manu@FreeBSD.org> arm64: rockchip: armclk: Do not change parent freq if CLK_SET_DRYRUN is set

MFC after: 3 days
ockchip/clk/rk_clk_armclk.c
0dd2c459cb8b825472baed5fe06d1866d862e7f3 28-Nov-2018 manu <manu@FreeBSD.org> arm64: Add rk_i2c and rk805 to GENERIC

MFC after: 1 month
onf/GENERIC
69fcb8bcc6fd94a3cee34afd9abf9f16e2653127 28-Nov-2018 manu <manu@FreeBSD.org> Add RK805 PMIC Support

RK805 is the companion PMIC for RK3328 SoC.
Add a driver for it with most of it's regulators supported.

MFC after: 1 month
ockchip/rk805.c
ockchip/rk805reg.h
d298f98568a648d5471f2a55aa81296785512fdb 28-Nov-2018 manu <manu@FreeBSD.org> rk_clk_composite: Fix set_mux

Pointy Hat: manu
ockchip/clk/rk_clk_composite.c
3887aaa49bdd09f59836edd83bdf20e9f90288a5 26-Nov-2018 vangyzen <vangyzen@FreeBSD.org> Prevent kernel stack disclosure in signal delivery

On arm64 and riscv platforms, sendsig() failed to zero the signal
frame before copying it out to userspace. Zero it.

On arm, I believe all the contents of the frame were initialized,
so there was no disclosure. However, explicitly zero the whole frame
because that fact could inadvertently change in the future,
it's more clear to the reader, and I could be wrong in the first place.

MFC after: 2 days
Security: similar to FreeBSD-EN-18:12.mem and CVE-2018-17155
Sponsored by: Dell EMC Isilon
rm64/machdep.c
1df86997a0cc579d451d15420058adf9c0b93077 26-Nov-2018 manu <manu@FreeBSD.org> arm64: Add evdev support to GENERIC
onf/GENERIC
7cafe98f916249c819fb28c75979f6ea8fc86042 20-Nov-2018 markj <markj@FreeBSD.org> Handle kernel superpage mappings in pmap_remove_l2().

PR: 233088
Reviewed by: alc, andrew, kib
Tested by: sbruno
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17981
rm64/pmap.c
84cb993b781f7f9add02cb4a5de468a143c170f7 19-Nov-2018 jchandra <jchandra@FreeBSD.org> gitv3_its: fixes for multiple GIC ITS blocks

First pass of support for multiple GIC ITS blocks with ACPI.
Changes are to:
* register the correct subset of interrupts with pic_register
in case of ACPI.
* initialize just the cpu interface for the first ITS, when
domain information is not avialable. This has to be done
until we split the per-CPU init to do LPI setup just once.
* remove duplicate check for the GIC ITS domain, the sc_cpus
are setup from domain, so the check again in per-CPU init
seems unnecessary.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D17841
rm64/gicv3_its.c
85b07f0305cf96494bd7997b472a4dcc1cdd5fd7 07-Nov-2018 jhb <jhb@FreeBSD.org> Drop the legacy ELF brandinfo for the old rtld from arm64 and riscv.

These architectures never shipped binaries with an rtld path of
/usr/libexec/ld-elf.so.1.

Reviewed by: markj
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D17876
rm64/elf_machdep.c
ddce67d5c4f394dbc5c8993b57cfe04d861ffa9f 06-Nov-2018 andrew <andrew@FreeBSD.org> Add the KUBSAN options to the arm64 and amd64 GENERIC kernel config files.
As the kernel file size may be too large to run with a stock loader comment
them out for now.

Sponsored by: DARPA, AFRL
onf/GENERIC
81a93c88249dabe233478017a5384e32a95d939d 05-Nov-2018 jhb <jhb@FreeBSD.org> Add a KPI for the delay while spinning on a spin lock.

Replace a call to DELAY(1) with a new cpu_lock_delay() KPI. Currently
cpu_lock_delay() is defined to DELAY(1) on all platforms. However,
platforms with a DELAY() implementation that uses spin locks should
implement a custom cpu_lock_delay() doesn't use locks.

Reviewed by: kib
MFC after: 3 days
nclude/cpu.h
d180d56f388cfca83d778e2ca1e9cc8c93356233 01-Nov-2018 jhb <jhb@FreeBSD.org> Don't enter DDB for fatal traps before panic by default.

Add a new 'debugger_on_trap' knob separate from 'debugger_on_panic'
and make the calls to kdb_trap() in MD fatal trap handlers prior to
calling panic() conditional on this new knob instead of
'debugger_on_panic'. Disable the new knob by default. Developers who
wish to recover from a fatal fault by adjusting saved register state
and retrying the faulting instruction can still do so by enabling the
new knob. However, for the more common case this makes the user
experience for panics due to a fatal fault match the user experience
for other panics, e.g. 'c' in DDB will generate a crash dump and
reboot the system rather than being stuck in an infinite loop of fatal
fault messages and DDB prompts.

Reviewed by: kib, avg
MFC after: 2 months
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D17768
rm64/trap.c
9425249c0dd229ab5cb776fe68bf469570de9ad0 01-Nov-2018 andrew <andrew@FreeBSD.org> Add the ARMv8.3 SCTLR_EL1 fields.

While here tag which architecture release fields were added and remove a
field that only existed in very early releases of the ARMv8 spec.

Sponsored by: DARPA, AFRL
rm64/locore.S
nclude/armreg.h
528d61996fcc40088e5ac9078aa2df7c3e654db4 01-Nov-2018 andrew <andrew@FreeBSD.org> Add the ARMv8.3 HCR_EL2 register fields.

MFC after: 1 month
Sponsored by: DARPA, AFRL
nclude/hypervisor.h
b0b341f2ef5262b9d0732526fd73df6b353f2fdf 01-Nov-2018 andrew <andrew@FreeBSD.org> Use the correct offsets for the trap frame in fork_trampoline.

Sponsored by: DARPA, AFRL
rm64/swtch.S
17b90b88bed1ae0b7b5b19eb4c8d9fd98b3ce0a7 31-Oct-2018 kevans <kevans@FreeBSD.org> Compile in VERBOSE_SYSINIT support by default, remain silent by default

The loader tunable 'debug.verbose_sysinit' may be used to toggle verbosity.
This is added to the debugging section of these kernconfs to be turned off
in stable branches for clarity of intent.

MFC after: never
onf/GENERIC
b7e68fddc0a5eb8f1f12a1dc3ec4a3ff1e053e42 31-Oct-2018 andrew <andrew@FreeBSD.org> Always set the MP_QUIRK_CPULIST quirk under ACPI. This needs a run time
check to only set it for emulators as the CPU list may be changed when
the emulator starts. Until this is working just always set it.

Sponsored by: DARPA, AFRL
rm64/mp_machdep.c
3ed26f3c08d48cddae39af13c7ed3bad0b86a8ea 31-Oct-2018 andrew <andrew@FreeBSD.org> Use pmap_invalidate_all rather than invalidating 512 level 2 entries in
the early pmap_mapbios/unmapbios code. It is even worse when there are
multiple L2 entries to handle as we would need to iterate over all pages.

Sponsored by: DARPA, AFRL
rm64/pmap.c
f34bbdd22f1d2e1a664893814b04004536ea4b6d 31-Oct-2018 andrew <andrew@FreeBSD.org> Remove function prototypes for functions removed in r339943.

Sponsored by: DARPA, AFRL
rm64/mp_machdep.c
e69beb989fb62e05f1687ee286e00c6efcebad6f 31-Oct-2018 andrew <andrew@FreeBSD.org> Fix some style(9) issues in the arm64 pmap_mapbios/unmapbios. Split lines
when they are too long.

Sponsored by: DARPA, AFRL
rm64/pmap.c
906e69828341f9ef8b4616dcde215cd6462fca91 31-Oct-2018 andrew <andrew@FreeBSD.org> Remove the unused arm64_cpu driver.

This was previously used for CPU initilisation, however this hasn't been
the case in a long time.

Sponsored by: DARPA, AFRL
rm64/mp_machdep.c
fcc5d25798bbb0319879b920bad51168675595c1 22-Oct-2018 brooks <brooks@FreeBSD.org> Consolidate identical ELF auxargs type defintions.

All platforms except powerpc use the same values and powerpc shares a
majority of them.

Go ahead and declare AT_NOTELF, AT_UID, and AT_EUID in favor of the
unused AT_DCACHEBSIZE, AT_ICACHEBSIZE, and AT_UCACHEBSIZE for powerpc.

Reviewed by: jhb, imp
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D17397
nclude/elf.h
0dd92926d8fe3e5641299efa035da5a722a1f3cc 22-Oct-2018 markj <markj@FreeBSD.org> Make it possible to disable NUMA support with a tunable.

This provides a chicken switch for anyone negatively impacted by
enabling NUMA in the amd64 GENERIC kernel configuration. With
NUMA disabled at boot-time, information about the NUMA topology
is not exposed to the rest of the kernel, and all of physical
memory is viewed as coming from a single domain.

This method still has some performance overhead relative to disabling
NUMA support at compile time.

PR: 231460
Reviewed by: alc, gallatin, kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17439
rm64/mp_machdep.c
f0a3a25e7334bd7d78f5276b845a05325df2b157 22-Oct-2018 andrew <andrew@FreeBSD.org> Stop advertising ARMv8.3 Pointer Authentication

This needs firmware and kernel support before userspace can use it. Until
then don't advertise it's available.

MFC after: 3 days
rm64/identcpu.c
c84f66e25a4dc3dc4600f045bf40baabe25f9ce8 22-Oct-2018 andrew <andrew@FreeBSD.org> Fix the ID_AA64ISAR0_EL1 dot product field shift.

It's 44 in the documentation, use this correct value.

MFC after: 3 days
nclude/armreg.h
3711ea9758ce804fc01ebea6d983c8706ecf0172 22-Oct-2018 andrew <andrew@FreeBSD.org> Correctly set the DAIF bits in new threads

We should only unmask interrupts when creating a new thread and leave the
other exceptions in teh same state as before creating the thread.

Reported by: jhibbits
Reviewed by: jhibbits
MFC after: 1 month
Sponsored by: https://reviews.freebsd.org/D17497
rm64/vm_machdep.c
nclude/armreg.h
9d144c64513cca420d1edece7a497e0d2688b774 07-Oct-2018 tuexen <tuexen@FreeBSD.org> Address the warning regarding duplicate option 'GEOM_PART_GPT' when
configuring kernels for i386, amd64, and arm64.
The 'GEOM_PART_GPT' option was added to the DEFAULTS configuration
in r337967.

Approved by: re (kib@)
Reviewed by: ler@
Differential Revision: https://reviews.freebsd.org/D17458
Sponsored by: Netflix, Inc.
onf/GENERIC
2a6fe8c6e833dd6b1f69ef56a36316d692573b07 01-Oct-2018 andrew <andrew@FreeBSD.org> Add kernel ifunc support on arm64.

Tested with ifunc resolvers in the kernel and module with calls from
kernel to kernel, module to kernel, and module to module.

Reviewed by: kib (previous version)
Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D17370
rm64/elf_machdep.c
rm64/machdep.c
nclude/ifunc.h
1ba09d88227e05f588caa6f0cc9edd196cc6b4ff 01-Oct-2018 manu <manu@FreeBSD.org> arm64: Raise again L3 table for early devmap

The initial raise in r336519 wasn't enough for using big resolution
(1920 x 1200 for example). Raise it again.

Reported by: bob prohaska <fbsd@www.zefox.net>
Tested by: bob prohaska <fbsd@www.zefox.net>
Approved by: re (gjb@)
nclude/pte.h
dfd4ab601b9384420e021d3565490ae4dbd7067b 28-Sep-2018 andrew <andrew@FreeBSD.org> Export ID_AA64ISAR{0,1}_EL1 to userland.

As with r338962 also export the instruction set attribute register. This
will allow userland to identify optional instructions the hardware
supports, for example in a future ifunc handler to decide which
implementation of a function to return.

Approved by: re (kib)
rm64/identcpu.c
4ddccdfc0ea59b9021a6b384d22cb83126461d77 27-Sep-2018 andrew <andrew@FreeBSD.org> Export ID_AA64PFR0_EL1 to userland

Create a user view of the ID_AA64PFR0_EL1 register with values common
across all CPUs.

Approved by: re (kib)
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D17301
rm64/identcpu.c
9fab1c4f949c5d9a46b26047d510c26f6ffc50d0 27-Sep-2018 andrew <andrew@FreeBSD.org> Move the undefined instruction handler to identcpu.c so we have access
to the registers from boot.

Approved by: re (kib)
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D17301
rm64/identcpu.c
rm64/undefined.c
nclude/undefined.h
80d99cf13b3b8611641577f180497779fc3d2344 19-Sep-2018 jhb <jhb@FreeBSD.org> Clear all of the VFP state in fill_fpregs().

Zero the entire FP register set structure returned for ptrace() if a
thread hasn't used FP registers rather than leaking garbage in the
fp_sr and fp_cr fields.

Reviewed by: emaste, andrew
Approved by: re (rgrimes)
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D17140
rm64/machdep.c
7f8b422b281cc771b2b1f92182900d14b02137b0 14-Sep-2018 bz <bz@FreeBSD.org> Set ident for GENERIC-MMCCAM to not announce itself as
GENERIC anymore.

Reviewed by: andrew
Approved by: re (gjb)
onf/GENERIC-MMCCAM
3dcd4f12acccf3dff4ab401658e2aecdf481f71d 08-Sep-2018 markj <markj@FreeBSD.org> Exclude the EFI framebuffer from phys_avail[] on arm64.

On the ThunderX the region occupied by the framebuffer is included in
the EFI map, so explicitly add it to the set of regions that aren't
managed by the physical memory allocator.

PR: 231064
Reviewed by: andrew
Approved by: re (gjb)
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17073
rm64/machdep.c
dcd2b35789e5209baadbb91bb5a63ea4351645bc 02-Sep-2018 kib <kib@FreeBSD.org> Catch exceptions during EFI RT calls on amd64.

This appeared to be required to have EFI RT support and EFI RTC
enabled by default, because there are too many reports of faulting
calls on many different machines. The knob is added to leave the
exceptions unhandled to allow to debug the actual bugs.

Reviewed by: kevans
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Approved by: re (rgrimes)
Differential revision: https://reviews.freebsd.org/D16972
rm64/efirt_machdep.c
nclude/efi.h
acc0761238af944cefd42b90e598ab22c28f858d 29-Aug-2018 kib <kib@FreeBSD.org> Remove {max/min}_offset() macros, use vm_map_{max/min}() inlines.

Exposing max_offset and min_offset defines in public headers is
causing clashes with variable names, for example when building QEMU.

Based on the submission by: royger
Reviewed by: alc, markj (previous version)
Sponsored by: The FreeBSD Foundation (kib)
MFC after: 1 week
Approved by: re (marius)
Differential revision: https://reviews.freebsd.org/D16881
rm64/pmap.c
3799d78beb4cf81baac99c1256126e10696fc4e3 25-Aug-2018 alc <alc@FreeBSD.org> Eliminate the arena parameter to kmem_free(). Implicitly this corrects an
error in the function hypercall_memfree(), where the wrong arena was being
passed to kmem_free().

Introduce a per-page flag, VPO_KMEM_EXEC, to mark physical pages that are
mapped in kmem with execute permissions. Use this flag to determine which
arena the kmem virtual addresses are returned to.

Eliminate UMA_SLAB_KRWX. The introduction of VPO_KMEM_EXEC makes it
redundant.

Update the nearby comment for UMA_SLAB_KERNEL.

Reviewed by: kib, markj
Discussed with: jeff
Approved by: re (marius)
Differential Revision: https://reviews.freebsd.org/D16845
rm64/busdma_bounce.c
rm64/mp_machdep.c
cf3e18f4f67cfda3665563628e099d92c3d0a334 23-Aug-2018 manu <manu@FreeBSD.org> a10_timer: Update the driver so we can use it on other SoC

a10_timer is currently use in UP allwinner SoC (A10 and A13).
Those don't have the generic arm timer.
The arm generic timecounter is broken in the A64 SoC, some attempts have
been made to fix the glitch but users still reported some minor ones.
Since the A64 (and all Allwinner SoC) still have this timer controller, rework
the driver so we can use it in any SoC.
Since it doesn't have the 64 bits counter on all SoC, use one of the
generic 32 bits counter as the timecounter source.

PR: 229644
onf/GENERIC
4bc02baf1fcb0041db95621185361d1ce4124cc9 23-Aug-2018 manu <manu@FreeBSD.org> arm64: GENERIC: Compile allwinner dtbs
onf/GENERIC
1d715a5168dcd2745d7228c383baeb8693cf0a37 22-Aug-2018 markj <markj@FreeBSD.org> Prepare the kernel linker to handle PC-relative ifunc relocations.

The boot-time ifunc resolver assumes that it only needs to apply
IRELATIVE relocations to PLT entries. With an upcoming optimization,
this assumption no longer holds, so add the support required to handle
PC-relative relocations targeting GNU_IFUNC symbols.
- Provide a custom symbol lookup routine that can be used in early boot.
The default lookup routine uses kobj, which is not functional at that
point.
- Apply all existing relocations during boot rather than filtering
IRELATIVE relocations.
- Ensure that we continue to apply ifunc relocations in a second pass
when loading a kernel module.

Reviewed by: kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D16749
rm64/elf_machdep.c
e843abc9ba1e04c17d28ea6400c0d3cf49a4435e 21-Aug-2018 gonzo <gonzo@FreeBSD.org> Add muge(4) to the arm64 GENERIC kernel

muge(4) is the USB ethernet adapter that is used in RPi 3B+. Shipping it
in GENERIC kernel allows using NFS root out of the box instead of either
building custom kernel or modifying loader.conf for early loading of if_muge.ko

No objections: emaste
onf/GENERIC
4ce21fcbeaf66e2031419afa4737448e99284ce0 21-Aug-2018 alc <alc@FreeBSD.org> Eliminate kmem_malloc()'s unused arena parameter. (The arena parameter
became unused in FreeBSD 12.x as a side-effect of the NUMA-related
changes.)

Reviewed by: kib, markj
Discussed with: jeff, re@
Differential Revision: https://reviews.freebsd.org/D16825
rm64/mp_machdep.c
rm64/pmap.c
71b5b012c470799caab3128c9b365585d08aa7e2 20-Aug-2018 alc <alc@FreeBSD.org> Eliminate kmem_alloc_contig()'s unused arena parameter.

Reviewed by: hselasky, kib, markj
Discussed with: jeff
Differential Revision: https://reviews.freebsd.org/D16799
rm64/busdma_bounce.c
d7e089b3ecdfd2dd7804a5764e7834deb63f6a00 19-Aug-2018 jmg <jmg@FreeBSD.org> use sbuf so that lines are printed together... As aarch64 often
has SMP enabled, lines can get intermixed with other console output
making these lines hard to read...

Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D16689
rm64/identcpu.c
a6f0cd92bb652977c71e2a49b551de78b2a3bd93 19-Aug-2018 manu <manu@FreeBSD.org> arm64: allwinner: Add aw_syscon driver to GENERIC

Recent DTS use the syscon for the emac controller.
We support this but since U-Boot is still using old DTS it was never
needed for us to add this support, but this is a problem when using upstream
recent DTS and will be when U-Boot will catch up.

While here add a new compatible to the aw_syscon driver as Linux changed it ...
onf/GENERIC
576f60d45098b8de343fcd2c0268a5826f6375a6 18-Aug-2018 alc <alc@FreeBSD.org> Oops. r338030 didn't eliminate the unused arena argument from all of
kmem_alloc_attr()'s callers. Correct that mistake.
rm64/busdma_bounce.c
5f8e558a6b8ac9a068352383f451692429462245 18-Aug-2018 jhb <jhb@FreeBSD.org> Make 'device crypto' lines more consistent.

- In configurations with a pseudo devices section, move 'device crypto'
into that section.
- Use a consistent comment. Note that other things common in kernel
configs such as GELI also require 'device crypto', not just IPSEC.

Reviewed by: rgrimes, cem, imp
Differential Revision: https://reviews.freebsd.org/D16775
onf/GENERIC
02e6d7f1d9668b221f4832d2c2fd4270a7bec21b 17-Aug-2018 imp <imp@FreeBSD.org> GPT is standard in x86 and arm64 land. Add it to DEFAULTS with the
others.

Differential Revision: https://reviews.freebsd.org/D16740
onf/DEFAULTS
d19b648edf665ed739ee83cb0d44ab93faafd443 15-Aug-2018 andrew <andrew@FreeBSD.org> Set the Execute Never flags in EFI device memory as required by the ARMv8
spec.

Sponsored by: DARPA, AFRL
rm64/efirt_machdep.c
21f5deb5e63cc1a3479834580d0eed32734f5323 14-Aug-2018 loos <loos@FreeBSD.org> Add support to the Marvell Xenon SDHCI controller.

Tested on Espresso.bin (37x0) and Macchiato.bin (8k) with SD cards and
eMMCs.

Obtained from: pfSense
Sponsored by: Rubicon Communications, LLC (Netgate)
onf/GENERIC
bda515985d2a2420bd69395f2cda00419e4df23c 14-Aug-2018 loos <loos@FreeBSD.org> Use the correct PTE when changing the attribute of multiple pages.

Submitted by: andrew (long time ago)
Sponsored by: Rubicon Communications, LLC (Netgate)
rm64/pmap.c
0b54196608763ba3fcdf80ffc7fa0c4d089702e9 14-Aug-2018 andrew <andrew@FreeBSD.org> Support reading from the arm64 ID registers from userspace.

Trap reads to the arm64 ID registers and write a safe value into them. This
will allow us to put more useful values in these later and have userland
check them to find what features the hardware supports.

These are currently safe defaults, but will later be populated with better
values from the hardware.

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D16533
rm64/undefined.c
7997bd2bb833c6c50136ac052689cde968280c4d 04-Aug-2018 kevans <kevans@FreeBSD.org> efirt: Don't enter EFI context early, convert addrs to KVA instead

efi_enter here was needed because efi_runtime dereference causes a fault
outside of EFI context, due to runtime table living in runtime service
space. This may cause problems early in boot, though, so instead access it
by converting paddr to KVA for access.

While here, remove the other direct PHYS_TO_DMAP calls and the explicit DMAP
requirement from efidev.

Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D16591
rm64/efirt_machdep.c
3792b2b1048e5f21008b035b4cad08325bb90b09 01-Aug-2018 kib <kib@FreeBSD.org> Add pmap_is_valid_memattr(9).

Discussed with: alc
Sponsored by: The FreeBSD Foundation, Mellanox Technologies
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D15583
rm64/pmap.c
730b0d841561531498351df659fdd623d1649918 31-Jul-2018 manu <manu@FreeBSD.org> nvmem: Add nvmem interface and helpers

The nvmem interface helps provider of nvmem data to expose themselves to consumer.
NVMEM is generally present on some embedded board in a form of eeprom or fuses.
The nvmem api are helpers for consumer to read/write the cell data from a provider.

Differential Revision: https://reviews.freebsd.org/D16419
onf/GENERIC
3f006c3cbffd64218885e8167846c1bcc6316fb0 31-Jul-2018 andrew <andrew@FreeBSD.org> Use int for the pcpu_ssbd argument. This is included from userland and may
not include the needed headers to get the bool definition.

Reported by: manu
Pointy hat to: andrew
Sponsored by: DARPA, AFRL
rm64/cpu_errata.c
nclude/pcpu.h
1b0b80446a0da8dc94667ce6cd713d54691d9f24 31-Jul-2018 andrew <andrew@FreeBSD.org> Implement the SSBD (CVE-2018-3639) workaround on arm64

This calls into the Arm Trusted Firmware to enable and disable the
workaround for the Speculative Store Bypass Disable (SSBD) issue, also
known as Spectre Variant 4.

As this may have a large performance overhead, and how exploitable SSBD is
is unknown we follow the Linux lead of allowing the administrator to select
between always on, always off, or only enabled in the kernel, with the
latter being the default.

PR: 228955
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D15819
rm64/cpu_errata.c
rm64/exception.S
rm64/genassym.c
nclude/pcpu.h
f9a484901c4f26d2def4274a997c98777d73a435 30-Jul-2018 andrew <andrew@FreeBSD.org> Enable VIMAGE on arm64 again. A workaround for modules with static VNET
variables has been committed so these should work now.

PR: 223670
Sponsored by: DARPA, AFRL
onf/GENERIC
0998c88bf5c1b4d3e262befe549210db3bf2875e 25-Jul-2018 markj <markj@FreeBSD.org> Simplify the arm64 implementation of pmap_mincore().

No functional change intended.

Reviewed by: alc
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D16427
rm64/pmap.c
c962905e0899849a139a3773e4d8a6d200d293c1 21-Jul-2018 alc <alc@FreeBSD.org> Eliminate a comment that doesn't apply to this pmap implementation.

Coalesce the variable definitions for PV entry management.

MFC after: 3 weeks
rm64/pmap.c
d1498f4e3a1a5e8627cf112af88fb2051c70e1d4 20-Jul-2018 markj <markj@FreeBSD.org> Add support for pmap_enter(psind = 1) to the arm64 pmap.

See the commit log messages for r321378 and r336288 for descriptions of
this functionality.

Reviewed by: alc
Differential Revision: https://reviews.freebsd.org/D16303
rm64/pmap.c
nclude/param.h
nclude/pmap.h
a8c7d789554db02da8612ec1f2e0778f6a1a8e46 20-Jul-2018 markj <markj@FreeBSD.org> Initialize the L3 page's wire count correctly after a L2 entry demotion.

Reviewed by: alc
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D16303
rm64/pmap.c
4ff235e6358595d095b4b9c2707ed0427efc78a5 19-Jul-2018 manu <manu@FreeBSD.org> arm64: Add vt_efifb to GENERIC

We can now have efifb being setup correctly.
Enjoy video output on some boards when you couldn't before.

Tested-On: Pine64
Tested-On: Pine64-LTS
Tested-On: Pinebook
onf/GENERIC
e3138eb7f6d69791687e2064603a4dc74d406949 19-Jul-2018 manu <manu@FreeBSD.org> Raise the size of L3 table for early devmap on arm64

Some driver (like efifb) needs to map more than the current L2_SIZE
Raise the size so we can map the framebuffer setup by the bootloader.

Reviewed by: cognet
rm64/pmap.c
nclude/pte.h
c2547796ac75efdc7b6b19c9103b6b229fdd479d 18-Jul-2018 markj <markj@FreeBSD.org> Port r324665 and r325285 to arm64.

These changes ensure that reclaim_pv_chunk() can be safely be
executed concurrently by multiple threads.

Reviewed by: alc
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D16304
rm64/pmap.c
nclude/pmap.h
68eda9fa0e9c708b37c569e6b13a5726a0e3bed1 17-Jul-2018 imp <imp@FreeBSD.org> Remove VM_FREELIST_ISADMA. It's not needed on these architectures.

Differential Review: https://reviews.freebsd.org/D16290
nclude/vmparam.h
3d41e80ced7f49ed9d1a2f3e0decd7dd4ed7bac6 13-Jul-2018 mw <mw@FreeBSD.org> Enable UART support for Xilinx Ultrascale+ SoCs

Xilinx Ultrascale+ are based on Cortex-A53 and use existing
UART driver (uart_dev_cdnc). Enable it in arm64 GENERIC config.

Submitted by: Michal Stanek <mst@semihalf.com>
Obtained from: Semihalf
onf/GENERIC
46d1d55afb74e2c0c3c00dd65d32a1efdb39bf69 09-Jul-2018 wma <wma@FreeBSD.org> ARM64: Add ThunderX2 CPU revision macro. Add ThunderX2 name in identcpu.c

Submitted by: Patryk Duda <pdk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Cavium
rm64/identcpu.c
nclude/cpu.h
9baa4fcab2c2f64128912da8f6e3413c754090af 08-Jul-2018 markj <markj@FreeBSD.org> Reuse the PV entry when updating a mapping in pmap_enter().

This addresses a problem described in r335784, where memory
pressure forces reclamation of a PV chunk and in rare cases leads to a
use-after-free of a page table page.

Reviewed by: alc, kib
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D16181
rm64/pmap.c
465d5a960efb8580e1607a345d834375717043da 06-Jul-2018 jhb <jhb@FreeBSD.org> Export a breakpoint() function to userland for arm and arm64.

Enable ptrace() tests using breakpoint() on these architectures.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D15191
nclude/cpufunc.h
d0467c6843444ca4c4e6209c7a22f125cab686ce 06-Jul-2018 emaste <emaste@FreeBSD.org> Regen arm64 linux sysent after r336043
inux/linux_proto.h
inux/linux_systrace_args.c
517445735e09c31afa0211cb3c51ba07d4d15207 06-Jul-2018 emaste <emaste@FreeBSD.org> Fix arm64 linuxulator clone() argument order

Linux/arm64 is CLONE_BACKWARDS - i.e., "Architecture has tls passed as
the 4th argument of clone(2), not the 5th one."

The linux clone() syscall has four different permutations of argument
order, depending on architecture - see the #ifdef CONFIG_CLONE_BACKWARDS
maze in Linux's kernel/fork.c.

Sponsored by: Turing Robotic Industries
inux/syscalls.master
ff20311f27958b751ed21c94a01ed31c8d787f0b 06-Jul-2018 mmacy <mmacy@FreeBSD.org> Back pcpu zone with domain correct pages

- Change pcpu zone consumers to use a stride size of PAGE_SIZE.
(defined as UMA_PCPU_ALLOC_SIZE to make future identification easier)

- Allocate page from the correct domain for a given cpu.

- Don't initialize pc_domain to non-zero value if NUMA is not defined
There are some misconceptions surrounding this field. It is the
_VM_ NUMA domain and should only ever correspond to valid domain
values as understood by the VM.

The former slab size of sizeof(struct pcpu) was somewhat arbitrary.
The new value is PAGE_SIZE because that's the smallest granularity
which the VM can allocate a slab for a given domain. If you have
fewer than PAGE_SIZE/8 counters on your system there will be some
memory wasted, but this is obviously something where you want the
cache line to be coming from the correct domain.

Reviewed by: jeff
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D15933
nclude/counter.h
2c07601d9517d2a43af0a86d69177f208cea9aa5 28-Jun-2018 emaste <emaste@FreeBSD.org> Add stub arm64 linuxulator VDSO ldscript

This needs to be revisited with the VDSO implementation, but is
sufficient to allow the linux64 module to build on arm64 for testing
and development.

Sponsored by: Turing Robotic Industries
inux/linux_vdso.lds.s
2f3f042a2671546048792c95b662b9e5a32cc51e 25-Jun-2018 andrew <andrew@FreeBSD.org> Make cpu_set_syscall_retval common between the existing FreeBSD ABI and
the Linuxulator. We need to translate error values onto Linux errno values
and return them to userspace when a syscall fails. We also need to preserve
x1 as all registers are preserved other than the return value.

Reviewed by: emaste
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D16008
rm64/vm_machdep.c
inux/linux_sysvec.c
f5a48c7673e1d13dd21654932d5ddbfd98ba196d 25-Jun-2018 emaste <emaste@FreeBSD.org> Initial arm64 linuxulator linux_sysvec

This is sufficient to run Linux arm64 'hello world' and other simple
binaries.

Reviewed by: andrew
Sponsored by: Turing Robotic Industries
Differential Revision: https://reviews.freebsd.org/D15834
inux/linux_sysvec.c
c9e59b691404ba51907f478f2f571b37ccf9dce0 23-Jun-2018 markj <markj@FreeBSD.org> Re-count available PV entries after reclaiming a PV chunk.

The call to reclaim_pv_chunk() in reserve_pv_entries() may free a
PV chunk with free entries belonging to the current pmap. In this
case we must account for the free entries that were reclaimed, or
reserve_pv_entries() may return without having reserved the requested
number of entries.

Reviewed by: alc, kib
Tested by: pho (previous version)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D15911
rm64/pmap.c
b5d43b277c08e7dacf5f3bf2c9597c379be0d2e9 20-Jun-2018 manu <manu@FreeBSD.org> Add pmap_mapdev_attr for arm64

This is needed for efifb.
arm and ricv pmap (the two arch with arm64 that uses subr_devmap) have very
different implementation so for now only add this for arm64.

Tested with efifb on Pine64 with a few other patches.

Reviewed by: cognet
Differential Revision: https://reviews.freebsd.org/D15294
rm64/pmap.c
nclude/pmap.h
f1972c21ab1a1b2ef36d08fc77fa820786db3fcf 20-Jun-2018 manu <manu@FreeBSD.org> if_rk_dwc: Disable setting delays for now

The values for tx/rx delays differs accross the different DTS.
Mainline Linux set it to 0x24/0x18
Mostly-Vendor u-boot (the one maintained and developped) to 0x18/0x18
Mostly-Vendor linux (the one maintained and developped) to 0x26/0x11

By experience only 0x18/0x18 works so until the issue is resolved rely on
the bootloader settings.
ockchip/if_dwc_rk.c
81b8419c39b0287cc9ff7561d286f6c99b96763b 20-Jun-2018 manu <manu@FreeBSD.org> rk_gpio: Read the correct register for gpio read

Reported by: jmcneill
ockchip/rk_gpio.c
31dbcedad5db07db61f4f3c16381fde19128cfb3 20-Jun-2018 manu <manu@FreeBSD.org> if_rk_dwc: Fix delays handling

The property are named {t,r}x_delay and not {t,r}-delay.
The upper bits of the register are a mask of which bits is allowed
to be written, set it otherwise we write nothing.
OF_getencprop returns <0 = for an error.

Pointy Hat: myself
Reported by: jmcneill (delay and mask bits)
ockchip/if_dwc_rk.c
36cf4d2a9c363b2799289221cf0fe7bd030bc107 19-Jun-2018 andrew <andrew@FreeBSD.org> Move common GIC interrupt numbers to the common header. These are the same
across the GICv2 and GICv3 drivers so we only need a single copy of them.

Sponsored by: Turing Robotic Industries
rm64/gic_v3_reg.h
54a64dc1608c903e7a23538587dc565f20b44ae0 18-Jun-2018 emaste <emaste@FreeBSD.org> Introduce arm64 linuxulator stubs

This provides stub implementations of arm64 Linux vdso and machdep,
ptrace, and futex sufficient for executing an arm64 Linux 'hello world'
binary.

Reviewed by: andrew
Sponsored by: Turing Robotic Industries
Differential Revision: https://reviews.freebsd.org/D15832
inux/linux_locore.s
inux/linux_machdep.c
inux/linux_ptrace.c
inux/linux_support.s
9c4fe6a8d5f49463f812550872dde619bf669d10 18-Jun-2018 emaste <emaste@FreeBSD.org> Regen arm64 linuxulator sysent files after r335323
inux/linux_syscall.h
inux/linux_syscalls.c
inux/linux_sysent.c
inux/linux_systrace_args.c
bda5b7d3142ffa114ebcf75ceec7c84fc628f3d0 18-Jun-2018 emaste <emaste@FreeBSD.org> arm64 linuxulator: add dup syscall entry

Missed in r333027

Sponsored by: Turing Robotic Industries Inc.
inux/syscalls.master
815c9f6cea663b81a3db439f0cffed69461ce895 15-Jun-2018 emaste <emaste@FreeBSD.org> arm64 linuxulator: add linux_dummy.c based on amd64

A later change should deduplicate the multiple copies of this file.

Sponsored by: Turing Robotic Industries
inux/linux_dummy.c
5cdfe055c6057dbc50b2118ac049fd714f65fbed 15-Jun-2018 emaste <emaste@FreeBSD.org> arm64: add arm64 linux.h based on i386 linuxulator and Linux headers

Sponsored by: Turing Robotic Industries
inux/linux.h
bdbf491b8eb8a76c53911b84166ce3f2575a8b40 14-Jun-2018 manu <manu@FreeBSD.org> rk_i2c: Add driver for the I2C controller present in RockChip SoC

This controller have a special mode for RX to help with smbus-like transfer
when the controller will automatically send the slave address, register address
and read the data. Use it when possible.
The same mode for TX is describe is the datasheet but is broken and have been
since ~10 years of presence of this controller in RockChip SoCs.

Attach this driver early at we need it to communicate with the PMIC early in the
boot.
Do not hook it to the kernel build for now.
ockchip/rk_i2c.c
dfb049178018b42c71b88bc9a4a34bf41d879784 14-Jun-2018 manu <manu@FreeBSD.org> rk3328: Add support for the i2c clocks
ockchip/clk/rk3328_cru.c
653ad25d247d17bd78a06b01a69d9122e807676f 14-Jun-2018 manu <manu@FreeBSD.org> if_dwc_rk: Add DesignWare driver for RockChip SoCs.

Add driver for the designware ethernet controller found in some RockChip SoCs.
The driver still rely on a lot of things setup by the bootloader like clocks
and phy mode.
But since netbooting is the only/easiest way to boot rockchip board at the
moment add the driver so other people can test/dev on thoses boards.
onf/GENERIC
ockchip/if_dwc_rk.c
75f4cd0c543171fb46b797898be06c2af080cf26 14-Jun-2018 manu <manu@FreeBSD.org> rk_armclk: Add the write mask to the register mux value

This was omitted in r334112 and r334996 which cause the PLL to not correctly
reparent, leaving the armclk to be derived from the APLL instead of the NPLL.
The arm core clock is now correctly set to 600Mhz via the assigned-clock present
in the DTB.
ockchip/clk/rk_clk_armclk.c
2cdc5636762a18da6b89ce42a44b62359bf42d47 14-Jun-2018 manu <manu@FreeBSD.org> rk_pll: Add support for mode

RockChip PLL have two modes controlled by a register, a "slow mode" (the
default one) where the frequency is derived from the 24Mhz oscillator on the
board, and a "normal" one when the pll take it's input from the real PLL output.

Default the mode to normal for all the PLLs.
ockchip/clk/rk3328_cru.c
ockchip/clk/rk_clk_pll.c
ockchip/clk/rk_clk_pll.h
bb5d08a4d089ce90085a86d25c531e58eeb456ab 14-Jun-2018 manu <manu@FreeBSD.org> rk_pinctrl: Only add gpio subnode

This is the only node we are interested in so do not waste time to test
creating device that will be either unused or fail as most of the nodes
don't have a compatible string.
ockchip/rk_pinctrl.c
299ca601ae60af17dbe48cc3e1c59983f395a97c 13-Jun-2018 andrew <andrew@FreeBSD.org> Add ThunderX2 to the list of CPUs we need to apply the branch predictor
hardening to.

Sponsored by: DARPA, AFRL
rm64/cpu_errata.c
5250c7456eea21a23fef50d9f9425e738b4dbed8 13-Jun-2018 andrew <andrew@FreeBSD.org> Switch to the SMCCC function for branch predictor hardening. The previous
method may not have worked as the firmware checks for the ARCH_WORKAROUND_1
function ID.

Sponsored by: DARPA, AFRL
rm64/cpu_errata.c
9e420665b6410c7acf0b510d5cecad5aa39fbb8b 13-Jun-2018 andrew <andrew@FreeBSD.org> Rename the ThunderX CPU identification macros to include the X. This is the
name people know the product by, and is consistent with the later SoC ID
macros.

Sponsored by: DARPA, AFRL
rm64/gic_v3.c
rm64/identcpu.c
avium/thunder_pcie_common.c
avium/thunder_pcie_fdt.c
nclude/cpu.h
d7f77d363991fa923e12d61cb42cf2e339b0354c 13-Jun-2018 andrew <andrew@FreeBSD.org> Add more Cavium CPU part numbers.

While here split the lists by vendor.

Sponsored by: DARPA, AFRL
nclude/cpu.h
5cacb41edfbbd44925defdb59d9cdefd0f4418f1 12-Jun-2018 db <db@FreeBSD.org> Add a driver for the BCM2835 Mini-UART as seen on the RPi3

Reviewed by: andrew
Approved by: andrew
Differential Revision: https://reviews.freebsd.org/D15684
onf/GENERIC
ce93b4cba0bdd719f9a7df9a9ef5ecfbbac00eee 12-Jun-2018 manu <manu@FreeBSD.org> arm64: rockchip: Correctly set armclk

Parent needs to be the same frequency as the armclk, not twice the freq.
The real divider is incremented by one so write it with - 1
The rate can be at index 0

Pointy Hat To: myself
ockchip/clk/rk_clk_armclk.c
128c3e8914637c967b3a241fa3377ea5d23fe60e 10-Jun-2018 andrew <andrew@FreeBSD.org> Remove the psci option from arm64. It is now a standard option as it is
required to boot correctly.

Sponsored by: DARPA, AFRL
rm64/cpu_errata.c
rm64/vm_machdep.c
onf/GENERIC
957e7993d4644c5151af1231077dfa954902dc62 10-Jun-2018 andrew <andrew@FreeBSD.org> Clean up handling of unexpected exceptions. Previously we would issue a
breakpoint instruction, however this would lose information that may be
useful for debugging.

These are now handled in a similar way to other exceptions, however it
won't exit out of the exception handler until it is known if we can
handle these exceptions in a useful way.

Sponsored by: DARPA, AFRL
rm64/exception.S
rm64/trap.c
11ea0b8f38c408a0d31cca9e8df09ceae018ba1b 08-Jun-2018 kib <kib@FreeBSD.org> Restore release semantic for the old thread unlock on arm64.

With the introduction of pmap_switch(), the DSB instruction on the
address map switch is not necessary executed, which is fixed by
changing the unlock store to release. Also remove comment which
documented pre-pmap_switch() code.

Reviewed by: andrew
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
rm64/swtch.S
8635cba20a74992256ee7b588f62c43160af0804 05-Jun-2018 kevlo <kevlo@FreeBSD.org> Since we don't enable BUF_TRACKING and FULL_BUF_TRACKING buffer debugging
options in GENERIC kernels on arm and arm64, there's no need to disable
them.

Sponsored by: MSI/FUNTORO
onf/GENERIC-NODEBUG
f358ddbed76f5ee85b0d247cd04609964cbdbbe5 31-May-2018 andrew <andrew@FreeBSD.org> Move the code to print the EFI memory table to a new function and call it
in teh bootverbose path after cninit().

This allows users to see these tables when booting with boot -v.

Sponsored by: DARPA, AFRL
rm64/machdep.c
0d79c92b374076cacf0a44513ab5a8089b134ef4 31-May-2018 andrew <andrew@FreeBSD.org> Fix the early spelling of bootverbose.

Sponsored by: DARPA, AFRL
rm64/machdep.c
a5b9b01ec3b6c580f46b5d422d8cb9e8f2bfdd80 31-May-2018 andrew <andrew@FreeBSD.org> Fix the memory attribute for EFI_MD_ATTR_UC on arm64.

This is defined as Device-nGnRnE in the UEFI spec (UEFI 2.4, section
2.3.6.1). This is the VM_MEMATTR_DEVICE type in FreeBSD.

Reported by: tuexen
Sponsored by: DARPA, AFRL
rm64/efirt_machdep.c
6e1d336d8eaeabd260e393c32d1e28f50f60a9ec 30-May-2018 andrew <andrew@FreeBSD.org> Remove max_pa, it's unused.

Sponsored by: DARPA, AFRL
rm64/pmap.c
c118994f1bbb0a0b5672471377835a61a3abd2c5 30-May-2018 andrew <andrew@FreeBSD.org> Further limit when we call pmap_fault.

We should only call pmap_fault in the kernel when accessing a userspace
address. As this should always happen through specific functions that set
a fault handler we can use this to limit calls to pmap_fault to when this
is set.

This should help with NULL pointer dereferences when we are unable to sleep
so we fall into the correct case.

Sponsored by: DARPA, AFRL
rm64/trap.c
b9da38390ce53cc2ab15e6d6bc39272b06b790ef 30-May-2018 andrew <andrew@FreeBSD.org> Push down the locking in pmap_fault to just be around the calls to
arm64_address_translate_*. There is no need to lock around the switch
statement as we only care about a few cases.

Sponsored by: DARPA, AFRL
rm64/pmap.c
a389f73841c957c8a673e99104bd5cd9e164da46 29-May-2018 andrew <andrew@FreeBSD.org> On ThunderX2 we need to be careful to only map the memory the firmware
lists in the EFI memory map. As such we need to reduce the mappings to
restrict them to not be the full 1G block. For now reduce this to a 2M
block, however this may be further restricted to be 4k page aligned as
other SoCs may require.

This allows ThunderX2 to boot reliably to userspace without performing
any speculative memory accesses to invalid physical memory.

This is a recommit of r334035 now that we can access the EFI Runtime data
through the DMAP region.

Tested by: tuexen
Sponsored by: DARPA, AFRL
rm64/pmap.c
598fd1c558f0d49752ba82a39c0e528b1baa3745 28-May-2018 manu <manu@FreeBSD.org> arm64: fix atomic_fcmpset_16

newval needs to be uint16_t

Reported by: andrew
nclude/atomic.h
99ece95da0de7d87fed17311a48e43d3340a790f 28-May-2018 manu <manu@FreeBSD.org> arm64: Add atomic_fcmpset_8 and atomic_fcmpset_16

Reviewed by: cognet
nclude/atomic.h
8a9c29613a69a895617139f481c0f1751165e2f6 28-May-2018 andrew <andrew@FreeBSD.org> Create a new function to walk the EFI memory table & run a callback for
each entry. We can then use this to ensure the RunTime data is mapped in
the DMAP, but not in phys_avail.

Sponsored by: DARPA, AFRL
rm64/machdep.c
ab7acde2c289ba66e704ac9f8a6008055dec98a3 24-May-2018 andrew <andrew@FreeBSD.org> Remove physmap from the arm64 machdep.h. This was missed in r334162.
nclude/machdep.h
efd8540f2e037e04fa7867b3ce4b7f3276bfb2f5 24-May-2018 andrew <andrew@FreeBSD.org> Allow us to read the physmap data into our own array and use this to build
the DMAP region on arm64.

We already have the needed information to build these tables, we just need
to extract it. This significantly simplifies the code.

Obtained from: ABT Systems Ltd
Sponsored by: Turing Robotic Industries
rm64/machdep.c
rm64/pmap.c
a9a76a3a09939e0497e9c7cae7f6693962439e13 24-May-2018 andrew <andrew@FreeBSD.org> Print the physmem tables under a verbose boot.

Obtained from: ABT Systems Ltd
Sponsored by: Turing Robotic Industries
rm64/machdep.c
554647942ab3b792ab993e0e993d03e9d03bdb54 24-May-2018 andrew <andrew@FreeBSD.org> Exclude memory from the /reserved-memory mappings with the no-map property
set. This memory must not be mapped by the operating system other than
under control of the device driver.

Obtained from: ABT Systems Ltd
Sponsored by: Turing Robotic Industries
rm64/machdep.c
87cbf3714660e76e9c280a8be632a334cffdc204 23-May-2018 manu <manu@FreeBSD.org> arm64: rockchip: Add proper armclock support

The core clock (armclk) on RockChip SoC is special.
It can derive it's clock from many PLLs but RockChip recommand to do it
from "apll" on old SoC and "npll" on new SoC. The reason for choosing npll
is that it's have less jitter and is more close to the arm core on the SoC.
r333314 added the core clock as a composite clock but due to it's specials
property we need to deal with it differently.
A new rk_clk_armclk type is added for this and it supports only the "npll"
as we don't run on old RockChip SoC that only have the "apll".
It will always reparent to "npll" and set the frequency according to a rate
table that is known to be good.
For now we set the "npll" to the desired frequency and just set the core clk
divider to 1 as its parent it just used for the core clk.
ockchip/clk/rk3328_cru.c
ockchip/clk/rk_clk_armclk.c
ockchip/clk/rk_clk_armclk.h
ockchip/clk/rk_cru.c
ockchip/clk/rk_cru.h
f4398fd133626c022465b6cc0ece6e7ac6db6852 23-May-2018 markj <markj@FreeBSD.org> Add GET_STACK_USAGE() for arm64.

Its absence meant that GEOM direct dispatch was disabled (the service
routines check the current thread's stack usage to determine whether
to hand off the request to a dedicated thread), and this change is
sufficient to enable direct dispatch by default.

Reviewed by: allanjude
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D15527
nclude/proc.h
ebf60a8eff3ee39978aae7d9ba7c579130f78488 22-May-2018 andrew <andrew@FreeBSD.org> Revert r334035 for now. It breaks the boot on some boards as er expect to
be able to read UEFI RuntimeData memory via the DMAP region.
rm64/pmap.c
3510d0415081f8f9bd270af52112498eaf93ff5a 22-May-2018 andrew <andrew@FreeBSD.org> On ThunderX2 we need to be careful to only map the memory the firmware
lists in the EFI memory map. As such we need to reduce the mappings to
restrict them to not be the full 1G block. For now reduce this to a 2M
block, however this may be further restricted to be 4k page aligned as
other SoCs may require.

This allows ThunderX2 to boot reliably to userspace without performing
any speculative memory accesses to invalid physical memory.

Sponsored by: DARPA, AFRL
rm64/pmap.c
0aa60204cc75e0e42bdcda387f9a3937b286f2ff 22-May-2018 andrew <andrew@FreeBSD.org> Stop using the DMAP region to map ACPI memory.

On some arm64 boards we need to access memory in ACPI tables that is not
mapped in the DMAP region. To handle this create the needed mappings in
pmap_mapbios in the KVA space.

Submitted by: Michal Stanek (mst@semihalf.com)
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D15059
rm64/pmap.c
be2fcd48cd3deac9a96073770f22d35085a9b209 22-May-2018 andrew <andrew@FreeBSD.org> Switch arm64 to use the same physmem code as 32-bit arm.

The main advantage of this is to allow us to exclude memory from being
used by the kernel. This may be from the memreserve property, or ranges
marked as no-map under the reserved-memory node.

More work is still needed to remove the physmap array. This is still used
for creating the DMAP region, however other patches need to be committed
before we can remove this.

Obtained from: ABT Systems Ltd
Sponsored by: Turing Robotic Industries
rm64/machdep.c
rm64/pmap.c
5d4a66a5afdf1751c8b180b5854ed2fa8d872299 21-May-2018 andrew <andrew@FreeBSD.org> Restrict the faulting addresses we call pmap_fault from to just those that
may fault due to superpage mappings being changed.

Sponsored by: DARPA, AFRL
rm64/trap.c
5594f376686605041d4e8f019247d6613da0da90 19-May-2018 markj <markj@FreeBSD.org> Enable kernel dump features in GENERIC for most platforms.

This turns on support for kernel dump encryption and compression, and
netdump. arm and mips platforms are omitted for now, since they are more
constrained and don't benefit as much from these features.

Reviewed by: cem, manu, rgrimes
Tested by: manu (arm64)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D15465
onf/GENERIC
a92d2449069a61b14144c6afd144129c15783955 18-May-2018 cognet <cognet@FreeBSD.org> Instead of ignoring the VFP registers, set the dumppcb's pcb_fpusaved
field, so that they are saved, as they may be used in the kernel, in the
EFI and the crypto code.

Reviewed by: andrew
rm64/vfp.c
359ac3e429953b5eed45c7f0de6b123e50a821a3 18-May-2018 andrew <andrew@FreeBSD.org> Enable the Qualcomm MSM UART driver. This is needed for some Qualcomm
Snapdragon SoCs.

Obtained from: ABT Systems Ltd
Sponsored by: Turing Robotic Industries
onf/GENERIC
841df736abbe270a4708893a33ee41ec9b69dbf0 17-May-2018 cognet <cognet@FreeBSD.org> In pmap_get_tables(), check that the L2 is indeed a table before attempting
to get the l3.
rm64/pmap.c
b2d2968d5f38c09465dd85e9246572f8bb93221f 17-May-2018 cognet <cognet@FreeBSD.org> In vfp_save_state(), don't bother trying to save the VFP registers if the
provided PCB doesn't have a pcb_fpusaved. All PCBs associated to a thread
should have one, but the dumppcb used when panic'ing doesn't.
rm64/vfp.c
440049bc681a19d03966730b42b2c1b85e21e4ef 16-May-2018 avg <avg@FreeBSD.org> followup to r332730/r332752: set kdb_why to "trap" for fatal traps

This change updates arm, arm64 and mips achitectures. Additionally, it
removes redundant checks for kdb_active where it already results in
kdb_reenter() and adds kdb_reenter() calls where they were missing.

Some architectures check the return value of kdb_trap(), but some don't.
I haven't changed any of that.

Some trap handling routines have a return code. I am not sure if I
provided correct ones for returns after kdb_reenter(). kdb_reenter
should never return unless kdb_jmpbufp is NULL for some reason.

Only compile tested for all affected architectures. There can be bugs
resulting from my poor understanding of architecture specific details.

Reported by: jhb
Reviewed by: jhb, eadler
MFC after: 4 weeks
Differential Revision: https://reviews.freebsd.org/D15431
rm64/trap.c
9d8b5dd26c6e9ee40414f41656b71037d3efe5c4 15-May-2018 andrew <andrew@FreeBSD.org> Increase the number of pages we allocate in the arm64 early boot. We are
already close to the limit so increasing the kernel size may cause it to
fail to boot when it runs past the end of allocated memory.

Reported by: manu
Sponsored by: DARPA, AFRL
rm64/locore.S
6caecdadde9d667125e71bcbe31b493f6a79843f 10-May-2018 manu <manu@FreeBSD.org> arm64: Add ALT_BREAK_TO_DEBUGGER to GENERIC

It is useful to enter kdb with an escape sequence.
While here move the USB_DEBUG with the others debug options and define
nooptions USB_DEBUG for GENERIC-NODEBUG
onf/GENERIC
onf/GENERIC-NODEBUG
2f6fad05d8b0068d670a4f7c0f7554e542bf30f2 07-May-2018 manu <manu@FreeBSD.org> arm64: rockchip: cru: Call clk_set_assigned

We need to call clk_set_assigned after all the clock have been registered
to set the parents/rates described in the dtb.
ockchip/clk/rk_cru.c
371e7881779cc60746aeafd3b7923c98f0064a64 07-May-2018 manu <manu@FreeBSD.org> arm64: rockchip: clk: Add support to reparent to clk_composite

All clk_composite type have the possibility to reparent (choosing another
parent to find a better frequency), add the support for that.
ockchip/clk/rk_clk_composite.c
5cf7874c16a19c5b349cdf222fd0601448bc4707 07-May-2018 manu <manu@FreeBSD.org> arm64: rk3328: Add pll rates tables

Add the known value to be safe for the rk3328 PLLs
ockchip/clk/rk3328_cru.c
c2b91cdf4e0dd52cd8bc3c5a1a7a11f434da90f1 07-May-2018 manu <manu@FreeBSD.org> arm64: rk: Add support for setting pll rate

Add support for setting pll rate. On RockChip SoC two kind of plls are
supported, integer mode and fractional mode.
The two modes are intended to support more frequencies for the core plls.
While here change the recalc method as it appears that the datasheet is
wrong on the calculation method.
ockchip/clk/rk_clk_pll.c
ockchip/clk/rk_clk_pll.h
d14e211d4fb7f15e9c14945c5764d731e7fe422f 07-May-2018 manu <manu@FreeBSD.org> arm64: rockchip: rk3328: Add armclk clock

Add the clock definition for the arm clock.
While here remove the indexes in the clock table as we will need clock
with a 0 index (non-exported clocks).
ockchip/clk/rk3328_cru.c
65dc9f377b966bea7409baec4073b3ecbeaf4809 01-May-2018 markj <markj@FreeBSD.org> Print the dump progress indicator after calling dump_start().

Dumpers may wish to print messages from an initialization hook; this
change ensures that such messages aren't mixed with output from the
generic dump code.

MFC after: 1 week
rm64/minidump_machdep.c
99aa44799969ed53fcb6528531aee1737b1930e8 26-Apr-2018 manu <manu@FreeBSD.org> arm64: rockchip: rk_gpio fix rk_gpio_pin_config32

Pointy Hat to: me
ockchip/rk_gpio.c
0b8ea57a55e66f1847b48bf4afd78ce141f0e3cf 26-Apr-2018 manu <manu@FreeBSD.org> arm64: rockchip: Add gpio controller driver

Add a driver that match on 'rockchip,gpio-bank', this compatible
string is found on almost all RockChip SoC so this driver is compatible
with almost all of the RockChip SoCs.

The only features missing for this driver are :
- Interrupts support
- Debouncing
ockchip/rk_gpio.c
a06b782967edf82abcab403b4eacbd2f2097b97c 26-Apr-2018 manu <manu@FreeBSD.org> arm64: rockchip: RK3328 CRU Add gpio gates

Add the gates for the gpio controller in order to properly support them.
ockchip/clk/rk3328_cru.c
42097f6556679fef655cb6e9b3e824cbbf60e650 26-Apr-2018 manu <manu@FreeBSD.org> arm64: rockchip: Rk3328 CRU Fix some offset for gates

Some offset of some clock gates where wrong, correct them so we can
use thoses clocks.

Pointy Hat to: me
ockchip/clk/rk3328_cru.c
5cd7e598a9132c5bc67fbe1e499f92a7ad6e3e38 26-Apr-2018 manu <manu@FreeBSD.org> arm64: rockchip: Add pinctrl driver

Add pinctrl driver for RockChip SoCs. This device manage which function
to set on which pin and some other properties like pull up/down, drive
strength etc ...
For now the driver only support RK3328 but it is versatile enough to
add support for other RockChip SoC in the future.
ockchip/rk_pinctrl.c
d2a2b25e7b461dfedde07924c623646938865721 26-Apr-2018 manu <manu@FreeBSD.org> arm64: rockchip: Add GRF driver

RockChip GRF (General Register Files) is present on almost all RockChip
SoC and is used to control some area of the system like iomuxing, gpio
or usb phy.
We need it to be probed and attached early in the boot process so
subclass syscon_generic and set the pass to BUS_PASS_BUS + BUS_PASS_ORDER_MIDDLE.
ockchip/rk_grf.c
00bb4a5b36091b5701dff4261be93bdca7e25ab8 26-Apr-2018 emaste <emaste@FreeBSD.org> arm64 linuxulator: add generated sysent files

From syscalls.master in r333027

Sponsored by: Turing Robotic Industries Inc.
inux/linux_proto.h
inux/linux_syscall.h
inux/linux_syscalls.c
inux/linux_sysent.c
inux/linux_systrace_args.c
5912a23aef6b42327c6e297a6dc8fb79f144d6ca 26-Apr-2018 emaste <emaste@FreeBSD.org> Add arm64 Linux syscall table

This is the first step (after the recent refactoring of some common
code) to supporting the Linuxulator on arm64.

Reviewed by: andrew
Sponsored by: Turing Robotic Industries Inc.
Differential Revision: https://reviews.freebsd.org/D15187
inux/Makefile
inux/syscalls.conf
inux/syscalls.master
c35e9275fc8c33900d745812422d45f6fbd7e630 17-Apr-2018 brooks <brooks@FreeBSD.org> Remove the unused fuwintr() and suiwintr() functions.

Half of implementations always failed (returned (-1)) and they were
previously used in only one place.

Reviewed by: kib, andrew
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D15102
rm64/support.S
rm64/trap.c
976dcf366cf1d68cbcd10c73914d73e48fc8a72b 10-Apr-2018 br <br@FreeBSD.org> Enable Qualcomm Debug Subsystem (QDSS) block on MSM8916 SoC.

This is required for ARM Coresight operation on Dragonboard 410c.

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14987
onf/GENERIC
ualcomm/qcom_gcc.c
38bf332628ce405e2c495df0145ad5dcc9783dd4 08-Apr-2018 gonzo <gonzo@FreeBSD.org> Fix one more OF_getprop_alloc instance missed in r332310

X-MFC-With: r332310
oresight/coresight.c
9d79658aab1a30f34fee169ce74bdff4ca405c18 06-Apr-2018 brooks <brooks@FreeBSD.org> Move most of the contents of opt_compat.h to opt_global.h.

opt_compat.h is mentioned in nearly 180 files. In-progress network
driver compabibility improvements may add over 100 more so this is
closer to "just about everywhere" than "only some files" per the
guidance in sys/conf/options.

Keep COMPAT_LINUX32 in opt_compat.h as it is confined to a subset of
sys/compat/linux/*.c. A fake _COMPAT_LINUX option ensure opt_compat.h
is created on all architectures.

Move COMPAT_LINUXKPI to opt_dontuse.h as it is only used to control the
set of compiled files.

Reviewed by: kib, cem, jhb, jtl
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14941
rm64/machdep.c
7d671d9b44b0b2e9671d6cda2f350eae2dbaee9d 05-Apr-2018 br <br@FreeBSD.org> Add support for the Coresight technology from ARM Ltd.

ARM Coresight is a solution for debug and trace of complex SoC designs.

This includes a collection of drivers for ARM Coresight interconnect
devices within a small Coresight framework.

Supported devices are:
o Embedded Trace Macrocell v4 (ETMv4)
o Funnel
o Dynamic Replicator
o Trace Memory Controller (TMC)
o CPU debug module

Devices are connected to each other internally in SoC and the
configuration of each device endpoints is described in FDT.

Typical trace flow (as found on Qualcomm Snapdragon 410e):
CPU0 -> ETM0 -> funnel1 -> funnel0 -> ETF -> replicator -> ETR -> DRAM
CPU1 -> ETM1 -^
CPU2 -> ETM2 -^
CPU3 -> ETM3 -^

Note that both Embedded Trace FIFO (ETF) and Embedded Trace Router (ETR)
are hardware configurations of TMC.

This is required for upcoming HWPMC tracing support.

This is tested on single-core system only.

Reviewed by: andrew (partially)
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14618
oresight/coresight-cmd.c
oresight/coresight-cpu-debug.c
oresight/coresight-dynamic-replicator.c
oresight/coresight-etm4x.c
oresight/coresight-etm4x.h
oresight/coresight-funnel.c
oresight/coresight-funnel.h
oresight/coresight-tmc.c
oresight/coresight-tmc.h
oresight/coresight.c
oresight/coresight.h
oresight/coresight_if.m
b45524a54baa3a984536092413195f1f4ddf07e3 04-Apr-2018 gonzo <gonzo@FreeBSD.org> Fix arm64 buildkernel target with "nooptions KDB"

Make kdb_trap in breakpoint exception handler conditional. If "options KDB"
is not present just panic with message that debugger is not enabled.

PR: 224653
rm64/trap.c
6fe54a534393de74240a489e0ad0c2a20005a03e 20-Mar-2018 emaste <emaste@FreeBSD.org> Rename assym.s to assym.inc

assym is only to be included by other .s files, and should never
actually be assembled by itself.

Reviewed by: imp, bdrewery (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D14180
rm64/bzero.S
rm64/copyinout.S
rm64/exception.S
rm64/locore.S
rm64/support.S
rm64/swtch.S
29870e52c032f8fd43ce9e468fc67a621234f7c1 13-Mar-2018 kevans <kevans@FreeBSD.org> EFIRT: SetVirtualAddressMap with 1:1 mapping after exiting boot services

This fixes a problem encountered on the Lenovo Thinkpad X220/Yoga 11e where
runtime services would try to inexplicably jump to other parts of memory
where it shouldn't be when attempting to enumerate EFI vars, causing a
panic.

The virtual mapping is enabled by default and can be disabled by setting
efi_disable_vmap in loader.conf(5).

Reviewed by: kib (earlier version)
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D14677
rm64/efirt_machdep.c
39dd4716fd84db3ff7a6071bef1b55e920582ea9 09-Mar-2018 andrew <andrew@FreeBSD.org> Use the correct address to write back to memory in the GICv3 ITS driver.
This seems to no be needed on supported hardware as they are cache-coherent,
however this may not be the case on all platforms.

Sponsored by: DARPA, AFRL
rm64/gicv3_its.c
d682824945606bc4d2cf440aabbf26045a95d128 07-Mar-2018 andrew <andrew@FreeBSD.org> Bump MAXCPUS on arm64. We are starting to see hardware with more than 96
cores so increase it to the same as amd64.

Sponsored by: DARPA, AFRL
Sponsored by: Cavium (Hardware)
nclude/param.h
ed08bbe6a22c1a97e7db76f146f6d7ccf13e7ce7 07-Mar-2018 andrew <andrew@FreeBSD.org> Create macros for the ACPI interrupt cross references. This is considered a
band aid until a better solution to find the correct interrupt controller
can be found.

While here fix one place in the GICv3 ITS driver where the offset wasn't
correctly applied.

Sponsored by: DARPA, AFRL
Sponsored by: Cavium (Hardware)
rm64/gic_v3_acpi.c
rm64/gicv3_its.c
rm64/nexus.c
nclude/intr.h
32b7f64a11f2bbf30509b7759a09b82b7aa4087c 07-Mar-2018 andrew <andrew@FreeBSD.org> Restrict the arm64 DMAP region to the 1G blocks where we have at least
one physical page. This is in preparation for limiting it further as this
is needed on some hardware, however testing has shown issues with further
restricting the DMAP and ACPI.

Sponsored by: DARPA, AFRL
Sponsored by: Cavium (Hardware)
rm64/pmap.c
8ea72b389428c22ed20d33f6baf2fa2d0565504b 05-Mar-2018 andrew <andrew@FreeBSD.org> Register each GICv3 ITS driver with a useful cross reference. We currently
only use the first driver, however this may change in the future and
hardware exists with multiple ITS devices.

Sponsored by: DARPA, AFRL
Sponsored by: Cavium (Hardware)
rm64/gicv3_its.c
82fd7d44b1e301a6de2f78d216b8ed44f35aaae2 05-Mar-2018 andrew <andrew@FreeBSD.org> In the ACPI GICv3 attach function call device_get_children to get the list
of children. We expect this to be populated when configuring the secondary
cores.

Sponsored by: DARPA, AFRL
Sponsored by: Cavium (Hardware)
rm64/gic_v3_acpi.c
d27cb27779af2bfbbd568a8876720e63dd0b6242 04-Mar-2018 kib <kib@FreeBSD.org> Unify bulk free operations in several pmaps.

Submitted by: Yoshihiro Ota
Reviewed by: markj
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D13485
rm64/pmap.c
f59ac355debccc892eaa539a39d2c9b42d35c8b0 03-Mar-2018 andrew <andrew@FreeBSD.org> Move setting the IRQ base and length into the common GICv3 ITS attach
function. This is common across both ACPI and FDT.

Sponsored by: ABT Systems Ltd
Sponsored by: Cavium (Hardware)
rm64/gicv3_its.c
50b474845367793c9943ce1869c09577fe84b4c3 03-Mar-2018 andrew <andrew@FreeBSD.org> Add the missing GICv3 dev info struct to the ACPI dev info.Previously we
would read from the resource list when querying from this.

Sponsored by: ABT Systems Ltd
Sponsored by: Cavium (Hardware)
rm64/gic_v3_acpi.c
b03552df20e89d8b242e2393ed34e0b5a129b37c 02-Mar-2018 andrew <andrew@FreeBSD.org> Count the number of children in the GICv3 ACPI attachment. This is needed
later for assigning IRQ vector space.

Sponsored by: DARPA, AFRL
Sponsored by: Cavium (Hardware)
rm64/gic_v3_acpi.c
ca026b5b2273c3f0e76355abf6a1e2bec5e1d5a6 28-Feb-2018 andrew <andrew@FreeBSD.org> Allow releasing APs to take more time, as long as we are making progress.
On large core count machines this can be slow while all the CPUs update
the online counter.

Sponsored by: DARPA, AFRL
Sponsored by: Cavium (Hardware)
rm64/mp_machdep.c
92b7e35d588549cd656da3243a4bfc8503f16869 28-Feb-2018 andrew <andrew@FreeBSD.org> Count the number of GIC redistributors in the ACPI tables. The GICv3 driver
needs this to allocate memory, and connect the CPUs to the interrupt
controller.

Sponsored by: DARPA, AFRL
Sponsored by: Cavium (Hardware)
rm64/gic_v3_acpi.c
c52197145fc0d4f645b7e81d5a76401aef94bf7a 26-Feb-2018 manu <manu@FreeBSD.org> rk_cru: Add missing break
ockchip/clk/rk_cru.c
b7a05ac56b723ebf0fb22d4229d4392294ff3883 26-Feb-2018 manu <manu@FreeBSD.org> rk3328: Add support for this SoC

* rk_cru is a cru driver that needs to be subclassed by
the real CRU driver
* rk_clk_pll handle the pll type clock on RockChip SoC, it's only read
only for now.
* rk_clk_composite handle the different composite clock types (with gate,
with mux etc ...)
* rk_clk_gate handle the RockChip gates
* rk_clk_mux handle the RockChip muxes (unused for now)
* Only clocks for supported devices are supported for now, the rest will be
added when driver support comes
* The assigned-clock* property are not handled for now so we rely a lot on the
bootloader to setup some initial values for some clocks.
onf/GENERIC
ockchip/clk/rk3328_cru.c
ockchip/clk/rk_clk_composite.c
ockchip/clk/rk_clk_composite.h
ockchip/clk/rk_clk_gate.c
ockchip/clk/rk_clk_gate.h
ockchip/clk/rk_clk_mux.c
ockchip/clk/rk_clk_mux.h
ockchip/clk/rk_clk_pll.c
ockchip/clk/rk_clk_pll.h
ockchip/clk/rk_cru.c
ockchip/clk/rk_cru.h
24fc3ea5c183e3a4a43d0630db4c5fe24e8ca0bb 26-Feb-2018 cognet <cognet@FreeBSD.org> In do_ast, make sure the interrupts are enabled before calling ast().
We can reach that point with IRQs disabled, and calling ast() with IRQs 
disabled can lead to a deadlock.
This should fix the freezes on arm64 under load.

Reviewed by: andrew
rm64/exception.S
91dbc8d21a22bc933926051ab9321ff7513c61b5 26-Feb-2018 pkelsey <pkelsey@FreeBSD.org> Greatly reduce the number of #ifdefs supporting the TCP_RFC7413 kernel option.

The conditional compilation support is now centralized in
tcp_fastopen.h and tcp_var.h. This doesn't provide the minimum
theoretical code/data footprint when TCP_RFC7413 is disabled, but
nearly all the TFO code should wind up being removed by the optimizer,
the additional footprint in the syncache entries is a single pointer,
and the additional overhead in the tcpcb is at the end of the
structure.

This enables the TCP_RFC7413 kernel option by default in amd64 and
arm64 GENERIC.

Reviewed by: hiren
MFC after: 1 month
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D14048
onf/GENERIC
a40522daab14c326b5aa38958a5973b6063cb564 24-Feb-2018 andrew <andrew@FreeBSD.org> Correctly set the 16kB page size field in the ITS BASER register. Some
new arm64 hardware, e.g. ThunderX2, seems to use this page size so was
failing to attach as the register value read back was incorrect.

While here fix the spelling on shareability.

Sponsored by: DARPA, AFRL
rm64/gicv3_its.c
807035047f6622be5e9564374cec244b3a9de204 23-Feb-2018 cem <cem@FreeBSD.org> Remove unused error return from API that cannot fail

No implementation of fpu_kern_enter() can fail, and it was causing needless
error checking boilerplate and confusion. Change the return code to void to
match reality.

(This trivial change took nine days to land because of the commit hook on
sys/dev/random. Please consider removing the hook or otherwise lowering the
bar -- secteam never seems to have free time to review patches.)

Reported by: Lachlan McIlroy <Lachlan.McIlroy AT isilon.com>
Reviewed by: delphij
Approved by: secteam (delphij)
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14380
rm64/vfp.c
nclude/vfp.h
f735e1eb15d21bda71b5985d15edacddf0fabdae 23-Feb-2018 imp <imp@FreeBSD.org> Do not include float interfaces when using libsa.

We don't support float in the boot loaders, so don't include
interfaces for float or double in systems headers. In addition, take
the unusual step of spiking double and float to prevent any more
accidental seepage.
nclude/_types.h
ee3d0fb8ef5def49a12c51a3030af447009a1ec5 20-Feb-2018 kib <kib@FreeBSD.org> vm_wait() rework.

Make vm_wait() take the vm_object argument which specifies the domain
set to wait for the min condition pass. If there is no object
associated with the wait, use curthread' policy domainset. The
mechanics of the wait in vm_wait() and vm_wait_domain() is supplied by
the new helper vm_wait_doms(), which directly takes the bitmask of the
domains to wait for passing min condition.

Eliminate pagedaemon_wait(). vm_domain_clear() handles the same
operations.

Eliminate VM_WAIT and VM_WAITPFAULT macros, the direct functions calls
are enough.

Eliminate several control state variables from vm_domain, unneeded
after the vm_wait() conversion.

Scetched and reviewed by: jeff
Tested by: pho
Sponsored by: The FreeBSD Foundation, Mellanox Technologies
Differential revision: https://reviews.freebsd.org/D14384
rm64/pmap.c
ba27b5187b587e8884e2d94794d6b136e6ab75dc 12-Feb-2018 jeff <jeff@FreeBSD.org> Make v_wire_count a per-cpu counter(9) counter. This eliminates a
significant source of cache line contention from vm_page_alloc(). Use
accessors and vm_page_unwire_noq() so that the mechanism can be easily
changed in the future.

Reviewed by: markj
Discussed with: kib, glebius
Tested by: pho (earlier version)
Sponsored by: Netflix, Dell/EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14273
rm64/efirt_machdep.c
rm64/pmap.c
8256f7ec08fe026d302c529a2ff8908609b454bc 12-Feb-2018 imp <imp@FreeBSD.org> Move __va_list and related defines to sys/sys/_types.h

__va_list and related defines are identical in all the
ARCH/include/_types.h files. Move them to sys/sys/_types.h

Sponsored by: Netflix
nclude/_types.h
7bc81b6db11ed4d602eb59655876fffe84a98803 08-Feb-2018 markj <markj@FreeBSD.org> Use vm_page_unwire_noq() instead of directly modifying page wire counts.

No functional change intended.

Reviewed by: alc, kib (previous revision)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D14266
rm64/pmap.c
rm64/uma_machdep.c
ef01ed01d824f2fedfc7517b00ba9bfd5229311d 01-Feb-2018 andrew <andrew@FreeBSD.org> Only promote userspace mappings to superpages. This was dropped in r328510,
however due to the break-before-make requirement on arm64 is is currently
unsafe to promote kernel pages.

Sponsored by: DARPA, AFRL
rm64/pmap.c
715591cf86f7469f7a766c808db6a918992adda9 28-Jan-2018 mmel <mmel@FreeBSD.org> Remove #endif forgotten in r328510.

Pointy hat: mmel
rm64/pmap.c
de41c46fdd0ee23b479fba9e27f8dc51ddd615b2 28-Jan-2018 mmel <mmel@FreeBSD.org> Fix handling of I-cache sync operations

- pmap_enter_object() can be used for mapping of executable pages, so it's
necessary to handle I-cache synchronization within it.

- Fix race in I-cache synchronization in pmap_enter(). The current code firstly
maps given page to target VA and then do I-cache sync on it. This causes
race, because this mapping become visible to other threads, before I-cache
is synced.
Do sync I-cache firstly (by using DMAP VA) and then map it to target VA.

- ARM64 ARM permits implementation of aliased (AIVIVT, VIPT) I-cache, but we
can use different that final VA for flushing it. So we should use full
I-cache flush on affected platforms. For now, and as temporary solution,
use full flush always.
rm64/cpufunc_asm.S
rm64/pmap.c
84789f49db2180c3c58a9845953a1b171b0cd0a4 27-Jan-2018 mmel <mmel@FreeBSD.org> Fix pmap_fault().

- special fault handling for break-before-make mechanism should be also
applied for instruction translation faults, not only for data translation
faults.

- since arm64_address_translate_...() functions are not atomic,
use these with disabled interrupts.
rm64/pmap.c
ced875130d74498a2efed66b4800f0a211ad5993 21-Jan-2018 pfg <pfg@FreeBSD.org> Revert r327828, r327949, r327953, r328016-r328026, r328041:
Uses of mallocarray(9).

The use of mallocarray(9) has rocketed the required swap to build FreeBSD.
This is likely caused by the allocation size attributes which put extra pressure
on the compiler.

Given that most of these checks are superfluous we have to choose better
where to use mallocarray(9). We still have more uses of mallocarray(9) but
hopefully this is enough to bring swap usage to a reasonable level.

Reported by: wosch
PR: 225197
rm64/busdma_bounce.c
e79f2b9178164cd2a849cf2496f988d5c7d67fa3 19-Jan-2018 nwhitehorn <nwhitehorn@FreeBSD.org> Remove SFBUF_OPTIONAL_DIRECT_MAP and such hacks, replacing them across the
kernel by PHYS_TO_DMAP() as previously present on amd64, arm64, riscv, and
powerpc64. This introduces a new MI macro (PMAP_HAS_DMAP) that can be
evaluated at runtime to determine if the architecture has a direct map;
if it does not (or does) unconditionally and PMAP_HAS_DMAP is either 0 or
1, the compiler can remove the conditional logic.

As part of this, implement PHYS_TO_DMAP() on sparc64 and mips64, which had
similar things but spelled differently. 32-bit MIPS has a partial direct-map
that maps poorly to this concept and is unchanged.

Reviewed by: kib
Suggestions from: marius, alc, kib
Runtime tested on: amd64, powerpc64, powerpc, mips64
nclude/vmparam.h
8673b42398665e236ad164b473cdaa708fdea5fc 18-Jan-2018 br <br@FreeBSD.org> Set the base address of translation table 0.

This fixes operation on Qualcomm Snapdragon and some other platforms.

During boot time on subsystems initialization we have some amount of
kernel threads created, then scheduler gives CPU time to each thread.
Eventually scheduler returns CPU execution back to thread 0. In this
case writing zero to ttbr0 in cpu_switch leads Qualcomm board to
reboot (asynchronously, CPU continues execution).

Similar to other kernel threads install a valid physical address
(kernel pmap) to user page table base register ttbr0.

Reviewed by: andrew
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D13536
rm64/machdep.c
35d8f24a88ef039ca331d7b9581a3ccb99b3f1f0 18-Jan-2018 andrew <andrew@FreeBSD.org> Add a pmap invalidate that doesn't call sched_pin.

When demoting DMAP pages curthread may be pointing to data within the
page we are demoting. Create a new invalidate that doesn't pin and use
it in the demote case.

As the demote has both interrupts disabled, and is within a critical section
this is safe from having the scheduler from switching to another CPU.

Reported by: loos
Reviewed by: loos
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D13955
rm64/pmap.c
af8b614ef3330a479f6d31592c9a41da4ca7fe10 15-Jan-2018 pfg <pfg@FreeBSD.org> arm: make some use of mallocarray(9).

Focus on code where we are doing multiplications within malloc(9). None of
these ire likely to overflow, however the change is still useful as some
static checkers can benefit from the allocation attributes we use for
mallocarray.

This initial sweep only covers malloc(9) calls with M_NOWAIT. No good
reason but I started doing the changes before r327796 and at that time it
was convenient to make sure the sorrounding code could handle NULL values.

X-Differential revision: https://reviews.freebsd.org/D13837
rm64/busdma_bounce.c
bc9177f3a22b15279228a85fa9186680f37067c5 12-Jan-2018 jeff <jeff@FreeBSD.org> Add support for NUMA domains to bus dma tags. This causes all memory
allocated with a tag to come from the specified domain if it meets the
other constraints provided by the tag. Automatically create a tag at
the root of each bus specifying the domain local to that bus if
available.

Reviewed by: jhb, kib
Tested by: pho
Sponsored by: Netflix, Dell/EMC Isilon
Differential Revision: https://reviews.freebsd.org/D13545
rm64/busdma_machdep.c
f375b4dd66bc709573ad4286b3a5ac6c705ba4a8 12-Jan-2018 jeff <jeff@FreeBSD.org> Implement NUMA support in uma(9) and malloc(9). Allocations from specific
domains can be done by the _domain() API variants. UMA also supports a
first-touch policy via the NUMA zone flag.

The slab layer is now segregated by VM domains and is precise. It handles
iteration for round-robin directly. The per-cpu cache layer remains
a mix of domains according to where memory is allocated and freed. Well
behaved clients can achieve perfect locality with no performance penalty.

The direct domain allocation functions have to visit the slab layer and
so require per-zone locks which come at some expense.

Reviewed by: Attilio (a slightly older version)
Tested by: pho
Sponsored by: Netflix, Dell/EMC Isilon
rm64/uma_machdep.c
fe961bddea8c4db5961ee01b31ee987107e97b94 12-Jan-2018 andrew <andrew@FreeBSD.org> Workaround Spectre Variant 2 on arm64.

We need to handle two cases:

1. One process attacking another process.
2. A process attacking the kernel.

For the first case we clear the branch predictor state on context switch
between different processes. For the second we do this when taking an
instruction abort on a non-userspace address.

To clear the branch predictor state a per-CPU function pointer has been
added. This is set by the new cpu errata code based on if the CPU is
known to be affected.

On Cortex-A57, A72, A73, and A75 we call into the PSCI firmware as newer
versions of this will clear the branch predictor state for us.

It has been reported the ThunderX is unaffected, however the ThunderX2 is
vulnerable. The Qualcomm Falkor core is also affected. As FreeBSD doesn't
yet run on the ThunderX2 or Falkor no workaround is included for these CPUs.

MFC after: 3 days
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D13812
rm64/cpu_errata.c
rm64/pmap.c
rm64/trap.c
nclude/pcpu.h
aff7b081066af2a8d9d20c46763f21622f792dfe 11-Jan-2018 andrew <andrew@FreeBSD.org> iAdd ACPI attachments the the GIC and GICv3 interrupt controller drivers.
For each we need to walk the MADT to find which we have, then add the
driver as needed. As each may have a child they will each walk the same
table to find these details.

Reviewed by: mmel
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8720
rm64/gic_v3.c
rm64/gic_v3_acpi.c
rm64/gic_v3_var.h
rm64/gicv3_its.c
048d22eb182d47fbcfd2c9019b816c916390d6da 11-Jan-2018 andrew <andrew@FreeBSD.org> Add the start of INTRNG support for ACPI.

This adds a new acpi_bus interface with a map_intr method. This is similar
to the Open Firmware map_intr method and allows us to create the needed
mapping from ACPI space to INTRNG space.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8617
rm64/nexus.c
7bd006d93c1ce097f69b13eab14e8c9c5a54c8a0 11-Jan-2018 mmel <mmel@FreeBSD.org> Initialize CONTEXTIDR register on secondary cores by zero,
not with undefined value from X1 register.

MFC after: 1 month
rm64/locore.S
34a4720159116bd947b25102ebc2f60dd5d7fe7a 09-Jan-2018 andrew <andrew@FreeBSD.org> Add a framework to install CPU errata on arm64. Each erratum can encode
a mask and value to compare with the Main ID Register. If these match then a
function is called to handle the installation of the erratum workaround.

No errata are currently handled, however this will change soon in a future
commit.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/cpu_errata.c
rm64/machdep.c
rm64/mp_machdep.c
nclude/cpu.h
9582669eaa6fb8d5ac5cabb12466614023e4c43f 08-Jan-2018 andrew <andrew@FreeBSD.org> Only install the new pagetable pointer into ttbr0_el1 when it differs from
the existing value.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/pmap.c
720167784985ac27e50c8f4989eb210b4f082c76 08-Jan-2018 andrew <andrew@FreeBSD.org> Move some of the common thread switching code into C. This will help with
future optimisations, e.g. using Address Space IDs (asid).

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/pmap.c
rm64/swtch.S
nclude/pmap.h
e9f76e5656a57a5ba719c30d3ed273b988c5820f 04-Jan-2018 kevans <kevans@FreeBSD.org> if_awg: Use syscon prop if it exists

The emac bindings that are landing in Linux 4.15 specify a syscon property
on the emac node that point to /soc/syscon. Use this property if it's
specified, but maintain backwards compatibility with the old method.

The older method is still used for boards that we get .dtb from u-boot, such
as pine64, that did not yet have stable emac bindings.

Tested on: Banana Pi-M3 (a83t)
Tested on: Pine64 (a64)
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D13296
onf/GENERIC
1642f8ba74084ae5f9b239b2ef80766ff232c3b4 25-Dec-2017 phk <phk@FreeBSD.org> Introduce an architecture-agnostic <sys/_stdarg.h> to reduce
platform divergence.

Only architectures which pass arguments in registers (mips)
and platforms which use really weird compilers (any?) would
need to augment the contents of <sys/_stdarg.h>

Convert x86, arm and arm64 architectures to use <sys/_stdarg.h>
nclude/stdarg.h
c8da6fae2c8073f216b7a35739f7dfa140c9a8d9 25-Dec-2017 kan <kan@FreeBSD.org> Do pass removing some write-only variables from the kernel.

This reduces noise when kernel is compiled by newer GCC versions,
such as one used by external toolchain ports.

Reviewed by: kib, andrew(sys/arm and sys/arm64), emaste(partial), erj(partial)
Reviewed by: jhb (sys/dev/pci/* sys/kern/vfs_aio.c and sys/kern/kern_synch.c)
Differential Revision: https://reviews.freebsd.org/D10385
rm64/gic_v3.c
rm64/gicv3_its.c
rm64/machdep.c
rm64/pmap.c
3d18a9d66f1815b256f10a6e6b496ae412109193 19-Dec-2017 kib <kib@FreeBSD.org> Add atomic_load(9) and atomic_store(9) operations.

They provide relaxed-ordered atomic access semantic. Due to the
FreeBSD memory model, the operations are syntaxical wrappers around
the volatile accesses. The volatile qualifier is used to ensure that
the access not optimized out and in turn depends on the volatile
semantic as implemented by supported compilers.

The motivation for adding the operation is to help people coming from
other systems or knowing the C11/C++ standards where atomics have
special type and require use of the special access operations. It is
still the case that FreeBSD requires plain load and stores of aligned
integer types to be atomic.

Suggested by: jhb
Reviewed by: alc, jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D13534
nclude/atomic.h
d81660feebdf285121d1946decba91a190e590e0 07-Dec-2017 bde <bde@FreeBSD.org> Move instantiation of msgbufp from 9 MD files to subr_prf.c.

This variable should be pure MI except possibly for reading it in MD
dump routines. Its initialization was pure MD in 4.4BSD, but FreeBSD
changed this in r36441 in 1998. There were many imperfections in
r36441. This commit fixes only a small one, to simplify fixing the
others 1 arch at a time. (r47678 added support for
special/early/multiple message buffer initialization which I want in
a more general form, but this was too fragile to use because hacking
on the msgbufp global corrupted it, and was only used for 5 hours in
-current...)
rm64/pmap.c
9e0a98e9455cd0358c8be59e659e2eaae194215b 04-Dec-2017 andrew <andrew@FreeBSD.org> Use the module pointer to find the address we need to map to in the early
arm64 boot sequence. This will be a virtual address in the kernel space
after the kernel and any modules loaded by loader so we can use this to
find the size of the kernel + modules. We can then add on a level 2 page for
the module data and round up the size to be aligned to a level 2 page.

This allows more than 8 MiB of modules to be loaded by loader, e.g. zfs.ko
and opensolaris.ko.

Reported by: Shawn Webb
MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/locore.S
5a2c860a91fca4941e9111fbeeaef1b119352535 03-Dec-2017 ed <ed@FreeBSD.org> Make COMPAT_FREEBSD32 part of GENERIC on arm64.

The cloudabi32.ko kernel modules can only be loaded on AMD64 and ARM64
by kernels built with COMPAT_FREEBSD32. Even though COMPAT_FREEBSD32
does not support the execution of native FreeBSD executables, do add it
to GENERIC, to make cloudabi32.ko usable.

According to size(1), this makes the kernel image approximately 0.7%
larger.

Reviewed by: andrew, imp, emaste
Differential Revision: https://reviews.freebsd.org/D13311
onf/GENERIC
0ac5e9bc89eccd8e741feb1ee5f0418836aa3498 30-Nov-2017 ed <ed@FreeBSD.org> Port cloudabi32.ko to FreeBSD/arm64.

This change adds an implementation of a sysent for running CloudABI
armv6 and armv7 binaries on FreeBSD/arm64. It is a somewhat literal copy
of the armv6 version, except that it's been patched up to use the proper
registers.

Just like for cloudabi32.ko on FreeBSD/amd64, we make use of a vDSO that
automatically pads system call parameters to 64-bit value. These are
stored in a buffer on the stack, meaning we need to use copyin() and
copyout() unconditionally.
loudabi32/cloudabi32_sysvec.c
155122ce53ace6592a7b85c4aea97a27fb29f723 30-Nov-2017 pfg <pfg@FreeBSD.org> SPDX: Consider code from Carnegie-Mellon University.

Interesting cases, most likely from CMU Mach sources.
nclude/profile.h
57c55a6746bd175e20d07710b7a3c2e6202affa0 28-Nov-2017 andrew <andrew@FreeBSD.org> Fix the direction we move the instruction to retuirn to after exiting the
kernel debugger. We should skip the breakpoint instruction, not execute the
instruction before it.

Pointy-hat to: andrew
MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/trap.c
042b16bcf30a476da57edcf615d1d3987ba5ac7a 28-Nov-2017 andrew <andrew@FreeBSD.org> When we exit the kernel debugger having entered because of a breakpoint
instruction we need to jump over the instruction. Without this we will
execute the same instruction again and enter into the debugger again.

PR: 223917
Reported by: emaste
MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/trap.c
dbbac96e0da2425986f3571d7795b44864591dfd 28-Nov-2017 kib <kib@FreeBSD.org> Fix index calculation for the page table pages for efirt 1:1 map.

Stop issuing pre-assigned number to enumerate all page table pages,
the assignment is incorrect. Instead automatically calculate the next
unused index. This index in fact does not serve any purpose except to
be unique to satisfy vm_page_grab() interface, we do not look up the
page by the index later.

Reported and tested by: emaste
Reviewed by: andrew
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
PR: 223906
Differential revision: https://reviews.freebsd.org/D13273
rm64/efirt_machdep.c
d5748f43822ce7c0e18b90cb0109086c069aade1 27-Nov-2017 andrew <andrew@FreeBSD.org> Set the kernel file name so the kern.bootfile sysctl is set. This allows
for the creation of kernel.old when running installkernel.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/machdep.c
8a2fa414aa05f4bb823f5c14b464e0222aee5ac4 26-Nov-2017 ed <ed@FreeBSD.org> Add a Saved Process Status Register bit for AArch32 execution mode.

The documentation on the Saved Process Status Register (SPSR) is a bit
weird; the M[4] bit is documented separately from M[3:0]. The M[4] bit
can be toggled to switch to 32-bit execution mode. This functionality is
orthogonal to M[3:0].

Change the definition of PSR_M_MASK to no longer include M[4]. Add a new
definition, PSR_AARCH32 that can be used to toggle 32-bit independently.
This bit will be used by the cloudabi32 code to force execution of
userspace code in 32-bit mode.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D13148
rm64/machdep.c
nclude/armreg.h
887cf5c14ef6bae675401119cd05d390ca495ea2 26-Nov-2017 ed <ed@FreeBSD.org> Correct some more places where TO_PTR() should be used.

These were missed in r326228.

MFC after: 2 weeks
loudabi64/cloudabi64_sysvec.c
d13a2ec25480e5b9cc94d3896dfa385d844e17c3 26-Nov-2017 ed <ed@FreeBSD.org> Use TO_PTR() to convert integers to pointers.

For FreeBSD/arm64's cloudabi32 support, I'm going to need a TO_PTR() in
this place. Also use it for all of the other source files, so that the
difference remains as minimal as possible.

MFC after: 2 weeks
loudabi64/cloudabi64_sysvec.c
d1deb1fb9ee591a8b9b2b32048f9db2297df3e2e 26-Nov-2017 ed <ed@FreeBSD.org> Make 32-bit system calls end up in svc_handler().

The nice thing about ARM64 is that it's pretty elegant to install
separate trap/exception handlers for 32-bit and 64-bit processes. That
said, for all other architectures (e.g., i386 on amd64) we always let
32-bit counterparts go through the regular system call codepath. Let's
do the same on ARM64.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D13146
rm64/exception.S
rm64/trap.c
nclude/armreg.h
167d484bb753f21c11ba289047c13e544fa6da22 26-Nov-2017 andrew <andrew@FreeBSD.org> Make the arm64 pmap_invalidate functions static inline. This fixes building
with DIAGNOSTIC.

PR: 223874
Reported by: emaste
MFC after: 1 week
rm64/pmap.c
40ebf27a670a93bad4c2b1e985cfcd4f9301f18f 24-Nov-2017 emaste <emaste@FreeBSD.org> Temporarily disable VIMAGE on arm64

Loading a kernel module with a static VNET_DEFINE'd variable (e.g.
if_lagg) currently results in a kernel panic.

PR: 223670
onf/GENERIC
f670022acb3c892024f7c5435b93de145c69ea30 24-Nov-2017 ed <ed@FreeBSD.org> Set CP15BEN in SCTLR to make memory barriers work in 32-bit mode.

Binaries generated by Clang for ARMv6 may contain these instructions:

MCR p15, 0, <Rd>, c7, c10, 5

These instructions are deprecated as of ARMv7, which is why modern
processors have a way of toggling support for them. On FreeBSD/arm64 we
currently disable support for these instructions, meaning that if 32-bit
executables with these instructions are run, they would crash with
SIGILL. This is likely not what we want.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D13145
rm64/locore.S
a73a92ed88a369c7b3304d2666a0e358e37cedf8 24-Nov-2017 ed <ed@FreeBSD.org> Add rudimentary support for building FreeBSD/arm64 with COMPAT_FREEBSD32.

Right now I'm using two Raspberry Pi's (2 and 3) to test CloudABI
support for armv6, armv7 and aarch64. It would be nice if I could
restrict this to just a single instance when testing smaller changes.
This is why I'd like to get COMPAT_CLOUDABI32 to work on arm64.

As COMPAT_CLOUDABI32 depends on COMPAT_FREEBSD32, at least for the ELF
loading, this change adds all of the bits necessary to at least build a
kernel with COMPAT_FREEBSD32. All of the machine dependent system calls
are still stubbed out, for the reason that implementations for these are
only useful if actual support for running FreeBSD binaries is added.
This is outside the scope of this work.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D13144
rm64/elf32_machdep.c
rm64/freebsd32_machdep.c
rm64/machdep.c
nclude/elf.h
nclude/param.h
nclude/proc.h
nclude/reg.h
nclude/vdso.h
9b06c6070c5c2a454f1e1ae3a3fedad9f8f1ebc4 24-Nov-2017 ed <ed@FreeBSD.org> Don't let cpu_set_syscall_retval() clobber exec_setregs().

Upon successful completion, the execve() system call invokes
exec_setregs() to initialize the registers of the initial thread of the
newly executed process. What is weird is that when execve() returns, it
still goes through the normal system call return path, clobbering the
registers with the system call's return value (td->td_retval).

Though this doesn't seem to be problematic for x86 most of the times (as
the value of eax/rax doesn't matter upon startup), this can be pretty
frustrating for architectures where function argument and return
registers overlap (e.g., ARM). On these systems, exec_setregs() also
needs to initialize td_retval.

Even worse are architectures where cpu_set_syscall_retval() sets
registers to values not derived from td_retval. On these architectures,
there is no way cpu_set_syscall_retval() can set registers to the way it
wants them to be upon the start of execution.

To get rid of this madness, let sys_execve() return EJUSTRETURN. This
will cause cpu_set_syscall_retval() to leave registers intact. This
makes process execution easier to understand. It also eliminates the
difference between execution of the initial process and successive ones.
The initial call to sys_execve() is not performed through a system call
context.

Reviewed by: kib, jhibbits
Differential Revision: https://reviews.freebsd.org/D13180
rm64/machdep.c
loudabi64/cloudabi64_sysvec.c
851e08541e235be654a8773760db195a53ecf094 23-Nov-2017 andrew <andrew@FreeBSD.org> Ensure we check the program state set in the trap frame on arm and arm64.
This value may be set by userspace so we need to check it before using it.
If this is not done correctly on exception return the kernel may continue
in kernel mode with all registers set to a userspace controlled value. Fix
this by moving the check into set_mcontext, and also add the missing
sanitisation from the arm64 set_regs.

Discussed with: security-officer@
MFC after: 3 days
Sponsored by: DARPA, AFRL
rm64/machdep.c
nclude/armreg.h
826cc84370e1cb8da43c7453d92f0aed1ecc415e 21-Nov-2017 andrew <andrew@FreeBSD.org> Add a driver for the EFI RTC. This uses the EFI Runtime Services to query
the system time.

As we seem to only read this time on boot, and this is the only source of
time on many arm64 machines we need to enable this by default there. As
this is not always the case with U-Boot firmware, or when we have been
booted from a non-UEFI environment we only enable the device driver when
the Runtime Services are present and reading the time doesn't result in an
error.

PR: 212185
Reviewed by: imp, kib
Tested by: emaste
Relnotes: yes
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12650
onf/GENERIC
e6d3918553e2ee1683231f0615ecc7cca33fa547 21-Nov-2017 andrew <andrew@FreeBSD.org> When fpcurthread is not the current thread it may be non-NULL. In this
case another thread has had the VFP unit enabled and will have its state
in the VFP registers along with it stored in memory. As such we don't need
to store the state, but do need to zero the fpcurthread pointer to stop
the VFP driver from using the enable fast path.

Reported by: emaste
Sponsored by: DARPA, AFRL
rm64/vfp.c
3c355d849c9773c9c1c59217da9511e500bb4e15 08-Nov-2017 jeff <jeff@FreeBSD.org> Replace manyinstances of VM_WAIT with blocking page allocation flags
similar to the kernel memory allocator.

This simplifies NUMA allocation because the domain will be known at wait
time and races between failure and sleeping are eliminated. This also
reduces boilerplate code and simplifies callers.

A wait primitive is supplied for uma zones for similar reasons. This
eliminates some non-specific VM_WAIT calls in favor of more explicit
sleeps that may be satisfied without new pages.

Reviewed by: alc, kib, markj
Tested by: pho
Sponsored by: Netflix, Dell/EMC Isilon
rm64/uma_machdep.c
d9baf196b0c673d170fa1fcc43d362f98d14a328 31-Oct-2017 andrew <andrew@FreeBSD.org> Allocate the ITS translation table with a 64k page alignment. This is the
largest alignment the ITS can require.

This fixes a bug with the ARM Architecture Envelope Model (AEM) where it
only allows 64k pages so will fail to attach the ITS device when this table
is not sufficiently aligned.

Sponsored by: DARPA, AFRL
rm64/gicv3_its.c
fa6594dcb9ead3075609a41365ed7c90962d150a 31-Oct-2017 andrew <andrew@FreeBSD.org> As with r325242 use mp_maxid when iterating over CPUs in the GICv3 driver.

Sponsored by: DARPA, AFRL
rm64/gic_v3.c
320f5e6d7856112cf1f5345d7d98c6ecb27700ad 31-Oct-2017 andrew <andrew@FreeBSD.org> Use mp_maxid when iterating over CPUs as we may have sparse id allocations.

Sponsored by: DARPA, AFRL
rm64/gicv3_its.c
45275e3a26b0ae5f7099ff00a72ec641eb1f9aa6 29-Oct-2017 eadler <eadler@FreeBSD.org> Update several more URLs

- Primarily http -> https
- Primarily FreeBSD project URLs
onf/GENERIC
onf/GENERIC-NODEBUG
onf/GENERIC-UP
9d60c6e2a4e874e3515c1383c1ccf12fb3f3d287 25-Oct-2017 manu <manu@FreeBSD.org> loader.efi: Make framebuffer commands available for arm64

Move framebuffer.{c,h} to sys/boot/efi/loader and add the efifb
related metadata and pass it to the kernel

Reviewed by: imp, andrew
Differential Revision: https://reviews.freebsd.org/D12757
nclude/metadata.h
1588800df4fe5fcdcbdef98316d8a043cc76e42f 23-Oct-2017 markj <markj@FreeBSD.org> Fix the VM_NRESERVLEVEL == 0 build.

Add VM_NRESERVLEVEL guards in the pmaps that implement transparent
superpage promotion using reservations.

Reviewed by: alc, kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D12764
rm64/pmap.c
be76b77ce71dd93ffadc4cb8ba57ce51a5550cfa 21-Oct-2017 mmel <mmel@FreeBSD.org> Add AT_HWCAP2 ELF auxiliary vector.
- allocate value for new AT_HWCAP2 auxiliary vector on all platforms.
- expand 'struct sysentvec' by new 'u_long *sv_hwcap2', in exactly
same way as for AT_HWCAP.

MFC after: 1 month
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D12699
nclude/elf.h
48b19927575e4d39a1369850d7c348c1f6cd8833 20-Oct-2017 bz <bz@FreeBSD.org> With r181803 on 2008-08-17 23:27:27Z the first VIMAGE commit went into
HEAD. Enable VIMAGE in GENERIC kernels and some others (where GENERIC does
not exist) on HEAD.

Disable building LINT-VIMAGE with VIMAGE being default.

This should give it a lot more exposure in the run-up to 12 to help
us evaluate whether to keep it on by default or not.
We are also hoping to get better performance testing.
The feature can be disabled using nooptions.

Requested by: many
Reviewed by: kristof, emaste, hiren
X-MFC after: never
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D12639
onf/GENERIC
c87fb69add95476d27620455d3aeece665434fe0 18-Oct-2017 markj <markj@FreeBSD.org> Move kernel dump offset tracking into MI code.

All of the kernel dump implementations keep track of the current offset
("dumplo") within the dump device. However, except for textdumps, they
all write the dump sequentially, so we can reduce code duplication by
having the MI code keep track of the current offset. The new
dump_append() API can be used to write at the current offset.

This is needed to implement support for kernel dump compression in the
MI kernel dump code.

Also simplify dump_encrypted_write() somewhat: use dump_write() instead
of duplicating its bounds checks, and get rid of the redundant offset
tracking.

Reviewed by: cem
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D11722
rm64/minidump_machdep.c
75b9c152228bc1fecf0bb6a5c559861988f38bf1 10-Oct-2017 andrew <andrew@FreeBSD.org> Support the EFI Runtime Services on arm64. As with amd64 we use the 1:1
mapping. This uses the new common code shared with amd64.

The RTC should only be accessed via EFI. There is no locking around it as
the spec only has this as a requirement for the PC-AT CMOS device.

Reviewed by: kib, imp
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12595
rm64/efirt_machdep.c
rm64/machdep.c
nclude/efi.h
abba1345bdfb02ff470078fa8f327c7532e0ac24 10-Oct-2017 andrew <andrew@FreeBSD.org> Add machine/fpu.h on arm64 for compatibility with amd64.

Sponsored by: DARPA, AFRL
nclude/fpu.h
d4a40bb2ce1b49bd231c0ffadaf095ea9bfbd531 10-Oct-2017 andrew <andrew@FreeBSD.org> Move the pmap_l0_index, etc. macros to pte.h. These will be used by the
EFI Runtime Services code.

Sponsored by: DARPA, AFRL
rm64/pmap.c
nclude/pte.h
57fa69eee6769e5e70233dc623478b48de7b07c9 02-Oct-2017 andrew <andrew@FreeBSD.org> Add a memory barrier to ensure the atomic write is visible to the other
CPUs before waking them up.

Sponsored by: DARPA, AFRL
rm64/mp_machdep.c
b35131985ba34d195fcd9e25a16a979fff5c628d 19-Sep-2017 jpaetzel <jpaetzel@FreeBSD.org> Fix indentation for r323068

PR: 220170
Reported by: lidl
MFC after: 3 days
Pointyhat to: jpaetzel
onf/GENERIC
334d8dd8528b0a1fc6bbd1c8df32b588cda95eba 15-Sep-2017 andrew <andrew@FreeBSD.org> Add the ARMv8.3 ID register fields. These were found in the A-Profile
exploration tools documentation:
https://developer.arm.com/products/architecture/a-profile/exploration-tools

Sponsored by: DARPA, AFRL
rm64/identcpu.c
nclude/armreg.h
54f410cc0e69aa4c8a34119ad7bb3e1427bb1a3c 14-Sep-2017 andrew <andrew@FreeBSD.org> Add support for handling undefined instructions in userspace and the
kernel. We can register callbacks to perform the required operation on the
saved registers before returning.

This is initially used to work around a bug in old versions of QEMU that
trigger such an exception when reading from an ID register when it should
load z zero value.

I expect this could be used with other exception types, e.g. to emulate
special register access from userland.

Sponsored by: DARPA, AFRL
rm64/identcpu.c
rm64/machdep.c
rm64/trap.c
rm64/undefined.c
nclude/undefined.h
e5ea82a50dd64a3e47767b132a16281242ff396d 14-Sep-2017 jhb <jhb@FreeBSD.org> Add AT_HWCAP and AT_EHDRFLAGS on all platforms.

A new 'u_long *sv_hwcap' field is added to 'struct sysentvec'. A
process ABI can set this field to point to a value holding a mask of
architecture-specific CPU feature flags. If an ABI does not wish to
supply AT_HWCAP to processes the field can be left as NULL.

The support code for AT_EHDRFLAGS was already present on all systems,
just the #define was not present. This is a step towards unifying the
AT_* constants across platforms.

Reviewed by: kib
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D12290
nclude/elf.h
a4e7c08871db8f27e90fc7bcc024459fe87a016f 11-Sep-2017 kibab <kibab@FreeBSD.org> Add MMCCAM-enabled kernel config for arm64

Approved by: imp (mentor)
Differential Revision: https://reviews.freebsd.org/D12114
onf/GENERIC-MMCCAM
87549eced1c8628115195446758b8be887b82594 11-Sep-2017 mw <mw@FreeBSD.org> Expand Marvell NIC description in arm64 GENERIC config

Suggested by: emaste
onf/GENERIC
9397c586784f35d825d1702bfcd0c633fe33208b 09-Sep-2017 mw <mw@FreeBSD.org> Enable compilation of Marvell NETA controller with arm64 GENERIC

This patch enables network operation on Marvell Armada 3700 SoC.

Submitted by: Patryk Duda <pdk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Semihalf
Differential Revision: https://reviews.freebsd.org/D12259
onf/GENERIC
365ae18aeb0d43d3d61ef8f5fb009eb054f1a35a 09-Sep-2017 mw <mw@FreeBSD.org> Add support for uart_mvebu driver arm64 GENERIC config

This patch enables console output on Armada 3700 SoCs
with kernel GENERIC.

Submitted by: Patryk Duda <pdk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Semihalf
Differential Revision: https://reviews.freebsd.org/D12251
onf/GENERIC
2aeaa86b170e1e50c637b5ad0ffa39c1472f274d 09-Sep-2017 mw <mw@FreeBSD.org> Enable compilation of Marvell EHCI driver in arm64 GENERIC

Enabled driver can be used on boards equipped with Marvell Armada 3700 SoC.

Submitted by: Patryk Duda <pdk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Semihalf
Differential Revision: https://reviews.freebsd.org/D12256
onf/GENERIC
f86c87df5a02f9e40b8f9d4f4d9da33ec67e07dc 09-Sep-2017 mw <mw@FreeBSD.org> Enable compilation of Marvell XHCI driver in arm64 GENERIC

Enabled driver can be used on boards equipped with Marvell Armada
3700/7k/8k SoCs.

Submitted by: Patryk Duda <pdk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Semihalf
Differential Revision: https://reviews.freebsd.org/D12253
onf/GENERIC
62b179d480932e885c889927c770f5a7ab2c3543 08-Sep-2017 andrew <andrew@FreeBSD.org> Not all CPUs handle reading ID_AA64MMFR2_EL1 (e.g. qemu), disable it for now.

Sponsored by: DARPA, AFRL
rm64/identcpu.c
03b729348a9586a4c79c9a52be2f64312af51f81 07-Sep-2017 andrew <andrew@FreeBSD.org> Fix the SVE ID field shift.

Sponsored by: DARPA, AFRL
nclude/armreg.h
62ba352441c48cb31da6b6bb4cb0e9c50d67d48a 07-Sep-2017 andrew <andrew@FreeBSD.org> Add the ATS1E1 case to the ID_AA64MMFR1_EL1 decoding.

Sponsored by: DARPA, AFRL
rm64/identcpu.c
cf3f211b57cd04cbf5d38b383d2d5776fade7047 07-Sep-2017 andrew <andrew@FreeBSD.org> Use the correct mask when printing undecoded fields from the
ID_AA64MMFR2_EL1 register.

Sponsored by: DARPA, AFRL
rm64/identcpu.c
2f25fd16e248fc7d2cba1291018af899618beed2 07-Sep-2017 andrew <andrew@FreeBSD.org> Fix a mismerge, make sure PRINT_ID_AA64_MMFR2 has a unique value.

Sponsored by: DARPA, AFRL
rm64/identcpu.c
e8eeadef690924b8c5546a6e00c30be275509e4e 07-Sep-2017 andrew <andrew@FreeBSD.org> Fix the value of ID_AA64ISAR1_DPB_SHIFT, the field is bits 3:0.

Sponsored by: DARPA, AFRL
nclude/armreg.h
ba989b6cad6eb8e0f446f397382c2a4ac2af111b 07-Sep-2017 andrew <andrew@FreeBSD.org> Add the ARMv8.2 ID register additions and use them to decode the register
values. As not all assemblers understand the new ID_AA64MMFR2_EL1 register
add a macro to access it. This seems to be safe for older CPUs to read this
new register, with them returning zero.

Sponsored by: DARPA, AFRL
rm64/identcpu.c
nclude/armreg.h
fd9059068ea0bda8c1c969732b1e22a59d435387 07-Sep-2017 andrew <andrew@FreeBSD.org> Uppercase the special register names in identcpu to be more consistent with
the other source files.

Sponsored by: DARPA, AFRL
rm64/identcpu.c
baa1d9fb58928dc1ab952f628139497f3610122f 07-Sep-2017 andrew <andrew@FreeBSD.org> Make the bit mask of ARMv8 ID registers to print sparse to keep values
close, but without having to change all values when new registers are added.

Sponsored by: DARPA, AFRL
rm64/identcpu.c
9ddc4a94bbc2965f454ea70695357eb4503dc7e2 07-Sep-2017 andrew <andrew@FreeBSD.org> Add more ARM Ltd parts to the list of knows CPUs.

Submitted by: Jon Brawn <jon@brawn.org>
rm64/identcpu.c
nclude/cpu.h
89cb1eb62a406bea42c394d330632198ec8feda0 05-Sep-2017 mw <mw@FreeBSD.org> Add Marvell RTC driver to arm64 GENERIC config

Marvell Armada 80x0/70x0 SoC family uses same RTC IP as
Armada 38x. This patch adds necessary files and enable driver in
GENERIC config.

Submitted by: Rafal Kozik <rk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Semihalf
Differential Revision: https://reviews.freebsd.org/D12200
onf/GENERIC
abba46522dded8e75673a9957ffebee8324eba3a 03-Sep-2017 mw <mw@FreeBSD.org> Add ARM Cortex A72 to CPU list

This change is required to properly detect CPUs
on Marvell Armada 80x0/70x0 SoC family.

Submitted by: Rafal Kozik <rk@semihalf.com>
Reviewed by: andrew, cognet (mentor)
Approved by: cognet (mentor)
Sponsored by: Semihalf
Differential Revision: https://reviews.freebsd.org/D12184
rm64/identcpu.c
nclude/cpu.h
bfd734f77c3410b0bfb5970372d9dcc75d5e8ddf 01-Sep-2017 jpaetzel <jpaetzel@FreeBSD.org> Revert r323087

This needs more thinking out and consensus, and the commit message
was wrong AND there was a typo in the commit.

pointyhat: jpaetzel
onf/GENERIC
612bb8539d6e9032f6751f629d3ba3c49534b9c9 01-Sep-2017 jpaetzel <jpaetzel@FreeBSD.org> Take options IPSEC out of GENERIC

PR: 220170
Submitted by: delphij
Reviewed by: ae, glebius
MFC after: 2 weeks
Differential Revision: D11806
onf/GENERIC
11ae0dae99729853f31680d38227132a2e6f33a4 31-Aug-2017 andrew <andrew@FreeBSD.org> Add support for quirks while enabling secondary CPUs. This uses the fdt
compatible string to check if the board is compatible with a given quirk.
It's possible this will be moved later, however as it's currently only used
by the MP code put it there.

So far the only instance of a quirk is when the list of CPUs may be
incorrect. This can happen on virtual machines with a hard coded
devicetree, but where the user may then set the number of CPUs as an
argument. This is the case on the ARM models so include the model specific
compat strings for these, including the spelling mistake found in some of
the OpenplatformPkg dtb files.

Sponsored by: DARPA, AFRL
rm64/mp_machdep.c
f7739d7e092d8732c6f89f4b3e8df007d620552c 31-Aug-2017 jpaetzel <jpaetzel@FreeBSD.org> Allow kldload tcpmd5

PR: 220170
MFC after: 2 weeks
onf/GENERIC
c96ec2ef5e7f716a77f2bd62d7252d16a5269b84 22-Aug-2017 andrew <andrew@FreeBSD.org> Remove an unneeded call to pmap_invalidate_all. This was never called as
the anyvalid variable is never set.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/pmap.c
9d83961b969676dd685d97f591bed3810fb7114e 22-Aug-2017 andrew <andrew@FreeBSD.org> Fix a bug in pmap_protect where we invalidate the wrong page. With this we
can now remove an unneeded call to invalidate all entries.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/pmap.c
fa34d7ce0d2235183a4c56330df4c192130d5693 22-Aug-2017 andrew <andrew@FreeBSD.org> Fix a comment on uncommitted work.
rm64/swtch.S
e9fee38ac473f09c4bd86d762a0a5a56fb88c7e5 22-Aug-2017 andrew <andrew@FreeBSD.org> Move the l0 pagetable address to struct mdproc. It is a property of the
whole process so should live there.

Sponsored by: DARPA, AFRL
rm64/genassym.c
rm64/pmap.c
rm64/swtch.S
rm64/vm_machdep.c
nclude/pcb.h
nclude/proc.h
10b736858eafc6bc86c21e1c35f0ee70de2ef02e 21-Aug-2017 andrew <andrew@FreeBSD.org> Improve the performance of the arm64 thread switching code.

The full system memory barrier around a TLB invalidation is stricter than
required. It needs to wait on accesses to main memory, with just the weaker
store variant before the invalidate. As such use the dsb istst, tlbi, dlb
ish sequence already used in pmap.

The tlbi instruction in this sequence is also unnecessarily using a
broadcast invalidate when it just needs to invalidate the local CPUs TLB.
Switch to a non-broadcast variant of this instruction.

Sponsored by: DARPA, AFRL
rm64/swtch.S
ce8e2801bf4f14b02a871155fcd99634c92f8849 18-Aug-2017 markj <markj@FreeBSD.org> Rename mkdumpheader() and group EKCD functions in kern_shutdown.c.

This helps simplify the code in kern_shutdown.c and reduces the number
of globally visible functions.

No functional change intended.

Reviewed by: cem, def
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D11603
rm64/minidump_machdep.c
f6dd3eb2237cb95d14471d2cd61ff5d509b9882d 18-Aug-2017 markj <markj@FreeBSD.org> Factor out duplicated kernel dump code into dump_{start,finish}().

dump_start() and dump_finish() are responsible for writing kernel dump
headers, optionally writing the key when encryption is enabled, and
initializing the initial offset into the dump device.

Also remove the unused dump_pad(), and make some functions static now that
they're only called from kern_shutdown.c.

No functional change intended.

Reviewed by: cem, def
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D11584
rm64/minidump_machdep.c
0fbf153cd03c3ffdae09cdbd2f40a172235f9552 17-Aug-2017 emaste <emaste@FreeBSD.org> arm64: return error instead of panic in unimplemented ptrace ops

We don't need a panic as a reminder that these need to be implemented.

Reported by: Shawn Webb
MFC after: 3 week
Sponsored by: The FreeBSD Foundation
rm64/machdep.c
1d4eae65c8e1733423a332095a4c88faa266cba6 13-Aug-2017 andrew <andrew@FreeBSD.org> Add support for multiple GICv3 ITS devices. For this we add sc_irq_base
and sc_irq_length to the softc to handle the base number of IRQs available,
make gicv3_get_nirqs return the number of available interrupt IDs, and
limit which CPUs we send interrupts to based on the numa domain.

The last point is only strictly needed on a dual socket ThunderX where we
are unable to send MSI/MSI-X interrupts between sockets.

Sponsored by: DARPA, AFRL
rm64/gic_v3.c
rm64/gic_v3_fdt.c
rm64/gicv3_its.c
43e4187c13cd64f17276ed7ded07772e9f04f3a9 12-Aug-2017 jhb <jhb@FreeBSD.org> Reliably enable debug exceptions on all CPUs.

Previously, debug exceptions were only enabled on the boot CPU if
DDB was enabled in the dbg_monitor_init() function. APs also called
this function, but since mp_machdep.c doesn't include opt_ddb.h, the
APs ended up calling an empty stub defined in <machine/debug_monitor.h>
instead of the real function. Also, if DDB was not enabled in the kernel,
the boot CPU would not enable debug exceptions.

Fix this by adding a new dbg_init() function that always clears the OS
lock to enable debug exceptions which the boot CPU and the APs call.
This function also calls dbg_monitor_init() to enable hardware breakpoints
from DDB on all CPUs if DDB is enabled. Eventually base support for
hardware breakpoints/watchpoints will need to move out of the DDB-only
debug_monitor.c for use by userland debuggers.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D12001
rm64/debug_monitor.c
rm64/machdep.c
rm64/mp_machdep.c
nclude/machdep.h
6e837ed7a2e9b658a29fa34f642c81d83411b70f 12-Aug-2017 jhb <jhb@FreeBSD.org> Don't panic for PT_GETFPREGS.

Only fetch the VFP state from the CPU if the thread whose registers are
being requested is the current thread. If a stopped thread's registers
are being fetched by a debugger, the saved state in the PCB is already
valid.

Reviewed by: andrew
MFC after: 1 week
rm64/machdep.c
1cab2038a47360cf934a53b71e44ee99dbeadcd9 11-Aug-2017 jhb <jhb@FreeBSD.org> Fix a typo.
rm64/vfp.c
ba47a8c8f3b45f5c79c25f852f6d1051cff2d36c 05-Aug-2017 andrew <andrew@FreeBSD.org> Mark each cpu in the appropriate cpuset_domain set. This allows devices to
handle cases where they can only run on a single domain.

To allow all devices access to this set we need to move reading the domain
earlier in the boot as it was previously handled in the CPU driver, however
this is too late for the GICv3 ITS driver.

Sponsored by: DARPA, AFRL
rm64/mp_machdep.c
nclude/param.h
00495d218afc0d05833e0560164de2f119473ee4 04-Aug-2017 andrew <andrew@FreeBSD.org> Start to teach the GICv3 driver about NUMA. On ThunderX we may have
multiple ITS devices, however we only want a single ITS device to be
configured on each CPU. To fix this only enable ITS when the node matches
the CPUs node.

Sponsored by: DARPA, AFRL
rm64/gic_v3.c
rm64/gic_v3_fdt.c
rm64/gic_v3_var.h
rm64/gicv3_its.c
0e7986f6d845c34aa65320295b7e7c4feb4887ef 04-Aug-2017 andrew <andrew@FreeBSD.org> Read the numa-node-id property from each CPU node. This will initially be
used to support the dual package ThunderX where we need to send MSI/MSI-X
interrupts to the same package as the device the interrupt came from.

Sponsored by: DARPA, AFRL
rm64/mp_machdep.c
c319a803c2fb690526a12f902837483f033bf8b9 02-Aug-2017 manu <manu@FreeBSD.org> arm64: Add Allwinner H5 SoC

Allwinner H5 is an H3 (arm32) with Cortex A53 cores.
Add support for it and enable it in GENERIC kernel config

Tested on: OrangePi PC2
onf/GENERIC
2b18a5ba483d49a2ca6dfc962f1fb0b9e5c5d953 26-Jul-2017 andrew <andrew@FreeBSD.org> Pass the last exception trap frame to kdb_trap. This allows show registers
in ddb to show the traps registers, and not the registers from within the
panic call.

Sponsored by: DARPA, AFRL
rm64/trap.c
44e79a530d7220917bc28bd454a6f149b7b662d8 18-Jul-2017 andrew <andrew@FreeBSD.org> Add support for passing FPU_KERN_NOCTX to fpu_kern_enter on arm64. This
will be used to call into UEFI from the kernel.

Sponsored by: DARPA, AFRL
rm64/vfp.c
nclude/pcb.h
nclude/vfp.h
e09217d3ae0224afde70d8673cd285eb9c818b3f 09-Jul-2017 manu <manu@FreeBSD.org> arm64: Add Allwinner I2C controller to GENERIC kernel
onf/GENERIC
c1dd7e88f806da752265a3e2fe358a273c3f82a2 02-Jul-2017 andrew <andrew@FreeBSD.org> In the arm64 pmap_remove, when removing a full superpage there is no need
to demote it to 512 pages, then remove each of these. We can just remove
the l2 map directly. This is what the intel pmaps already do.

Sponsored by: DARPA, AFRL
rm64/pmap.c
40cc52c95344bba6b3753d2e59ce5c4f56e1941b 01-Jul-2017 andrew <andrew@FreeBSD.org> Remove all calls to cpu_dcache_wb_range from the arm64 pmap code. These
were unneeded as we tell the tlb the pagetables are in cached memory. This
gives us a small, but statistically significant improvement over just
removing the PTE_SYNC cases.

While here remove PTE_SYNC, it's now unneeded.

Sponsored by: DARPA, AFRL
rm64/pmap.c
d1caaa930005d8e83736591419549f62a1596d76 01-Jul-2017 jah <jah@FreeBSD.org> Clean up MD pollution of bus_dma.h:

--Remove special-case handling of sparc64 bus_dmamap* functions.
Replace with a more generic mechanism that allows MD busdma
implementations to generate inline mapping functions by
defining WANT_INLINE_DMAMAP in <machine/bus_dma.h>. This
is currently useful for sparc64, x86, and arm64, which all
implement non-load dmamap operations as simple wrappers
around map objects which may be bus- or device-specific.

--Remove NULL-checked bus_dmamap macros. Implement the
equivalent NULL checks in the inlined x86 implementation.
For non-x86 platforms, these checks are a minor pessimization
as those platforms do not currently allow NULL maps. NULL
maps were originally allowed on arm64, which appears to have
been the motivation behind adding arm[64]-specific barriers
to bus_dma.h, but that support was removed in r299463.

--Simplify the internal interface used by the bus_dmamap_load*
variants and move it to bus_dma_internal.h

--Fix some drivers that directly include sys/bus_dma.h
despite the recommendations of bus_dma(9)

Reviewed by: kib (previous revision), marius
Differential Revision: https://reviews.freebsd.org/D10729
rm64/busdma_machdep.c
nclude/bus_dma.h
nclude/bus_dma_impl.h
013aea6ae4398ff82e44b632d94ee3b77da76ed2 30-Jun-2017 andrew <andrew@FreeBSD.org> Remove a blank line accidentally added in r320403.
nclude/atomic.h
f03abdb06189ce1ee07b2ce8df2320f07143df2f 27-Jun-2017 andrew <andrew@FreeBSD.org> Add parentheses missed in r320388

Sponsored by: DARPA, AFRL
nclude/endian.h
20ce0f47dfdcdc105526135ac69e1d2a351627d2 27-Jun-2017 andrew <andrew@FreeBSD.org> Some of the atomic_clear_* functions were incorrectly defined to be an
atomic add. Correct these, fixing a NULL-pointer dereference in netgraph.

PR: 220273
MFC after: 3 days
Sponsored by: DARPA, AFRL
nclude/atomic.h
4947a513987ebdece228984ac44919f5b13f5d7d 26-Jun-2017 andrew <andrew@FreeBSD.org> In _bswap16 and _bswap32 cast constant values to the appropriate type. This is
similar to what is done in the x86 code.

Sponsored by: DARPA, AFRL
nclude/endian.h
cc79d0fbd57f623d019c65d4e4d99ff4ffd1bbc4 25-Jun-2017 andrew <andrew@FreeBSD.org> Stop calling cpu_dcache_wb_range from PTE_SYNC.

We set the shareability attributes in TCR_EL1 on boot. These tell the
hardware the pagetables are in cached memory so there is no need to flush
the entries from the cache to memory.

This has about 4.2% improvement in system time and 2.7% improvement in
user time for a buildkernel -j48 on a ThunderX.

Keep the old code for now to allow for further comparisons.
rm64/pmap.c
e2a14c603f20af5eaf11dfafcd181d6d82269dd7 12-Jun-2017 kib <kib@FreeBSD.org> Move struct syscall_args syscall arguments parameters container into
struct thread.

For all architectures, the syscall trap handlers have to allocate the
structure on the stack. The structure takes 88 bytes on 64bit arches
which is not negligible. Also, it cannot be easily found by other
code, which e.g. caused duplication of some members of the structure
to struct thread already. The change removes td_dbg_sc_code and
td_dbg_sc_nargs which were directly copied from syscall_args.

The structure is put into the copied on fork part of the struct thread
to make the syscall arguments information correct in the child after
fork.

This move will also allow several more uses shortly.

Reviewed by: jhb (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
X-Differential revision: https://reviews.freebsd.org/D11080
rm64/trap.c
loudabi64/cloudabi64_sysvec.c
7b6fe97487582283defc708e1db9515d6ca341da 12-Jun-2017 kib <kib@FreeBSD.org> Make struct syscall_args visible to userspace compilation environment
from machine/proc.h, consistently on all architectures.

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
X-Differential revision: https://reviews.freebsd.org/D11080
nclude/proc.h
4fce328caea831040429f0720004a85bb0d43bb2 09-Jun-2017 andrew <andrew@FreeBSD.org> Allow the arm64 machine/vfp.h to be included without first including
machine/pcb.h. It he latter is only needed for struct pcb.
nclude/vfp.h
7f9edbc6077d6f259ed8ef1b7a63b8450acf4b9f 09-Jun-2017 andrew <andrew@FreeBSD.org> Store the read-only thread pointer when scheduling a new thread. This is
not currently set, however we may wish to set it later.
rm64/genassym.c
rm64/swtch.S
nclude/pcb.h
2fe3ed022a964060b589bd17a02d23260238bf8a 02-Jun-2017 cognet <cognet@FreeBSD.org> - Don't bother flushing the data cache for pages we're about to unmap, there's
no need to.
- Remove pmap_is_current(), pmap_[pte|l3]_valid_cacheable as there were only
used to know if we had to write back pages.
- In pmap_remove_pages(), don't bother invalidating each page in the TLB,
we're about to flush the whole TLB anyway.

This makes make world 8-9% faster on my hardware.

Reviewed by: andrew
rm64/pmap.c
66ff0fc9c001ae124b45557dce1f8c91e358205c 23-May-2017 kib <kib@FreeBSD.org> Add COMPAT_FREEBSD11 on arm64, the arch is almost tier-1.

Discussed with: andrew, emaste
Sponsored by: The FreeBSD Foundation
onf/GENERIC
107bf6208595837e71e0018f742ca260bebd6984 16-May-2017 hselasky <hselasky@FreeBSD.org> Avoid use of contiguous memory allocations in busdma when possible.

This patch improves the boundary checks in busdma to allow more cases
using the regular page based kernel memory allocator. Especially in
the case of having a non-zero boundary in the parent DMA tag. For
example AMD64 based platforms set the PCI DMA tag boundary to
PCI_DMA_BOUNDARY, 4GB, which before this patch caused contiguous
memory allocations to be preferred when allocating more than PAGE_SIZE
bytes. Even if the required alignment was less than PAGE_SIZE bytes.

This patch also fixes the nsegments check for using kmem_alloc_attr()
when the maximum segment size is less than PAGE_SIZE bytes.

Updated some comments describing the code in question.

Differential Revision: https://reviews.freebsd.org/D10645
Reviewed by: kib, jhb, gallatin, scottl
MFC after: 1 week
Sponsored by: Mellanox Technologies
rm64/busdma_bounce.c
e1e13d1ad13a0ad5582477432658325367ddcc0e 13-May-2017 andrew <andrew@FreeBSD.org> Allocate a cacheline when reading or writing to write through memory. The
hardware will still write to memory, however following reads will be from
the cache.

MFC after: 1 week
Sponsored by: DARPA, AFRL
nclude/armreg.h
bf70d7ed98d582a978cf03eed8ec37eafe393da9 13-May-2017 andrew <andrew@FreeBSD.org> Add the VM_MEMATTR_WRITE_THROUGH memory type to arm64 and use it to support
VM_MEMATTR_WRITE_COMBINING in the kernel. This fixes a bug where Xorg would
use write back cached memory for its graphics buffers. This would produce
artifacts on the screen as cachelines were written to memory.

MFC after: 1 week
Sponsored by: DARPA, AFRL
nclude/vm.h
7a5edbcf70144666ddbbd33f6890fe2ad265ff09 05-May-2017 andrew <andrew@FreeBSD.org> Add reclaim_pv_chunk on arm64. This is based on the amd64 code so should
operate similarly, other than not needing the delayed invalidation.

It has been tested with artificial injection of vm_page_alloc failures
while running 'sort /dev/zero'.

Reviewed by: alc, kib
MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D10574
rm64/pmap.c
17b89157e4c519959f15865b494ac15ce1b61c87 03-May-2017 andrew <andrew@FreeBSD.org> Print all virtual addresses in the show vtop ddb command. The results may
be different with PAN enabled.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/machdep.c
de3e0757f9bf5f4eb92f154f63a9cc9016ccea54 24-Apr-2017 andrew <andrew@FreeBSD.org> Call the PSCI reset from cpu_reset on arm64. When rebooting from DDB the
kernel calls this directly so the event handler is not called, meaning
the computer fails to reboot.

Tested by: cognet
MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/vm_machdep.c
acc61195fcd1646aa95bb1f60827144112485ec7 20-Apr-2017 andrew <andrew@FreeBSD.org> Restrict the arm64 supervisor all instructions to only allow a zero
immediate value for system calls. We may wish to use other values in the
future for other purposes.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/trap.c
d8ea52fd1f3d5338a1e765becd3cea266ac3bfaa 20-Apr-2017 andrew <andrew@FreeBSD.org> Push loading curthread into assembly in the synchronous exception handlers.
This will help investigating the performance impact of moving parts of the
switch statement in do_el0_sync into assembly.

Sponsored by: DARPA, AFRL
rm64/exception.S
rm64/genassym.c
rm64/trap.c
77758299a31c141f2ecbacb154a124618afd1931 17-Apr-2017 emaste <emaste@FreeBSD.org> Remove trailing whitespace from r317061
nclude/counter.h
21ead51d79e6c9c2eb017eff84a3c8ae660a79b5 17-Apr-2017 glebius <glebius@FreeBSD.org> - Remove 'struct vmmeter' from 'struct pcpu', leaving only global vmmeter
in place. To do per-cpu stats, convert all fields that previously were
maintained in the vmmeters that sit in pcpus to counter(9).
- Since some vmmeter stats may be touched at very early stages of boot,
before we have set up UMA and we can do counter_u64_alloc(), provide an
early counter mechanism:
o Leave one spare uint64_t in struct pcpu, named pc_early_dummy_counter.
o Point counter(9) fields of vmmeter to pcpu[0].pc_early_dummy_counter,
so that at early stages of boot, before counters are allocated we already
point to a counter that can be safely written to.
o For sparc64 that required a whole dummy pcpu[MAXCPU] array.

Further related changes:
- Don't include vmmeter.h into pcpu.h.
- vm.stats.vm.v_swappgsout and vm.stats.vm.v_swappgsin changed to 64-bit,
to match kernel representation.
- struct vmmeter hidden under _KERNEL, and only vmstat(1) is an exclusion.

This is based on benno@'s 4-year old patch:
https://lists.freebsd.org/pipermail/freebsd-arch/2013-July/014471.html

Reviewed by: kib, gallatin, marius, lidl
Differential Revision: https://reviews.freebsd.org/D10156
nclude/counter.h
nclude/pcpu.h
57634430237527708cb45463c5630be1cb51c9e0 17-Apr-2017 glebius <glebius@FreeBSD.org> All these files need sys/vmmeter.h, but now they got it implicitly
included via sys/pcpu.h.
rm64/minidump_machdep.c
rm64/uma_machdep.c
ab6a57c4a3830429394aed71bfd8559f0cb2890e 13-Apr-2017 andrew <andrew@FreeBSD.org> Rather than checking if the top bit in a virtual address is a 0 or 1
compare against VM_MAXUSER_ADDRESS as we should have been doing.

Sponsored by: DARPA, AFRL
rm64/pmap.c
rm64/trap.c
5730f7ccdc34cf19749315e41d7240a77f5f723e 13-Apr-2017 andrew <andrew@FreeBSD.org> Set the arm64 Execute-never bits in more places.

We need to set the Execute-never bits when mapping device memory as the
hardware may perform speculative instruction fetches.

Set the Privileged Execute-ever bit on userspace memory to stop the kernel
if it is tricked into executing it.

Reviewed by: kib
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D10382
rm64/pmap.c
de2c97b5fd57ebe24bed81892d779a96488efb42 13-Apr-2017 kan <kan@FreeBSD.org> Use proper fields to check for interrupt trigger mode.
rm64/gic_v3.c
d73bb06e63891e9a9ceb0dfddaded039a7424948 13-Apr-2017 andrew <andrew@FreeBSD.org> In ARMv8.1 ARM has added a process state bit to disable access to userspace
from the kernel. Make use of this to restrict accessing userspace to just
the functions that explicitly handle crossing the user kernel boundary.

Reported by: kib
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D10371
rm64/copyinout.S
rm64/machdep.c
rm64/mp_machdep.c
rm64/support.S
nclude/armreg.h
nclude/asm.h
nclude/cpufunc.h
a19f569aadd741316cd76ec1766d63054d65793f 13-Apr-2017 andrew <andrew@FreeBSD.org> Add SCTLR bits added in ARMv8.1 and ARMv8.2 and start to use them in the
early boot code.

Sponsored by: DARPA, AFRL
rm64/locore.S
nclude/armreg.h
e4d185768e49323a6fa2340af697fa1ff2970da9 12-Apr-2017 andrew <andrew@FreeBSD.org> Start to use the User and Privileged execute-never bits in the arm64
pagetables. This sets both bits when entering an address we know shouldn't
be executed.

I expect we could mark all userspace pages as Privileged execute-never to
ensure the kernel doesn't branch to one of these addresses.

While here add the ARMv8.1 upper attributes.

Reviewed by: alc, kib (previous version)
MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D10360
rm64/pmap.c
nclude/pte.h
e10e5b06891d7643a24e08129449536c0166699b 12-Apr-2017 andrew <andrew@FreeBSD.org> Use the unprivileged variant of the load and store instructions most
places possible in the kernel. This forces these functions to fail if
userspace is unable to access a given memory location, even if it is in
the user memory range.

This will simplify adding Privileged Access Never support later.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/copyinout.S
rm64/support.S
26b779b185c633429fa938e24f03eaa7e94c45bd 10-Apr-2017 kib <kib@FreeBSD.org> Do not lose dirty bits for removing PROT_WRITE on arm64.

Arm64 pmap interprets accessed writable ptes as modified, since
ARMv8.0 does not track Dirty Bit Modifier in hardware. If writable bit
is removed, page must be marked as dirty for MI VM.

This change is most important for COW, where fork caused losing
content of the dirty pages which were not yet scanned by pagedaemon.

Reviewed by: alc, andrew
Reported and tested by: Mark Millard <markmi@dsl-only.net>
PR: 217138, 217239
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
rm64/pmap.c
33064e92a2a5dad758daf4c8d3d41075aa6cf0f0 09-Apr-2017 pkelsey <pkelsey@FreeBSD.org> Corrected misspelled versions of rendezvous.

The MFC will include a compat definition of smp_no_rendevous_barrier()
that calls smp_no_rendezvous_barrier().

Reviewed by: gnn, kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D10313
nclude/counter.h
139ec191bba953edabbe947c2368b969bcccea81 07-Apr-2017 kan <kan@FreeBSD.org> Define 'lr' as x30 on aarch64

GNU toolchain does not recognize LR as standard register alias,
but clang does. Use of #define will work on both. Place the
definition into central machine/asm.h instead of patching every
affected file, as requested by plaftorm maintainers.

Reviews by: andrew, emaste, imp
Differential Revision: https://reviews.freebsd.org/D10307
nclude/asm.h
254458ab34d09446aa706c54b434ef77d387e200 26-Mar-2017 bde <bde@FreeBSD.org> Fix printing of negative offsets (typically from frame pointers) again.
I fixed this in 1997, but the fix was over-engineered and fragile and
was broken in 2003 if not before. i386 parameters were copied to 8
other arches verbatim, mostly after they stopped working on i386, and
mostly without the large comment saying how the values were chosen on
i386. powerpc has a non-verbatim copy which just changes the uncritical
parameter and seems to add a sign extension bug to it.

Just treat negative offsets as offsets if they are no more negative than
-db_offset_max (default -64K), and remove all the broken parameters.

-64K is not very negative, but it is enough for frame and stack pointer
offsets since kernel stacks are small.

The over-engineering was mainly to go more negative than -64K for the
negative offset format, without affecting printing for more than a
single address.

Addresses in the top 64K of a (full 32-bit or 64-bit) address space
are now printed less well, but there aren't many interesting ones.
For arches that have many interesting ones very near the top (e.g.,
68k has interrupt vectors there), there would be no good limit for
the negative offset format and -64K is a good as anything.
nclude/db_machdep.h
6b8dbe92e829ba458a1efbc9d8a64b4ae5315f45 24-Mar-2017 imp <imp@FreeBSD.org> Add 'device iic' to bring in userland I2C driver.

Submitted by: karl@
onf/GENERIC
63254ceea69dc619384a4e6fff3c7f86708b45a4 23-Mar-2017 ed <ed@FreeBSD.org> Stop providing the compat_3_brand.

As of r315860, the ELF image activator works fine for CloudABI without it.

Reviewed by: kib
MFC after: 2 weeks
loudabi64/cloudabi64_sysvec.c
da63ef1f6051592816b187682e10634a1b930b58 22-Mar-2017 kib <kib@FreeBSD.org> Update r315753 with the proper flag name.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
loudabi64/cloudabi64_sysvec.c
a22b5a31351362d7b7d11669184173d73e29aa38 22-Mar-2017 kib <kib@FreeBSD.org> Add a flag BI_BRAND_ONLY_STATIC to specify that the brand only
matches static binaries.

Interpretation of the 'static' there is that the binary must not
specify an interpreter. In particular, shared objects are matched by
the brand if BI_CAN_EXEC_DYN is also set.

This improves precision of the brand matching, which should eliminate
surprises due to brand ordering.

Revert r315701.

Discussed with and tested by: ed (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
loudabi64/cloudabi64_sysvec.c
fc95dfd2e518d81b639d093c276000df89bc45dc 22-Mar-2017 ed <ed@FreeBSD.org> Set the interpreter path to /nonexistent.

CloudABI executables are statically linked and don't have an
interpreter. Setting the interpreter path to NULL used to work
previously, but r314851 introduced code that checks the string
unconditionally. Running CloudABI executables now causes a null pointer
dereference.

Looking at the rest of imgact_elf.c, it seems various other codepaths
already leaned on the fact that the interpreter path is set. Let's just
go ahead and pick an obviously incorrect interpreter path to appease
imgact_elf.c.

MFC after: 1 week
loudabi64/cloudabi64_sysvec.c
d4532eaf623461492cbb712df29453daf69dc853 16-Mar-2017 andrew <andrew@FreeBSD.org> If ofw_bus_msimap fails don't try to use the invalid MSI/MSI-X parent node.

Sponsored by: ABT Systems Ltd
avium/thunder_pcie_pem_fdt.c
a03d613777c9af84784d162b97daa96d6484220d 15-Feb-2017 andrew <andrew@FreeBSD.org> Load the new sp_el0 with interrupts disabled in fork_trampoline. If an
interrupt arrives in fork_trampoline after sp_el0 was written we may then
switch to a new thread, enter userland so change this stack pointer, then
return to this code with the wrong value. This fixes this case by moving
the load of sp_el0 until after interrupts have been disabled.

Reported by: Mark Millard (markmi@dsl-only.net)
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D9593
rm64/swtch.S
33bc722d7756f692b0a189035211e0daf0d2b902 15-Feb-2017 andrew <andrew@FreeBSD.org> Port the Linux AMX 10G network driver to FreeBSD as axgbe. It is unlikely
we will import a newer version of the Linux code so the linuxkpi was not
used.

This is still missing 10G support, and multicast has not been tested.

Reviewed by: gnn
Obtained from: ABT Systems Ltd
Sponsored by: SoftIron Inc
Differential Revision: https://reviews.freebsd.org/D8549
onf/GENERIC
d431cd542b4503ced11f6239f233d0f2d3835378 07-Feb-2017 andrew <andrew@FreeBSD.org> Push reading of ESR_EL1 to assembly. Among other uses this will allow us
to expose this to signal handlers, e.g. for the clang sanitizers.

Sponsored by: DARPA, AFRL
rm64/exception.S
rm64/genassym.c
rm64/trap.c
nclude/frame.h
3484d055de92c0b686ed28fdfa59278e4e8f28d1 06-Feb-2017 andrew <andrew@FreeBSD.org> Remove arm64_tlb_flushID_SE, it's unused and may be wrong.

Sponsored by: ABT Systems Ltd
rm64/cpufunc_asm.S
nclude/cpufunc.h
2318a8473378d1af39e373df53f07b13fdb8f32f 06-Feb-2017 kib <kib@FreeBSD.org> Update arm and arm64 counters MD bits.

On arm64 use atomics. Then, both arm and arm64 do not need a critical
section around update. Replace all cpus loop by CPU_FOREACH().
This brings arm and arm64 counter(9) implementation closer to current
amd64, but being more RISC-y, arm* version cannot avoid atomics.

Reported by: Alexandre Martins <alexandre.martins@stormshield.eu>
Reviewed by: andrew
Tested by: Alexandre Martins, andrew
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
nclude/counter.h
c24073c855975ad6e695d782796f830a47a41243 04-Feb-2017 kib <kib@FreeBSD.org> Define the vm_ooffset_t and vm_pindex_t types as machine-independend.

The types are for the byte offset and page index in vm object. They
are similar to off_t, which is defined as 64bit MI integer. Using MI
definitions will allow to provide consistent MD values of vm
object-related maximum sizes.

Reviewed by: alc
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
nclude/_types.h
6a21064647735c7a243182d7e258d3126e06024d 28-Jan-2017 cognet <cognet@FreeBSD.org> Implement atomic_fcmpset_* for arm and arm64.
nclude/atomic.h
d0e035a60f3cdfa0bbacab6defbea1eeec673359 23-Jan-2017 bz <bz@FreeBSD.org> Remove a static function declaration for a function not implemented.

Makes head code compile on 10.3 and cleanup is never wrong.

MFC after: 3 days
rm64/mp_machdep.c
be72efbdd4153ffe6f66ccec839342a79d0e89d2 17-Jan-2017 ed <ed@FreeBSD.org> Catch up with changes to structure member names.

Pointer/length pairs are now always named ${name} and ${name}_len.
loudabi64/cloudabi64_sysvec.c
efab05d61248138a71288118da40303e97092b2a 10-Jan-2017 sbruno <sbruno@FreeBSD.org> Migrate e1000 to the IFLIB framework:
- em(4) igb(4) and lem(4)
- deprecate the igb device from kernel configurations
- create a symbolic link in /boot/kernel from if_em.ko to if_igb.ko

Devices tested:
- 82574L
- I218-LM
- 82546GB
- 82579LM
- I350
- I217

Please report problems to freebsd-net@freebsd.org

Partial review from jhb and suggestions on how to *not* brick folks who
originally would have lost their igbX device.

Submitted by: mmacy@nextbsd.org
MFC after: 2 weeks
Relnotes: yes
Sponsored by: Limelight Networks and Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D8299
onf/GENERIC
66a7c34b9e91c7cb4378e486a2d610338a341d3f 18-Dec-2016 jchandra <jchandra@FreeBSD.org> Add virtio_pci to GENERIC arm64 conf

virtio_pci was missing from the GENERIC arm64 configuration, while
other virtio devices are present. Adding it will allow us to boot
the GENERIC kernel on QEMU with virtio storage and networking.
onf/GENERIC
a38cab8fcb7c494b67fbd0095402200ec87df0af 18-Dec-2016 jchandra <jchandra@FreeBSD.org> Initialize GIC[DR]_IGROUPRn registers for GICv3

In case where GICD_CTLR.DS is 1, the IGROUPR registers are RW in
non-secure state and has to be initialized to 1 for the
corresponding interrupts to be delivered as Group 1 interrupts.

Update gic_v3_dist_init() and gic_v3_redist_init() to initialize
GICD_IGROUPRn and GICR_IGROUPRn respectively to address this. The
registers can be set unconditionally since the writes are ignored
in non-secure state when GICD_CTLR.DS is 0.

This fixes the hang on boot seen when running qemu-system-aarch64
with machine virt,gic-version=3
rm64/gic_v3.c
rm64/gic_v3_reg.h
3b55e84275a89982bb77adcab0487f875722db0c 12-Dec-2016 andrew <andrew@FreeBSD.org> Enable ACPI on arm64. It's not yet functional, but it will help keeping the
code building until it is ready.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
onf/GENERIC
f63c437216e0309e4a319c2c95a2f8ca061c0bca 10-Dec-2016 def <def@FreeBSD.org> Add support for encrypted kernel crash dumps.

Changes include modifications in kernel crash dump routines, dumpon(8) and
savecore(8). A new tool called decryptcore(8) was added.

A new DIOCSKERNELDUMP I/O control was added to send a kernel crash dump
configuration in the diocskerneldump_arg structure to the kernel.
The old DIOCSKERNELDUMP I/O control was renamed to DIOCSKERNELDUMP_FREEBSD11 for
backward ABI compatibility.

dumpon(8) generates an one-time random symmetric key and encrypts it using
an RSA public key in capability mode. Currently only AES-256-CBC is supported
but EKCD was designed to implement support for other algorithms in the future.
The public key is chosen using the -k flag. The dumpon rc(8) script can do this
automatically during startup using the dumppubkey rc.conf(5) variable. Once the
keys are calculated dumpon sends them to the kernel via DIOCSKERNELDUMP I/O
control.

When the kernel receives the DIOCSKERNELDUMP I/O control it generates a random
IV and sets up the key schedule for the specified algorithm. Each time the
kernel tries to write a crash dump to the dump device, the IV is replaced by
a SHA-256 hash of the previous value. This is intended to make a possible
differential cryptanalysis harder since it is possible to write multiple crash
dumps without reboot by repeating the following commands:
# sysctl debug.kdb.enter=1
db> call doadump(0)
db> continue
# savecore

A kernel dump key consists of an algorithm identifier, an IV and an encrypted
symmetric key. The kernel dump key size is included in a kernel dump header.
The size is an unsigned 32-bit integer and it is aligned to a block size.
The header structure has 512 bytes to match the block size so it was required to
make a panic string 4 bytes shorter to add a new field to the header structure.
If the kernel dump key size in the header is nonzero it is assumed that the
kernel dump key is placed after the first header on the dump device and the core
dump is encrypted.

Separate functions were implemented to write the kernel dump header and the
kernel dump key as they need to be unencrypted. The dump_write function encrypts
data if the kernel was compiled with the EKCD option. Encrypted kernel textdumps
are not supported due to the way they are constructed which makes it impossible
to use the CBC mode for encryption. It should be also noted that textdumps don't
contain sensitive data by design as a user decides what information should be
dumped.

savecore(8) writes the kernel dump key to a key.# file if its size in the header
is nonzero. # is the number of the current core dump.

decryptcore(8) decrypts the core dump using a private RSA key and the kernel
dump key. This is performed by a child process in capability mode.
If the decryption was not successful the parent process removes a partially
decrypted core dump.

Description on how to encrypt crash dumps was added to the decryptcore(8),
dumpon(8), rc.conf(5) and savecore(8) manual pages.

EKCD was tested on amd64 using bhyve and i386, mipsel and sparc64 using QEMU.
The feature still has to be tested on arm and arm64 as it wasn't possible to run
FreeBSD due to the problems with QEMU emulation and lack of hardware.

Designed by: def, pjd
Reviewed by: cem, oshogbo, pjd
Partial review: delphij, emaste, jhb, kib
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4712
rm64/minidump_machdep.c
6218b7c9eda9e8dc24c7c2243e4109782e6a2b1a 10-Dec-2016 gnn <gnn@FreeBSD.org> This adds a configuration for arm64 users that track CURRENT but
don't need the extra debug facilities. Copied from the amd64
configuration of the same name.

Submitted by: Nikolai Lifanov
Reviewed by: emaste
MFC after: 2 weeks
onf/GENERIC-NODEBUG
7571ef95c158150722cc67d8220e25f5a6fef65d 08-Dec-2016 alc <alc@FreeBSD.org> Previously, vm_radix_remove() would panic if the radix trie didn't
contain a vm_page_t at the specified index. However, with this
change, vm_radix_remove() no longer panics. Instead, it returns NULL
if there is no vm_page_t at the specified index. Otherwise, it
returns the vm_page_t. The motivation for this change is that it
simplifies the use of radix tries in the amd64, arm64, and i386 pmap
implementations. Instead of performing a lookup before every remove,
the pmap can simply perform the remove.

Reviewed by: kib, markj
Differential Revision: https://reviews.freebsd.org/D8708
rm64/pmap.c
03d27c7f998a7b844995c327a2fa4ec66cdfd5ee 07-Dec-2016 andrew <andrew@FreeBSD.org> Add ACPI support to the arm64 mp code. We use the Multiple APIC Description
Table to find the CPUs to find the CPUs to start. Currently we assume PSCI,
however this assumption is shared with the FDT code.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
3eceb86920012f6405f38b0f7e9361f0a09e06f7 06-Dec-2016 andrew <andrew@FreeBSD.org> Add the missing gic_common.h include for GIC_BUS_FDT.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic_v3_fdt.c
579121cbb19a9ab6c2b17cda79be2809c3e462c2 06-Dec-2016 andrew <andrew@FreeBSD.org> Create two GIC ivars to find the bus type and GIC hardware version. These
will be used by the gicv2m and ITS ACPI drivers to only attach to the
correct parent.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic_v3.c
rm64/gic_v3_fdt.c
rm64/gic_v3_reg.h
rm64/gic_v3_var.h
ab1758c433baa800995e4cc163bc3543c8932752 06-Dec-2016 andrew <andrew@FreeBSD.org> Move the common bit manipulation macros from the GICv3 header to the
common GIC header file.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic_v3.c
rm64/gic_v3_reg.h
rm64/gicv3_its.c
f612a55546af57de5ded3fa199887186c7f3a334 06-Dec-2016 andrew <andrew@FreeBSD.org> Move the stored signal mask later in the jump buf. It was being clobbered
by a later store of a VFP register.

Sponsored by: ABT Systems Ltd
nclude/setjmp.h
626d27c314bbfba76d75cb3db60da6a5df14bf2e 06-Dec-2016 andrew <andrew@FreeBSD.org> Add the full set of GICD_PIDR* registers

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic_v3_reg.h
d06930414eb6bf79afd94782baaecd9886c8b6af 05-Dec-2016 andrew <andrew@FreeBSD.org> Remove an unneeded header. It was used when we had an arm64 specific GIC
driver, however with INTRNG this has been merged into a common arm driver.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic.h
73431e62ab73e6c0b23910651239326b5316089c 30-Nov-2016 andrew <andrew@FreeBSD.org> Fix the alloc function the ThunderX PCIe driver calls, the previous
function may not exist when FDT is removed from the kernel.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
avium/thunder_pcie_common.c
ea32464c7dce9ce5e890eeb54c05a508895cce76 29-Nov-2016 andrew <andrew@FreeBSD.org> We only use the cpu0 variable in the FDT code.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
963d0b0ea3e0aa0666b8ee56a5515fe3bcf301e2 28-Nov-2016 andrew <andrew@FreeBSD.org> Fix ISS_DATA_DFSC_MASK, it should include all bits in the field.
nclude/armreg.h
6585173eb9b6afe22b3794d4ccc5b88e96532648 25-Nov-2016 andrew <andrew@FreeBSD.org> Fix the TLB conflict abort value. This should be a no-op as we don't use
this value in the code.
nclude/armreg.h
5da5f04a5432390586a02fd9535cb18fc1fa3a89 24-Nov-2016 gonzo <gonzo@FreeBSD.org> [rpi3] Move SOC_BRCM_BCM2837 from UP config to SMP one

Now that BCM283x source are buildable with SMP option it cam be moved to
GENERIC SMP config. SMP itself does not work on RPi3 yet due to lack of
PSCI monitor which is work in progress at the moment
onf/GENERIC
onf/GENERIC-UP
f748600d8f654836dd6a7258b85fca79c23575fb 23-Nov-2016 andrew <andrew@FreeBSD.org> Mark cpu_find_cpu0_fdt as FDT only. It's only called when this is set, and
the kernel is using FDT.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
a922c709e8b9a8a58de5deb3d5464a8740ac149d 23-Nov-2016 andrew <andrew@FreeBSD.org> Remove the unneeded ofw_cpu_reg function signature, it's not used in this
file.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
233931aa3effa0542aab759a8a033c75feed852d 22-Nov-2016 andrew <andrew@FreeBSD.org> Add the missing DEVMETHOD_ENDs to the arm64 nexus ACPI and FDT drivers.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/nexus.c
d477fa31613f85a7718e71875228788941852082 21-Nov-2016 andrew <andrew@FreeBSD.org> Add support to find the arm64 serial using the ACPI tables. This uses the
Serial Port Console Redirection Table to find the device to use.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
cpica/acpi_machdep.c
nclude/acpica_machdep.h
083cb45b1a1a5e5913f700f5e69d13d49406c91e 21-Nov-2016 andrew <andrew@FreeBSD.org> To allow for an ACPI attachment to the generic PCIe driver split off the
FDT attachment to a new file. A separate ACPI attachment will then be added
to allow arm64 servers with ACPI to use it over FDT.

This should also help with merging this with the ofwpci driver, with
further work needed to remove restrictions this driver places on resource
allocation.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D7319
avium/thunder_pcie_common.c
avium/thunder_pcie_fdt.c
fdf9ecfbb65333aa81a7b0f69bafa3156517357d 16-Nov-2016 andrew <andrew@FreeBSD.org> Include the SMSC LAN91C111 driver, this is found in some of the ARM models.
onf/GENERIC
30f99dbeefc844112679b1d34c674426abfb24c8 08-Nov-2016 bdrewery <bdrewery@FreeBSD.org> Fix improper use of "its".

Sponsored by: Dell EMC Isilon
rm64/identcpu.c
bbef42a78d6309315a492fdd244029fc399f72cf 26-Oct-2016 andrew <andrew@FreeBSD.org> Pull the common FDT interrupt values into a new header rather than be magic
numbers.

Sponsored by: DARPA, AFRL
rm64/gic_v3.c
c825d2846d979e85a81c19015a84c692725fb71c 26-Oct-2016 andrew <andrew@FreeBSD.org> Only release CPUs when they exist.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/mp_machdep.c
c8fd0bbfccad0b0396fc6d44214e3cccc77ad74d 26-Oct-2016 andrew <andrew@FreeBSD.org> Use nitems to get the correct number of registers to read when dumping
them. Previously this would walk past the end of the array and print
whatever happened to be after the trapframe struct.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/trap.c
d01cc693f9f15b213eea9de936cb4382da2f7d0b 25-Oct-2016 andrew <andrew@FreeBSD.org> Create a new PSCI error code and use it to signal that starting the CPU is
impossible as the PSCI firmware is missing.

Sponsored by: ABT Systmes Ltd
rm64/mp_machdep.c
96474ac2aa6c872da3da8f0a263f9d67220f931f 25-Oct-2016 andrew <andrew@FreeBSD.org> Import the Cortex String memcpy and memmove into the kernel. On ThunderX
these show a 9-10% reduction in user and system time for a buildworld -j48.

Obtained from: ABT Systems Ltd
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
rm64/bcopy.c
rm64/memcpy.S
rm64/memmove.S
26ea90652063b25c7e56c917fbcf41575e53c577 24-Oct-2016 andrew <andrew@FreeBSD.org> Increase CACHE_LINE_SHIFT to 7 as cache lines are 128 bytes on ThunderX.

MFC after: 1 week
Sponsored by: ABT Systems Ltd
nclude/param.h
ec78fa7f7f607585b32b07a87891dcf041ce5c57 20-Oct-2016 wma <wma@FreeBSD.org> Driver for PCI Ethernet NIC on Alpine V1 and V2.

Obtained from: Semihalf
Submitted by: Michal Stanek <mst@semihalf.com>
Sponsored by: Annapurna Labs
Reviewed by: wma
Differential Revision: https://reviews.freebsd.org/D7814
onf/GENERIC
75768c2bf49145fc06075e9c78b24c6fe12a9cde 20-Oct-2016 andrew <andrew@FreeBSD.org> Add support for the fpu_kern(9) KPI on arm64. It hooks into the existing
VFP code to store the old context, with lazy loading of the new context
when needed.

FPU_KERN_NOCTX is missing as this is unused in the crypto code this has
been tested with, and I am unsure on the requirements of the UEFI
Runtime Services.

Reviewed by: kib
Obtained from: ABT Systeems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8276
rm64/trap.c
rm64/vfp.c
nclude/pcb.h
nclude/vfp.h
3f40ad67ccb7363c8eede53af6035baffbe0f6cb 19-Oct-2016 gonzo <gonzo@FreeBSD.org> Move SOC_BRCM_BCM2837 to GENERIC-UP since does not support SMP
onf/GENERIC
onf/GENERIC-UP
2f351dfb1095bd8a894b7353c0778c9d40e67092 18-Oct-2016 gonzo <gonzo@FreeBSD.org> Retire RPI3 kernel config in favour of GENERIC-UP

Policy for FreeBSD/arm64 kernel config is the same one as for x86
architectures: provide GENERIC kernel bootable on as many systems
as possible. Since there is no SMP support for RPi 3 yet new kernel
config was introduced: GENERIC-UP, which is effectively GENERIC with
SMP option disabled
onf/GENERIC
onf/GENERIC-UP
onf/RPI3
bd90949460fbd8702fb8075b63e48e6e0acfe791 18-Oct-2016 andrew <andrew@FreeBSD.org> Add PCB_FP_USERMASK so we can mask off floating point flags that should
not be sent to userspace, for example the future flag to tell when we are
using floating point in the kernel.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/machdep.c
rm64/trap.c
nclude/pcb.h
4a327f5e898a4ea02f591e4c6b3541139c7363ba 15-Oct-2016 andrew <andrew@FreeBSD.org> Fix the build, struct vfpstate needs to be visible to userspace as it's
part of struct pcb which is in a header used in libutil.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
nclude/vfp.h
2a35286dafe66442817e9bd50c57c664044f7517 15-Oct-2016 andrew <andrew@FreeBSD.org> Check we are in a critical section when calling vfp_discard. As we may call
it with a NULL thread pointer only check when it is non-NULL.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/vfp.c
8c172582e24f3d37137f6f8fd5262d649e9fc71e 14-Oct-2016 gonzo <gonzo@FreeBSD.org> Make BRCM2837 port conform FreeBSD/ARM64 guidelines

- Rename SOC_BCM2837 to SOC_BRCM_BCM2837, put it to opt_soc.h
- do not use files.XXX files, just move required sources to
conf/files.arm64 and make them depend on soc_brcm_bcm2837

Suggested by: andrew
roadcom/bcm2837/files.bcm2837
onf/RPI3
0058087fc060a59f0679306ef73145445c161554 14-Oct-2016 andrew <andrew@FreeBSD.org> Create macros for the MAIR memory attributes. While here add an uncached
memory type, however the VM code still needs to be taught about this.

MFC after: 1 week
Sponsored by: ABT Systems Ltd
rm64/locore.S
nclude/armreg.h
f689fd5a63a6e44d052c44504e24717c08883327 14-Oct-2016 jhb <jhb@FreeBSD.org> Drop support for using mmap() with /dev/kmem.

Using the device pager with /dev/kmem is not stable since KVA mappings
are transient, but the device pager caches the PA associated with a
given offset forever. Interestingly, mips' implementation of
memmap() already refused requests for /dev/kmem.

Note that kvm_read/kvm_write do not use mmap, but use read and write on
/dev/kmem, so this should not affect libkvm users.

Reviewed by: kib
MFC after: 2 months
rm64/mem.c
42e43cc8828775444579a1d2106a91f17d570ff6 14-Oct-2016 gonzo <gonzo@FreeBSD.org> Remove redundant kern_clocksource.c. It is already in conf/files.arm64

Reported by: andrew
roadcom/bcm2837/files.bcm2837
4d2f7af1cf1f20508da6a4999d8c4eba89e499a6 14-Oct-2016 andrew <andrew@FreeBSD.org> Rework how we store the VFP registers in the pcb. This will be used when
creating a floating-point context within the kernel without having to move
the stored values in memory.

Sponsored by: The FreeBSD Foundation
rm64/machdep.c
rm64/vfp.c
rm64/vm_machdep.c
nclude/pcb.h
nclude/vfp.h
b966860f8c84ed88fa86adf75ef4a62b5c2fe749 14-Oct-2016 gonzo <gonzo@FreeBSD.org> Add initial Raspberry Pi 3 support

RPI3 kernel config builds kernel compatible with latest upstream device
tree and firmware: https://github.com/raspberrypi/firmware/tree/master/boot
As of today it's 597c662a613df1144a6bc43e5f4505d83bd748ca

Default console is PL01x, so pi3-disable-bt dt overlay should be configured
in config.txt and stock U-Boot should be patched to use proper serial port.

Yet unsupported: SMP, VCHIQ, RNG driver. RNG requires some work due to
upstream device tree incompatibility.

Multiple people contributed to this work over time: db@, loos@, manu@
roadcom/bcm2837/files.bcm2837
onf/RPI3
75e242c27e430ad45d44e9d44a3d2f2465f884e4 13-Oct-2016 andrew <andrew@FreeBSD.org> Move printing the AArch64 ID registers to a new SYSINIT, the previous
location only prints them when booting on SMP with multiple cores.

MFC after: 1 week
Sponsored by: DARPA, AFRL
rm64/identcpu.c
rm64/mp_machdep.c
081e8d8587ccaea4d9c72ee4225dd3f1647ecff5 13-Oct-2016 imp <imp@FreeBSD.org> Fix building on i386 and arm. But 'public domain' headers on the files
with no creative content. Include "lost" changes from git:
o Use /dev/efi instead of /dev/efidev
o Remove redundant NULL checks.

Submitted by: kib@, dim@, zbb@, emaste@
nclude/efi.h
043456d8a0c5c0e510ffb28f2976da091b069cd0 12-Oct-2016 gonzo <gonzo@FreeBSD.org> INTRNG: Propagate IRQ activation error to API consumer

Keep resource state consistent with INTRNG state - if intr_activate_irq
fails - deactivate resource and propagate error to calling function

Reviewed by: mmel
rm64/nexus.c
62030781cdc239b37135b2ce6041c2df3d933595 12-Oct-2016 jtl <jtl@FreeBSD.org> In the TCP stack, the hhook(9) framework provides hooks for kernel modules
to add actions that run when a TCP frame is sent or received on a TCP
session in the ESTABLISHED state. In the base tree, this functionality is
only used for the h_ertt module, which is used by the cc_cdg, cc_chd, cc_hd,
and cc_vegas congestion control modules.

Presently, we incur overhead to check for hooks each time a TCP frame is
sent or received on an ESTABLISHED TCP session.

This change adds a new compile-time option (TCP_HHOOK) to determine whether
to include the hhook(9) framework for TCP. To retain backwards
compatibility, I added the TCP_HHOOK option to every configuration file that
already defined "options INET". (Therefore, this patch introduces no
functional change. In order to see a functional difference, you need to
compile a custom kernel without the TCP_HHOOK option.) This change will
allow users to easily exclude this functionality from their kernel, should
they wish to do so.

Note that any users who use a custom kernel configuration and use one of the
congestion control modules listed above will need to add the TCP_HHOOK
option to their kernel configuration.

Reviewed by: rrs, lstewart, hiren (previous version), sjg (makefiles only)
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D8185
onf/GENERIC
7706f244fc67accb0fc06ca956d87224042215c2 04-Oct-2016 jkim <jkim@FreeBSD.org> Merge ACPICA 20160930.
0b0178b3a680d2d00fd267ff015b3e1bf0baaa0d 21-Sep-2016 kib <kib@FreeBSD.org> Add a way for the architecture to specify the calling ABI for methods
in the EFI Runtime Services Table. On amd64, the calling conventions
are MS.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
nclude/efi.h
bfcbca20d87058dea8bfb9088225c05cbcf73504 21-Sep-2016 wma <wma@FreeBSD.org> Add support for SPI-mapped MSI interrupts in GICv3.

PIC_SETUP_INTR implementation in GICv3 did not allow
for setting up interrupts without included FDT
description. GICv2m-like MSI interrupts, which map
MSI messages to SPI interrupt lines, may not have
a description in FDT. Add support for such interrupts
by setting the trigger and polarity to the appropriate
values for MSI (edge, high) and get the hardware
IRQ number from the corresponding ISRC.

Obtained from: Semihalf
Submitted by: Michal Stanek <mst@semihalf.com>
Sponsored by: Annapurna Labs
Reviewed by: wma
Differential Revision: https://reviews.freebsd.org/D7662
rm64/gic_v3.c
3edc270a7dfedcb5fcaec80c1483839f48c8d6c5 21-Sep-2016 wma <wma@FreeBSD.org> Register GICv3 xref.

This allows other drivers to retrieve interrupt parent node.

Obtained from: Semihalf
Submitted by: Michal Stanek <mst@semihalf.com>
Sponsored by: Annapurna Labs
Reviewed by: wma, zbb
Differential Revision: https://reviews.freebsd.org/D7568
rm64/gic_v3_fdt.c
e53ee72b9aab672bb20c0abe7535ef17042e64b0 20-Sep-2016 wma <wma@FreeBSD.org> Add driver for PCIe root complex on Annapurna Alpine platform.

The driver subclasses pci-host-generic and additionally
performs configuration of vendor-specific PCIe registers.

Obtained from: Semihalf
Submitted by: Michal Stanek <mst@semihalf.com>
Sponsored by: Annapurna Labs
Reviewed by: wma
Differential Revision: https://reviews.freebsd.org/D7571
onf/GENERIC
90ee5f51f280af9c6565ddb764bd933e16b6f387 18-Sep-2016 kib <kib@FreeBSD.org> Consolidate four efi_next_descriptor() definitions.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week
rm64/machdep.c
15d64966130a3257b1d792e84f93dd3a77ee7442 13-Sep-2016 andrew <andrew@FreeBSD.org> Add a missing opt_ddb.h include from places that include debug_moditor.h
to fix the build.

Obtained from: ABT Systems Ltd
X-MFC with: r305771
Sponsored by: The FreeBSD Foundation
rm64/db_trace.c
rm64/debug_monitor.c
4a70bd772c5f4d31c5cf26e15bd87c1e76441a96 13-Sep-2016 andrew <andrew@FreeBSD.org> Fix the arm64 kernel build when DDB is disabled, debug_monitor.c depends on
DDB, and is unused when it's disabled.

Obtained from: ABT Systems Ltd
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
nclude/debug_monitor.h
6a0e9d7faa375786f96ed97924d72786b9ef2fbd 13-Sep-2016 andrew <andrew@FreeBSD.org> Add a warning about a known erratum we have observed on ThunderX pass 1.1.
As this is evaluation hardware with only a few users, and there is a lack
of information add a warning when booting on this hardware.

Reported by: cognet
Obtained from: ABT Systems Ltd
MFC after: Instant
Sponsored by: The FreeBSD Foundation
rm64/identcpu.c
44f29780e8a789103b731d1fbf53cf772cd2c5f2 10-Sep-2016 alc <alc@FreeBSD.org> Various changes to pmap_ts_referenced()

Move PMAP_TS_REFERENCED_MAX out of the various pmap implementations and
into vm/pmap.h, and describe what its purpose is. Eliminate the archaic
"XXX" comment about its value. I don't believe that its exact value, e.g.,
5 versus 6, matters.

Update the arm64 and riscv pmap implementations of pmap_ts_referenced()
to opportunistically update the page's dirty field.

On amd64, use the PDE value already cached in a local variable rather than
dereferencing a pointer again and again.

Reviewed by: kib, markj
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D7836
rm64/pmap.c
41d6813369789b4df8a3676498a6715f4fa2613f 08-Sep-2016 andrew <andrew@FreeBSD.org> Trap msr/mrs instructions. These are privileged arm64 instructions and
shouldn't normally be used.

Obtained from: ABT Systems Ltd
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
rm64/trap.c
471921aaf50574046b251fed7b9d2c86154b2856 08-Sep-2016 andrew <andrew@FreeBSD.org> Don't panic when we don't handle a userland exception, not all we may see
are currently handled.

Obtained from: ABT Systems Ltd
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
rm64/trap.c
a66a58c9636e8e99aee3d7ccdd41c6151763ac72 07-Sep-2016 andrew <andrew@FreeBSD.org> When synchronising the instruction and data caches we only need to clean
the data cache to the point of unification. This is the point where the
two caches are unified to a single unified cache so cleaning past here
is just extra unneeded work.

This was noticed when investigating r305545.

Reported by: bz
Obtained from: ABT Systems Ltd
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
rm64/cpufunc_asm.S
4bde709fd323534eebeca6c08ea421b14e729d65 07-Sep-2016 andrew <andrew@FreeBSD.org> Only call cpu_icache_sync_range when inserting an executable page. If the
page is non-executable the contents of the i-cache are unimportant so this
call is just adding unneeded overhead when inserting pages.

While doing research using gem5 with an O3 pipeline and 1k/32k/1M iTLB/L1
iCache/L2 Bjoern Zeeb (bz@) observed a fairly high rate of calls into
arm64_icache_sync_range() from pmap_enter() along with a high number of
instruction fetches and iTLB/iCache hits.

Limiting the calls to arm64_icache_sync_range() to only executable pages,
we observe the iTLB and iCache Hit going down by about 43%. These numbers
are quite misleading when looked at alone as at the same time instructions
retired were reduced by 19.2% and instruction fetches were reduced by 38.8%.
Overall this reduced the runtime of the test program by 22.4%.

On Juno hardware, in steady-state, running the same test, using the cycle
count to determine runtime, we do see a reduction of up to 28.9% in runtime.

While these numbers certainly depend on the program executed, we expect an
overall performance improvement.

Reported by: bz
Obtained from: ABT Systems Ltd
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
0da791731a7e963ebaceeadeddd552c398b76e1a 07-Sep-2016 wma <wma@FreeBSD.org> Introduce support for Annapurna Alpine CCU and NB devices

This commit adds drivers for Alpine Cache Coherency Unit
and North Bridge Service whose task is to configure
the system fabric and enable cache coherency.

Obtained from: Semihalf
Submitted by: Michal Stanek <mst@semihalf.com>
Sponsored by: Annapurna Labs
Reviewed by: wma
Differential Revision: https://reviews.freebsd.org/D7565
onf/GENERIC
fe422ee664c9f70c09c77d5bae26a736c77ae7a7 06-Sep-2016 jmcneill <jmcneill@FreeBSD.org> Add generic device-tree cpufreq driver.
onf/GENERIC
9b7353a5667fdef03080854e9b37b9aac961f272 04-Sep-2016 andrew <andrew@FreeBSD.org> Enable superpages on arm64 by default. These seem to be stable, having
survived multiple world and kernel builds, and of poudriere building full
package sets.

I have observed a 3% reduction in buildworld times with superpages enabled,
however further testing is needed to see if this is observed in other
workloads.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
fb5804c98d42ad4c49e051a7f4e99de93dfb2b51 03-Sep-2016 markj <markj@FreeBSD.org> Remove support for idle page zeroing.

Idle page zeroing has been disabled by default on all architectures since
r170816 and has some bugs that make it seemingly unusable. Specifically,
the idle-priority pagezero thread exacerbates contention for the free page
lock, and yields the CPU without releasing it in non-preemptive kernels. The
pagezero thread also does not behave correctly when superpage reservations
are enabled: its target is a function of v_free_count, which includes
reserved-but-free pages, but it is only able to zero pages belonging to the
physical memory allocator.

Reviewed by: alc, imp, kib
Differential Revision: https://reviews.freebsd.org/D7714
rm64/pmap.c
e25bfb37e510ce3bad66e400ceda0e671b2a1363 03-Sep-2016 jmcneill <jmcneill@FreeBSD.org> Add support for Allwinner A64 thermal sensors.
onf/GENERIC
d31d50139668a885a31713b8c2d01840623ca806 02-Sep-2016 andrew <andrew@FreeBSD.org> Add a pc_clock pcpu field and use it to implement cpu_est_clockrate. This
will allow drivers that manage the clock frequency to communicate this with
the reset of the kernel.

Reported by: jmcneill
MFC after: 1 week
Sponsored by: ABT Systems Ltd
rm64/machdep.c
nclude/pcpu.h
afff8d4f96cd21f0d4739aeaf1572c7b700c5b4f 01-Sep-2016 andrew <andrew@FreeBSD.org> Fix arm64 superpages bugs in pmap_enter:
* Pass the correct virtual address when demoting a superpage
* Use the correct l3 table after demoting a superpage
* Remove an invalid KASSERT hit demoting then promoting a superpage [1]

With this it is believed that superpages on arm64 is stable.

Reported by: [1] cognet
Obtained from: ABT Systems Ltd
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
2d05ee6e7e0622a1d59ede33bb37423248bffeb0 31-Aug-2016 andrew <andrew@FreeBSD.org> Also handle instruction traps. We might hit these when the page we are
executing is being promoted to a superpage.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/trap.c
098b8c899e4995d0d888359cf3e377c6065ee582 30-Aug-2016 andrew <andrew@FreeBSD.org> Because we need to use a break-before-make sequence when promoting pages
there is a short period where functions that walk the kernel page table
without locking them may see an invalid entry. One solution would be to add
locking to these functions, however some may be called from locations where
we are unable to sleep.

Until a better solution can be found stop promoting pages in the kernel
pmap so these functions work as expected.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
2a1daa1aaa0a17d6abfe3c165b3fdad3121da2ad 27-Aug-2016 andrew <andrew@FreeBSD.org> Print both the kernel read and write translation in DDB when asking for
a virtual to physical translation. These may be different, e.g. when a
page is mapped as read-only.

MFC after: 1 month
Sponsored by: ABT Systems Ltd
rm64/machdep.c
dad1fd77352af6fd8e7e7d0b84e1271577de2ec5 25-Aug-2016 andrew <andrew@FreeBSD.org> Fix an assert, it should check if, when moving from 1 l1 to 512 l2 blocks,
the l2 entry is a block type and not an l3 page.

While here fix the string to correct the level name and add a missing ')'.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
71e464bcf5b466112799a9f6638cc2c0d71b7055 25-Aug-2016 jmcneill <jmcneill@FreeBSD.org> Add support for Allwinner A64.

Reviewed by: andrew, manu
Relnotes: yes
onf/GENERIC
ee05666725d63513808d3d9a1ef09905864f78f0 25-Aug-2016 andrew <andrew@FreeBSD.org> Map coherent memory in a non-coherent dma tag as uncached. This is similar
to what the 32-bit arm code does, with the exception that it always assumes
the tag is non-coherent.

Tested by: jmcneill
Obtained from: ABT Systems Ltd
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
rm64/busdma_bounce.c
34d23267ebe9fefc362be3f80023dc651755185c 24-Aug-2016 andrew <andrew@FreeBSD.org> Allow superpages on arm64 to be enabled. It is disabled for now to allow
for more testing across more hardware.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
8f1e7a3f6c4550adcbfa2873ee96b559a67f1e29 24-Aug-2016 andrew <andrew@FreeBSD.org> Add support to promote and demote managed superpages. This is the last part
needed before enabling superpages on arm64. This code is based on the amd64
pmap with changes as needed to handle the differences between the two
architectures.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
83c9cebd9acf33b702f415c64f3401b21de7074b 24-Aug-2016 andrew <andrew@FreeBSD.org> Teach the parts of the arm64 pmap that need to iterate over pages to also
iterate over superpages. We don't yet create these, but soon will.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
nclude/pmap.h
c0aa6fd20953d3b4f27aa23fc270a2ccf7cdac8c 24-Aug-2016 ed <ed@FreeBSD.org> Convert pointers obtained from the threadattr_t structure with TO_PTR().

In all of these source files, the userspace pointer size corresponds
with the kernelspace pointer size, meaning that casting directly works.
As I'm planning on making 32-bit execution on 64-bit systems work as
well, use TO_PTR() here as well, so that the changes between source
files remain minimal.
loudabi64/cloudabi64_sysvec.c
0da2d65bcde921c21496126aa4c6899789d635e5 23-Aug-2016 andrew <andrew@FreeBSD.org> Also adjust the virtual address passed to vm_page_pa_tryrelock.

Reported by: alc
Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
7118e239c177736b53da39c822e2a1da68565bf1 23-Aug-2016 andrew <andrew@FreeBSD.org> Map memory as read-only in pmap_enter_quick_locked as is done in other
pmap implementations.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
1cbf59a47e5b8db0d3e93e4d078dcb01a67578a2 23-Aug-2016 andrew <andrew@FreeBSD.org> If we find we have a superpage in pmap_enter_quick_locked return without
trying to add a new level 3 page.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
55e7533c34a681915ed7f82063c61439ea41bcb8 23-Aug-2016 andrew <andrew@FreeBSD.org> Include the offset the virtual address is within an L1 or L2 block when
finding the vm_page_t in pmap_extract_and_hold. Previously it would return
the vm_page_t of the first page in a block. This would cause issues when,
for example, fsck reads from a device into the middle of a superpage. In
this case the read call would write to the start of the block, and not to
the buffer passed in.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
50998bac38e7dee6db6af6e340bc1e28f3c7d735 22-Aug-2016 andrew <andrew@FreeBSD.org> Fix the arm64 non-SMP build, active_irq is a uint64_t so cast it through
a uintmax_t.

Obtained from: ABT Systems Ltd
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
rm64/gic_v3.c
69b3f4f99de389a87f62062cede1d217f2ead481 22-Aug-2016 andrew <andrew@FreeBSD.org> Ensure map is valid, even before userland exists and the fault address
register points to an address in the userland range.

Obtained from: ABT Systems Ltd
MFC after: 1 week
Sponsored by: the FreeBSD Foundation
rm64/trap.c
7e316a8d58231996b6e8db8b323b0f4330853dc4 22-Aug-2016 andrew <andrew@FreeBSD.org> Fix pmap_update_entry, pmap_invalidate_range takes the end address, not
the size.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
08039971657fe3f95f779862bf367dda44128ee1 22-Aug-2016 andrew <andrew@FreeBSD.org> Use switch statements in pmap_remove_pages. While only one level of
pagetable is supported more will be added soon to support removing
superpages.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
af3020226e8f73f26b6cbfd4dc6a6f7a50e5d74d 22-Aug-2016 andrew <andrew@FreeBSD.org> Use pmap_update_entry in pmap_enter when updating an entry with a new
physical address. This is required when either mapping is writeable.

While here remove an unneeded call to pmap_pde, we already have the pde
from earlier in the function.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
78d22fef3ec6a70813d65d799e4a8f66b0b4e9b1 22-Aug-2016 andrew <andrew@FreeBSD.org> Add sysctls to report on superpages statistics. While here add extra
logging to these paths.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
ab6de4608bcc6650fbaa19be757f14d357206bd5 22-Aug-2016 andrew <andrew@FreeBSD.org> Add a size argument to pmap_update_entry.
Make use of this in pmap_promote_l2.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
23cbe89ff6381c6d1e43a468f9e91d8240781335 22-Aug-2016 andrew <andrew@FreeBSD.org> Add KASSERTS in pmap_alloc_l3 to ensure we are not encountering superpages
when we don't yet expect them;

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
28257ccca857e1627df74e13f3f3b8db0d1bec59 19-Aug-2016 mmel <mmel@FreeBSD.org> INTRNG: Rework handling with resources. Partially revert r301453.
- Read interrupt properties at bus enumeration time and store
it into global mapping table.
- At bus_activate_resource() time, given mapping entry is resolved and
connected to real interrupt source. A copy of mapping entry is attached
to given resource.
- At bus_setup_intr() time, mapping entry stored in resource is used
for delivery of requested interrupt configuration.
- For MSI/MSIX interrupts, mapping entry is created within
pci_alloc_msi()/pci_alloc_msix() call.
- For legacy PCI interrupts, mapping entry must be created within
pcib_route_interrupt() by pcib driver itself.

Reviewed by: nwhitehorn, andrew
Differential Revision: https://reviews.freebsd.org/D7493
rm64/nexus.c
e56264ca1709ba14b8c3ff6f3f8f784a8614ed25 17-Aug-2016 kib <kib@FreeBSD.org> Implement userspace gettimeofday(2) with HPET timecounter.

Right now, userspace (fast) gettimeofday(2) on x86 only works for
RDTSC. For older machines, like Core2, where RDTSC is not C2/C3
invariant, and which fall to HPET hardware, this means that the call
has both the penalty of the syscall and of the uncached hw behind the
QPI or PCIe connection to the sought bridge. Nothing can me done
against the access latency, but the syscall overhead can be removed.
System already provides mappable /dev/hpetX devices, which gives
straight access to the HPET registers page.

Add yet another algorithm to the x86 'vdso' timehands. Libc is updated
to handle both RDTSC and HPET. For HPET, the index of the hpet device
to mmap is passed from kernel to userspace, index might be changed and
libc invalidates its mapping as needed.

Remove cpu_fill_vdso_timehands() KPI, instead require that
timecounters which can be used from userspace, to provide
tc_fill_vdso_timehands{,32}() methods. Merge i386 and amd64
libc/<arch>/sys/__vdso_gettc.c into one source file in the new
libc/x86/sys location. __vdso_gettc() internal interface is changed
to move timecounter algorithm detection into the MD code.

Measurements show that RDTSC even with the syscall overhead is faster
than userspace HPET access. But still, userspace HPET is three-four
times faster than syscall HPET on several Core2 and SandyBridge
machines.

Tested by: Howard Su <howard0su@gmail.com>
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D7473
rm64/machdep.c
nclude/md_var.h
nclude/vdso.h
3987d44b33e6ac9c52e0f05a1be755225276ed40 15-Aug-2016 andrew <andrew@FreeBSD.org> Add the ARMv8.1 identification registers to the list we print when booting.

MFC after: 1 week
Sponsored by: ABT Systems Ltd
rm64/identcpu.c
nclude/armreg.h
3de878798da74f2d9b1bf7708d5c7441e79086e7 12-Aug-2016 andrew <andrew@FreeBSD.org> Implement promotions and demotions in the arm64 pmap code. For now we don't
promote memory as I am not sure all the demotion cases are handled, however
it is useful to implement pmap_page_set_memattr. This is used, for example,
when mapping uncached memory for bus_dma(9).

pmap_page_set_memattr needs to demote the DMAP region as on ARM we need to
ensure all mappings to the same physical address have the same attributes.

Reviewed by: kib
Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6987
rm64/pmap.c
rm64/trap.c
nclude/pmap.h
cc2c089a3f6ee8e49ab43bce67cee1530d34306a 10-Aug-2016 ed <ed@FreeBSD.org> Provide the CloudABI vDSO to its executables.

CloudABI executables already provide support for passing in vDSOs. This
functionality is used by the emulator for OS X to inject system call
handlers. On FreeBSD, we could use it to optimize calls to
gettimeofday(), etc.

Though I don't have any plans to optimize any system calls right now,
let's go ahead and already pass in a vDSO. This will allow us to
simplify the executables, as the traditional "syscall" shims can be
removed entirely. It also means that we gain more flexibility with
regards to adding and removing system calls.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D7438
loudabi64/cloudabi64_sysvec.c
de87d9ee50cbe860e9759ed437c54c56234a89c2 10-Aug-2016 ed <ed@FreeBSD.org> Make cpu_set_user_tls() work when called on the running thread.

On all the other architectures, this function can also be called on the
currently running thread. In this case, we shouldn't fix up the address
in the PCB, but also patch up the register itself. Otherwise it will not
become active and will simply become overwritten by the next switch.

Reviewed by: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D7437
rm64/vm_machdep.c
8085fcd61932af79b3f772f5ce09dda96ca03721 10-Aug-2016 andrew <andrew@FreeBSD.org> Uncomment the vm.kvm_size and vm.kvm_free sysctls. These work as expected so
there is no reason to leave them commented out.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
ef9b5a10db5a3864e2be60ff11000cae18f373ff 10-Aug-2016 andrew <andrew@FreeBSD.org> Implement pmap_align_superpage on arm64 based on the amd64 implementation.
This will be needed when superpage support is added.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
548a1c11c8954cc27b2f1a1c59305ecb793c5896 04-Aug-2016 andrew <andrew@FreeBSD.org> Remove the pvh_global_lock lock from the arm64 pmap. It is unneeded on arm64
as invalidation will have completed before the pmap_invalidate_* functions
have complete.

Discussed with: alc, kib
Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
852b5d78aeb33364cd414b674866c5de1ca5163f 02-Aug-2016 andrew <andrew@FreeBSD.org> Remove trailing whitespace from the arm64 pmap

Obtained from: ABT Systems Ltd
MFC after: 3 weeks
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
4ad2fb1d62d94516452316e1ba493cac357efae9 01-Aug-2016 andrew <andrew@FreeBSD.org> Add a kernel variable to let the user to select their preferred order
between ACPI and FDT. This will be needed on machines with both, e.g. the
SoftIron Overdrive 3000. The kernel will accept one or more comma separated
values of either 'acpi' or 'fdt'. Any other values are skipped.

To set it the user can either set it on the loader command line, or
in loader.conf e.g. in loader.conf:
kern.cfg.order=acpi,fdt

This will try using ACPI then FDT. If none of the selected options work the
kernel tries to use one to get the serial console, then panics.

Reviewed by: emaste (earlier version)
Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D7274
rm64/machdep.c
rm64/mp_machdep.c
rm64/nexus.c
nclude/machdep.h
6d69f3b524f4d2d8566c0528bf36ec07dd64d1c4 01-Aug-2016 andrew <andrew@FreeBSD.org> Add the fields for the PAR_EL1 register. This is used when performing an
address lookup with the AT instructions.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
nclude/armreg.h
6b8643488bfb25c0252b076680bd66d6dfd9ae02 31-Jul-2016 andrew <andrew@FreeBSD.org> Add the Data Fault Status Code values to the ESR_ELx registers for when the
fault code is a Data Abort.

Obtained from: AT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
nclude/armreg.h
0a693f4c34ecdaa7fc39dc9f6ad6996409d15e55 31-Jul-2016 andrew <andrew@FreeBSD.org> Extract the common parts of pmap_kenter_device to a new function. This will
be used when superpage support is added.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
bb62efcae4e9b0afb74bc4cafa84a180b8013f1d 31-Jul-2016 andrew <andrew@FreeBSD.org> Fix the comment above pmap_invalidate_page. tlbi will invalidate the tlb
on all CPUs.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
b354001ca974946b660275d9a8c28082fc1b5345 31-Jul-2016 andrew <andrew@FreeBSD.org> Relax the barriers around a TLB invalidation to only wait on
inner-shareable memory accesses. There is no need for full system barriers.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
4a23ba00127049a3052e0865e250c869df772326 29-Jul-2016 andrew <andrew@FreeBSD.org> Add a generic EHCI USB driver based on the Allwinner A10 driver. It is ACPI
only for now, but wouldn't be too difficult to add support for FDT.

Reviewed by: hselasky
Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D7352
onf/GENERIC
d72c4813789a6b395550d97d650c5d118a13e8dd 25-Jul-2016 andrew <andrew@FreeBSD.org> Remove an unused variable.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
707ad09b4bc946c42e56f65c34b7f5ea398317c0 25-Jul-2016 andrew <andrew@FreeBSD.org> Fix a typo in a string in a KASSERT to sanity check the CPU IDs.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
476d8ee1bc0214ea454e6b0c3a5d5bb750ea47f4 25-Jul-2016 andrew <andrew@FreeBSD.org> Rework how we number CPUs on arm64 to try and keep clusters together.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
68aea47caea067ec356ba758b14ef2fdf9ed19a6 25-Jul-2016 andrew <andrew@FreeBSD.org> Enable the generic OHCI driver on arm64

Obtained from: ABT Systems Ltd
MFC after: 1 month
Relnotes: yes
Sponsored by: The FreeBSD Foundation
onf/GENERIC
fcb5c9368b4303b7d859742f69e50da7eaccd553 24-Jul-2016 mav <mav@FreeBSD.org> Add more UEFI/e820 memory types from latest specifications.

This is only cosmetics.

MFC after: 2 weeks
rm64/machdep.c
ee8ec7fd7de0c994a10e01a60514a694a60dc105 15-Jul-2016 andrew <andrew@FreeBSD.org> Implement bus_print_child to print the resources used by the ITS driver.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/gic_v3_fdt.c
ac309831015a118a609df505ccd56123e98d95b3 14-Jul-2016 andrew <andrew@FreeBSD.org> Finish removing the non-INTRNG support from sys/arm64.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/autoconf.c
rm64/mp_machdep.c
rm64/nexus.c
nclude/intr.h
115fedf756b2efa3b91b85c962a41b8ec4dcea3c 14-Jul-2016 andrew <andrew@FreeBSD.org> Remove the non-INTRNG support from the ThunderX PCIe drivers.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
avium/thunder_pcie_pem_fdt.c
0fda24e3e685a1d2a42e8e3b3dfaf29b6c32a7c3 14-Jul-2016 andrew <andrew@FreeBSD.org> Move gic_v3_irqsrc into the GICv3 driver source as it's only needed there.
Remove unused macros from the GICv3 header.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/gic_v3.c
rm64/gic_v3_var.h
b5c878c2c5d80e9a8c2b467674439e50f1ace923 14-Jul-2016 andrew <andrew@FreeBSD.org> Move structures only used by the GICv3 ITS driver from a shared header to
the ITS driver file. There is no need for other drivers to need to know
about these structures.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/gic_v3_var.h
rm64/gicv3_its.c
8e14fcd0049512de6ce52d720679773ea492dd3d 14-Jul-2016 andrew <andrew@FreeBSD.org> Remove the non-INTRNG support from the GICv3 interrupt controller driver.
This is no longer needed.

Obtained from: ABT Systems Ltd
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
rm64/gic_v3.c
rm64/gic_v3_fdt.c
rm64/gic_v3_var.h
8c24a12208272fba70cd59f5ad895a74e7fba6c4 13-Jul-2016 andrew <andrew@FreeBSD.org> Add memmmap on arm64 so we can mmap /dev/mem and /dev/kmem.

Obtained from: ABT Systems Ltd
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
rm64/mem.c
nclude/memdev.h
23a48963e2f6ec70bd38c64598b82da17396b3c4 12-Jul-2016 ache <ache@FreeBSD.org> Undo r302601, WCHAR_MAX may not be a valid wchar value.
nclude/_types.h
97629b93ece8a3df44c741a2e64952a45d66f8b8 12-Jul-2016 ache <ache@FreeBSD.org> I don't know why unsigned int is choosed for wchar_t here, but WCHAR_MAX
should be <= WINT_MAX. It is bigger, __UINT_MAX > INT32_MAX
nclude/_types.h
887bfdc0a4b0ca83e9fe5de34ddc93dfdb60275a 08-Jul-2016 ed <ed@FreeBSD.org> Don't forget to set sa->narg for CloudABI system calls.

It turns out that this value is not used within the system call code
under normal conditions, except when using tracing tools like ktrace.
If we forget to set this value, it is set to random garbage. This may
cause ktrace to hang indefinitely, making it impossible to kill.

Reported by: Michael Plass
PR: 210800
MFC before: 11.0-RELEASE
loudabi64/cloudabi64_sysvec.c
27c3442d81b85c7249a285855300b955e73e85c8 06-Jul-2016 andrew <andrew@FreeBSD.org> Remove the old pre-INTRNG arm64 interrupt framework. GENERIC was switched
to INTRNG in r301565 with the old code no longer being built by default with
no reports of issues on any supported hardware.

Approved by: re (gjb)
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic.c
rm64/gic_acpi.c
rm64/gic_fdt.c
rm64/gic_v3_its.c
rm64/intr_machdep.c
rm64/pic_if.m
2da75cc60b6cad55da71ccdd5aa00da2a015dec6 30-Jun-2016 wma <wma@FreeBSD.org> ARM64: fix DMAP calculation

Use arithmetic operators instead of logical. This fixes
DMAP ranges calculation for ThunderX Dual Socket.

Obtained from: Semihalf
Sponsored by: Cavium
Reviewed by: zbb
Differential Revision: https://reviews.freebsd.org/D7023
Approved by: re (gjb)
nclude/vmparam.h
4c327142ba2577ba04fa3df00bb2c992d4420495 22-Jun-2016 andrew <andrew@FreeBSD.org> Fix a race when the hardware has raised an exception with an unknown or
uncategorised reason. We need to read the fault address register before
enabling interrupts as the interrupt handler may cause this register to
change.

Approved by: re (marius, kib)
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/trap.c
496a3b1f658d32d5ed000ef2ee765245a29a6c67 16-Jun-2016 kib <kib@FreeBSD.org> Update comments for the MD functions managing contexts for new
threads, to make it less confusing and using modern kernel terms.

Rename the functions to reflect current use of the functions, instead
of the historic KSE conventions:
cpu_set_fork_handler -> cpu_fork_kthread_handler (for kthreads)
cpu_set_upcall -> cpu_copy_thread (for forks)
cpu_set_upcall_kse -> cpu_set_upcall (for new threads creation)

Reviewed by: jhb (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Approved by: re (hrs)
Differential revision: https://reviews.freebsd.org/D6731
rm64/vm_machdep.c
loudabi64/cloudabi64_sysvec.c
9fcae3cab65bcdaf59557b3d7f0c578ea7d5e09d 07-Jun-2016 andrew <andrew@FreeBSD.org> Switch arm64 to use intrng by default. The old interrupt handling code can
still be used, however this is expected to be removed soon.

Obtained from: ABT Systems Ltd
Relnotes: yes
Sponsored by: The FreeBSD Foundation
onf/GENERIC
onf/GENERIC-INTRNG
612e6958e467e3c02b826846aba5b685d486c07c 05-Jun-2016 skra <skra@FreeBSD.org> INTRNG - change the way how an interrupt mapping data are provided
to the framework in OFW (FDT) case.

This is a follow-up to r301451.

Differential Revision: https://reviews.freebsd.org/D6634
rm64/gic_v3.c
rm64/nexus.c
a00165d816704b0ac2235fdeb0fce0e705654674 03-Jun-2016 andrew <andrew@FreeBSD.org> Add the GICv3 ITS intrng driver. As the interface to the interrupt
framework has significantly changed the driver has moved to a new file.
While it shares some code with the existing driver this has been modified
to work better with the intrng framework.

This has been tested on the ThunderX servers in the netperf cluster and has
been used to boot them for other testing, including DTrace and hwpmc.

With this we can use intrng on all supported arm64 platforms I was able to
test on. It is expected we will move to intrng soon, and disable the old
arm64 interrupt framework.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6437
rm64/gic_v3.c
rm64/gic_v3_fdt.c
rm64/gic_v3_var.h
rm64/gicv3_its.c
nclude/intr.h
e284c699128750fb35d923b717ff364947b1144e 01-Jun-2016 zbb <zbb@FreeBSD.org> Return real error value instead of hard-coded ENXIO (fix after r300149)

It is possible to return real error value in case of gic_v3_attach()
failure instead of hard-coded ENXIO.

Obtained from: Semihalf
Sponsored by: Cavium
rm64/gic_v3_fdt.c
9d76db4d3787237edd12f62ae8cbe5ad3f836dd8 31-May-2016 andrew <andrew@FreeBSD.org> dpcpu_init should have also passed in the calculated cpuid, not the
devicetree ID.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
05549901e3baed5dce6fb9ac7ddf211276c68e2c 31-May-2016 andrew <andrew@FreeBSD.org> Allow the kernel to boot on a CPU where the devicetree has numbered it with
a non-zero ID. To do this we increment the cpuid of any CPUs with a smaller
devicetree ID by one to stop them conflicting with the boot CPU.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
ff690eb3c3a8504be572207b54b2f6e27a98f77a 31-May-2016 andrew <andrew@FreeBSD.org> Enable setting BF_COHERENT on DMA tags. This allows the kernel to start
using the cache handling functions.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/busdma_bounce.c
83cf90e9e393da4dce653e25f3eeeba92d704fbc 31-May-2016 andrew <andrew@FreeBSD.org> Mark the ThunderX and generic PCI drivers as cache-coherent when we know
this to be the case. This will mean we don't try and handle the cache in
bus_dmamap_sync when it is not needed.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6605
avium/thunder_pcie_fdt.c
avium/thunder_pcie_pem.c
avium/thunder_pcie_pem.h
1860a225caf7329b3bc2038750624b9d0b47139a 20-May-2016 andrew <andrew@FreeBSD.org> Extract the correct bits from the GICD_TYPER register. The interrupt count
is encoded in the bottom 5 bits.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic_v3_reg.h
8dc7f1d8c74b3019a56fa6291f6076e83417ff2b 20-May-2016 andrew <andrew@FreeBSD.org> Add more useful GICv3 register definitions. While here fix
GITS_CBASER_CACHE_MASK to use the correct shift macro.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic_v3_reg.h
bddd81e0129cac97cbc5e77260b82ab67ac0b91f 20-May-2016 andrew <andrew@FreeBSD.org> Filter out BUS_DMASYNC_POSTWRITE sync operations, there is nothing for us
to do on these.

Reported by: wma
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/busdma_bounce.c
81413fa2b42fae95ce6ea4a21f8da41285c0fba4 20-May-2016 andrew <andrew@FreeBSD.org> Enable NEW_PCIB on arm64.

Obtained from: ABT Systems Ltd
Relnotes: yes
Sponsored by: The FreeBSD Foundation
onf/DEFAULTS
353b498d9da0b9258b169da73a007a346c8830af 20-May-2016 andrew <andrew@FreeBSD.org> Handle PCI_RES_BUS on the generic and ThunderX PCIe drivers. This has been
tested on the Pass 1.1 and 2.0 ThunderX machines in the Netperf cluster.

Reviewed by: jhb
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6453
avium/thunder_pcie_pem.c
659d6355141cf8fcadf9a85d41a45e69470feeb7 19-May-2016 andrew <andrew@FreeBSD.org> Define PCI_RES_BUS for NEW_PCIB

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
nclude/resource.h
5755bcaba97673b1bb0729757de00966d600513f 18-May-2016 andrew <andrew@FreeBSD.org> Return the struct intr_pic pointer from intr_pic_register. This will be
needed in later changes where we may not be able to lock the pic list lock
to perform a lookup, e.g. from within interrupt context.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic_v3_fdt.c
6fac2e64afa4d919233ab8b0aa6271dae818e86a 18-May-2016 zbb <zbb@FreeBSD.org> Add support for MSI/MSIX deallocation on GICv3-ITS

Allow to deallocate previously allocated ITS device along with
its interrupts. Interrupt numbers are being freed when the last
LPI number is no longer busy.

Reviewed by: wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D6351
rm64/gic_v3_its.c
rm64/gic_v3_var.h
083ca1a6aff022bc3199f2bccabb04aa5019127f 17-May-2016 bz <bz@FreeBSD.org> The GIC (v2 at least) has a bit in the TYPER register to indicate whether the GIC
supports the Security Extensions or not. This bit is not the same as the CPU one.
Currently we are not checking for either before trying to write to the special
registers. This can lead to problems on hardware or simulators that do not
provide the security extensions. Add the missing checks. Their interactions with
the CPU flag is not entirely clear to me but using a macro will make it easier
to quickly adjust the condition once the CPU bits are sorted as well.

Reviewed by: br
Sponsored by: DARPA/AFRL
Differential Revision: https://reviews.freebsd.org/D6397
rm64/gic.c
rm64/gic.h
b4b3af68b5d52029bca29260dc9f0a8bb34795c5 17-May-2016 andrew <andrew@FreeBSD.org> Add an arm64 kernel config to help testing intrng. It is expected this
config will only last a few weeks until we switch to this interrupt
framework.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
onf/GENERIC-INTRNG
bc26af2c81686b92e2bc96a86420ff329ad7141a 17-May-2016 andrew <andrew@FreeBSD.org> Clean up the GICv3 intrng code:
* In gic_v3_attach free the correct data on failure.
* Implement gic_v3_teardown_intr.
* Update the panic string when enabling/disabling an invalid interrupt.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic_v3.c
e7850ad5c771464440f026cb67826820204f0a71 16-May-2016 andrew <andrew@FreeBSD.org> Add intrng support to the GICv3 driver. It lacks ITS support so won't handle
MSI or MSI-X interrupts, however this is enought to boot FreeBSD under the
ARM Foundation Model with a GICv3 interrupt controller.

Approved by: ABT Systems Ltd
Relnotes: yes
Sponsored by: The FreeBSD Foundation
rm64/gic_v3.c
rm64/gic_v3_fdt.c
rm64/gic_v3_var.h
04f04889ae9e7774ef6623b73ee511e92fda05f8 16-May-2016 andrew <andrew@FreeBSD.org> Move the call to intr_pic_init_secondary to the same place as in the
non-intrng case.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
58f81b7be844cc5e8ca91c80cd6f1f402889c087 16-May-2016 andrew <andrew@FreeBSD.org> Add support for intrng to arm64. As the GICv3 drivers will need to be
updated, and until further testing can be done, this is disabled for now.

It is expected arm64 will switch to this interface, and the old interface
will be removed before 11.0 is released.

Obtained from: ABT Systems Ltd
Relnotes: yes
Sponsored by: The FreeBSD Foundation
rm64/autoconf.c
rm64/mp_machdep.c
rm64/nexus.c
nclude/intr.h
26e3729fc0da3eab13ab771cd569b951c8554c2a 16-May-2016 andrew <andrew@FreeBSD.org> Teach the ThunderX PCI PEM driver about intrng. This will be used later
when arm64 is supported by intrng.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
avium/thunder_pcie_pem_fdt.c
cb60d232484543011e4ff8ef6297e9bde00cd4b7 16-May-2016 andrew <andrew@FreeBSD.org> Add a pcib interface for use by interrupt controllers that need to
translate the pci rid to a controller ID. The translation could be based
on the 'msi-map' OFW property, a similar ACPI option, or hard-coded for
hardware lacking the above options.

Reviewed by: wma
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic_v3_its.c
rm64/gic_v3_var.h
avium/thunder_pcie_fdt.c
avium/thunder_pcie_pem.c
avium/thunder_pcie_pem_fdt.c
297d2b0fcc3a5bc7cd3668b9a0fa9e477c9df578 13-May-2016 andrew <andrew@FreeBSD.org> Add support to the arm64 busdma to handle the cache. For now this is
disabled, however when we enable it it will default to assume memory is
not cache-coherent, unless either the tag was created or the parent was
marked as cache-coherent.

Obtained from: ABT Systems Ltd
Relnotes: yes
Sponsored by: The FreeBSD Foundation
rm64/busdma_bounce.c
nclude/cpufunc.h
6d7e8f78349d22d1cb70a72433ac63fa69375f84 12-May-2016 andrew <andrew@FreeBSD.org> Rename the internal BUC_DMA_* flags to BF_* so they won't conflict with
the flags in sys/bus_dma.h.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/busdma_bounce.c
131c1c256eeabf9d92c99abc49ddcb78374e9b56 12-May-2016 andrew <andrew@FreeBSD.org> Restrict the memory barriers in bus_dmamap_sync to just the operations
where it's needed.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/busdma_bounce.c
8c3616af7a86b9e4fe851a057069a0567780fd4c 11-May-2016 andrew <andrew@FreeBSD.org> Call busdma_swi from swi_vm as is done from other architectures.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/vm_machdep.c
nclude/md_var.h
4ec608a6fec569d8e99a33edc4bd110aaef1727e 11-May-2016 andrew <andrew@FreeBSD.org> On arm64 always create a bus_dmamap_t object. This will be use to hold the
list of memory that the kernel will need to sync when operating with a
non-cache coherent DMA engine.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/busdma_bounce.c
c1c72529c87f6bd10669894317ec3b2240ec14c3 11-May-2016 andrew <andrew@FreeBSD.org> Add data barriers to the arm64 bus_dmamap_sync function. We need these
to ensure ordering between the CPU and device. As the CPU and DMA target
may be in different shareability domains they need to be full system
barriers.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/busdma_bounce.c
bc6d6fd49afd087d08a4e38e522c0068f75e8763 11-May-2016 zbb <zbb@FreeBSD.org> Fix I/O coherence issues on ThunderX when SMP is disabled

To maintain coherence between cache and DMA memory appropriate
shareability flags need to be set in the PTE regardless of SMP
option.

Reviewed by: wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D6231
nclude/pte.h
e08747edd2f2b5a853923dbf1483e6ce3bf5ab5a 10-May-2016 andrew <andrew@FreeBSD.org> Push the logic to talk with the MSI/MSI-X interrupt controller to the FDT
attachment. This is where it will live when we import intrng as it will
need to look at either the msi-parent or msi-map FDT properties.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
avium/thunder_pcie_pem.c
avium/thunder_pcie_pem_fdt.c
eb663acb54bc8677cc47448aa68f2e2393d5455b 05-May-2016 jhb <jhb@FreeBSD.org> Native PCI-express HotPlug support.

PCI-express HotPlug support is implemented via bits in the slot
registers of the PCI-express capability of the downstream port along
with an interrupt that triggers when bits in the slot status register
change.

This is implemented for FreeBSD by adding HotPlug support to the
PCI-PCI bridge driver which attaches to the virtual PCI-PCI bridges
representing downstream ports on HotPlug slots. The PCI-PCI bridge
driver registers an interrupt handler to receive HotPlug events. It
also uses the slot registers to determine the current HotPlug state
and drive an internal HotPlug state machine. For simplicty of
implementation, the PCI-PCI bridge device detaches and deletes the
child PCI device when a card is removed from a slot and creates and
attaches a PCI child device when a card is inserted into the slot.

The PCI-PCI bridge driver provides a bus_child_present which claims
that child devices are present on HotPlug-capable slots only when a
card is inserted. Rather than requiring a timeout in the RC for
config accesses to not-present children, the pcib_read/write_config
methods fail all requests when a card is not present (or not yet
ready).

These changes include support for various optional HotPlug
capabilities such as a power controller, mechanical latch,
electro-mechanical interlock, indicators, and an attention button.
It also includes support for devices which require waiting for
command completion events before initiating a subsequent HotPlug
command. However, it has only been tested on ExpressCard systems
which support surprise removal and have none of these optional
capabilities.

PCI-express HotPlug support is conditional on the PCI_HP option
which is enabled by default on arm64, x86, and powerpc.

Reviewed by: adrian, imp, vangyzen (older versions)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D6136
onf/GENERIC
726fcfebaba27d4673ab531ae33234ce097787cc 05-May-2016 zbb <zbb@FreeBSD.org> Fix GICv3 build after r299090

Obtained from: Semihalf
Sponsored by: Cavium
rm64/gic_v3.c
rm64/gic_v3_fdt.c
rm64/gic_v3_its.c
rm64/gic_v3_var.h
6cf149ba99d9dcc8dfe6cbfb47234e22a42e38fa 26-Apr-2016 andrew <andrew@FreeBSD.org> Disable ACPI on arm64 ad it has only had minimal testing and is causing
boot issues when booting with FDT. It is planned to re-enable this at a
later date.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
onf/GENERIC
778cc5a8110efa54481c3c539aab11ad44e1efad 26-Apr-2016 br <br@FreeBSD.org> Move arm's devmap to some generic place, so it can be used
by other architectures.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D6091
Sponsored by: DARPA, AFRL
Sponsored by: HEIF5
rm64/machdep.c
nclude/devmap.h
nclude/vmparam.h
b40ffdceea7e4c7e883265a00fc36fa28d67bec7 25-Apr-2016 andrew <andrew@FreeBSD.org> Use the yield instruction in the arm64 cpu_spinwait. This instruction is
a hint to the hardware the software is not performing a task.

Sponsored by: ABT Systems Ltd
nclude/cpu.h
729533413f4e97bb65852c22fb914805e2759b5a 21-Apr-2016 pfg <pfg@FreeBSD.org> sys: use our roundup2/rounddown2() macros when param.h is available.

rounddown2 tends to produce longer lines than the original code
and when the code has a high indentation level it was not really
advantageous to do the replacement.

This tries to strike a balance between readability using the macros
and flexibility of having the expressions, so not everything is
converted.
rm64/pmap.c
afb6bf1779d9255622c56b34a8b430dd7b3ea462 20-Apr-2016 andrew <andrew@FreeBSD.org> Group the ThunderX PCIe PEM newbus methods to help find them.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
avium/thunder_pcie_pem.c
cb592fde3d0e10ef4e3ec9b362332aa7fcbe742e 20-Apr-2016 andrew <andrew@FreeBSD.org> Pull out the MSI/MSI-X handling calls to simplify future intrng
integration.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
avium/thunder_pcie_pem.c
20566177418590640449d2b402ddaa8e307b265e 15-Apr-2016 pfg <pfg@FreeBSD.org> arm: for pointers replace 0 with NULL.

These are mostly cosmetical, no functional change.

Found with devel/coccinelle.
rm64/nexus.c
474f44319db073b7d8964667a071f3d68b074d92 14-Apr-2016 andrew <andrew@FreeBSD.org> Fix the types for the start, end, and count arguments to
arm_gic_fdt_alloc_resource. These were the old u_long where they should be
rman_res_t. Both of these are the same size on arm64 so this is just for
correctness, and would not have led to incorrect behaviour.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic_fdt.c
6d3c93b78f97448511c80efa6ac07fae0c315c3d 14-Apr-2016 andrew <andrew@FreeBSD.org> Set the upper limit of the DMAP region to the limit of RAM as was found in
the physmap. This will reduce the likelihood of an issue where we have
device memory mapped in the DMAP. This can only happen if it is within the
same 1G block of normal memory.

Reviewed by: kib
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5938
rm64/pmap.c
nclude/vmparam.h
f015fe6dedade0efbb0930b9dfc1d03bdfac1da0 13-Apr-2016 emaste <emaste@FreeBSD.org> arm64: Avoid null dereference in its_init_cpu

its_init_cpu() is called from gic_v3_init_secondary(), and its_sc will
be NULL if its did not attach.

Sponsored by: The FreeBSD Foundation
rm64/gic_v3_its.c
fcf07e4e7b654a94ba56b3179d5676cedb347fd8 13-Apr-2016 andrew <andrew@FreeBSD.org> Document the memory ranges within the kernel region to help with debugging
to track down which region an address is from.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
nclude/vmparam.h
66ef37d31b7a05867939b9709d3b9efe30f3602a 13-Apr-2016 andrew <andrew@FreeBSD.org> Increase the arm64 kernel address space to 512GB, and the DMAP region to
2TB. The latter can be increased in 512GB chunks by adjusting the lower
address, however more work will be needed to increase the former.

There is still some work needed to only create a DMAP region for the RAM
address space as on ARM architectures all mappings should have the same
memory attributes, and these will be different for device and normal memory.

Reviewed by: kib
Obtained from: ABT Systems Ltd
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5859
rm64/locore.S
rm64/pmap.c
nclude/vmparam.h
b63211eed5e20b9ad1b7649373ab39eff72e2521 10-Apr-2016 pfg <pfg@FreeBSD.org> Cleanup unnecessary semicolons from the kernel.

Found with devel/coccinelle.
avium/thunder_pcie_pem.c
fa19c68f9fc264035f6529ad9e6c229e27bdb83a 07-Apr-2016 zbb <zbb@FreeBSD.org> Fix interrupts delivery on ThunderX for VF IDs beyond 8

SR-IOV devices usually use Alternative Routing ID (ARI).
In that case slot/device is always assumed to be 0 and
function/identifier is extended to 8 bits.

Fix interrupts delivery to VF IDs beyond 8 by using a correct
DevID if ARI is enabled.

Reviewed by: jhb, wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5855
rm64/gic_v3_its.c
d28cfa553c23d80caf0e6c73ed4b8213769a93a6 06-Apr-2016 andrew <andrew@FreeBSD.org> Use PHYS_IN_DMAP to check if a physical address is within the DMAP region.

Approved by: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/pmap.c
c287d03447df3c2a6e83106c8fa7f3c78897b008 06-Apr-2016 andrew <andrew@FreeBSD.org> Cleanup the early pagetable creation code in preperation for increasing
the size of the arm64 DMAP region.

Approved by: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/locore.S
d532565c6da2dee9fa355e47534a76d44af6b7b1 06-Apr-2016 andrew <andrew@FreeBSD.org> Allow vmparam.h to be included from assembly files on arm64.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/copyinout.S
rm64/genassym.c
rm64/locore.S
rm64/support.S
nclude/vmparam.h
e55c02e6f81af885baabf454832a1908013331f5 06-Apr-2016 ed <ed@FreeBSD.org> Make CloudABI's way of doing TLS more friendly to userspace emulators.

We're currently seeing how hard it would be to run CloudABI binaries on
operating systems cannot be modified easily (Windows, Mac OS X). The
idea is that we want to just run them without any sandboxing. Now
that CloudABI executables are PIE, this is already a bit easier, but TLS
is still problematic:

- CloudABI executables want to write to the %fs, which typically
requires extra system calls by the emulator every time it needs to
switch between CloudABI's and its own TLS.

- If CloudABI executables overwrite the %fs base unconditionally, it
also becomes harder for the emulator to store a backup of the old
value of %fs. To solve this, let's no longer overwrite %fs, but just
%fs:0.

As CloudABI's C library does not use a TCB, this space can now be used
by an emulator to keep track of its internal state. The executable can
now safely overwrite %fs:0, as long as it makes sure that the TCB is
copied over to the new TLS area.

Ensure that there is an initial TLS area set up when the process starts,
only containing a bogus TCB. We don't really care about its contents on
FreeBSD.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D5836
loudabi64/cloudabi64_sysvec.c
f5a4347e1c8f2e48ca0d0185b8c27fb25c428c27 06-Apr-2016 wma <wma@FreeBSD.org> Implement dtrace_getupcstack in ARM64

Allow using DTRACE for performance analysis of userspace
applications - the function call stack can be captured.
This is almost an exact copy of AMD64 solution.

Obtained from: Semihalf
Sponsored by: Cavium
Reviewed by: emaste, gnn, jhibbits
Differential Revision: https://reviews.freebsd.org/D5779
nclude/frame.h
5b3779aaeeec2fe5cef5eef2e0ea28bde63b0ced 04-Apr-2016 andrew <andrew@FreeBSD.org> Add a table to map from the FreeBSD CPUID space to the GIC CPUID space. On
many SoCs these two are the same, however there is no requirement for this
to be the case, e.g. on the ARM Juno we boot on what the GIC thinks of as
CPU 2, but FreeBSD numbers it CPU 0.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/gic.c
5a20a1f054e90e27fd03970e519dd91213d04f32 04-Apr-2016 andrew <andrew@FreeBSD.org> Reduce the diff for when we switch to intrng. The IPI interrupts will be
split out to multiple handlers.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
153a55cf7ff4f26dd754308d5dade9cbfb85ba63 04-Apr-2016 wma <wma@FreeBSD.org> arm64: pagezero improvement

This change has been provided to improve pagezero call performance.

Submitted by: Dominik Ermel <der@semihalf.com>
Obtained from: Semihalf
Sponsored by: Cavium
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D5741
rm64/machdep.c
rm64/pmap.c
rm64/support.S
nclude/machdep.h
da4a81138784c8d127dc19e2a4429b3573969afd 04-Apr-2016 wma <wma@FreeBSD.org> Add bzero.S to ARM64 machdep

Add fille missing from https://svnweb.freebsd.org/changeset/base/297536
rm64/bzero.S
4ba80483c09bada259d55bd321495c2af37bd702 04-Apr-2016 wma <wma@FreeBSD.org> arm64: bzero optimization

This optimization attempts to utylize as wide as possible register store instructions to zero large buffers.
The implementation, if possible, will use 'dc zva' to zero buffer by cache lines.

Speedup: 60x faster memory zeroing

Submitted by: Dominik Ermel <der@semihalf.com>
Obtained from: Semihalf
Sponsored by: Cavium
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D5726
rm64/machdep.c
nclude/armreg.h
910e4d679c8dddad93fe7407ff0625354b39c285 31-Mar-2016 ed <ed@FreeBSD.org> Make Position Independent Executables work for CloudABI.

- Set BI_CAN_EXEC_DYN, so we can execute ET_DYN ELF files in addition to
regular ET_EXECs.
- Provide an AT_BASE entry in the auxiliary vector, so the executable
knows at which address it got loaded and can apply relocations.
loudabi64/cloudabi64_sysvec.c
b45ea0fe804ce62632a75922341348fdbe352633 31-Mar-2016 andrew <andrew@FreeBSD.org> Add support for 4 level pagetables. The userland address space has been
increased to 256TiB. The kernel address space can also be increased to be
the same size, but this will be performed in a later change.

To help work with an extra level of page tables two new functions have
been added, one to file the lowest level table entry, and one to find the
block/page level. Both of these find the entry for a given pmap and virtual
address.

This has been tested with a combination of buildworld, stress2 tests, and
by using sort to consume a large amount of memory by sorting /dev/zero. No
new issues are known to be present from this change.

Reviewed by: kib
Obtained from: ABT Systems Ltd
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5720
rm64/genassym.c
rm64/locore.S
rm64/machdep.c
rm64/minidump_machdep.c
rm64/pmap.c
rm64/swtch.S
rm64/vm_machdep.c
nclude/machdep.h
nclude/pcb.h
nclude/pmap.h
nclude/pte.h
nclude/vmparam.h
68e9a48abbf297dc6891e141adfa1e5ba920d0a8 29-Mar-2016 andrew <andrew@FreeBSD.org> Read the CPU ID for the current CPU from the GIC. The GIC may have a
different ID space than the kernel. Because of this we need to read the
ID from the hardware. The hardware will provide this value to the CPU by
reading any of the first 8 Interrupt Processor Targets Registers.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5706
rm64/gic.c
9141d11115042913496baa463193e17cb83fb6ac 24-Mar-2016 wma <wma@FreeBSD.org> arm64: Fixing user space boudary checking in copyinout.S

Big buffer size could cause integer overflow and as a result
attempt to copy beyond VM_USERMAX_ADDRESS.

Fixing copyinstr boundary checking where compared value has been
overwritten by accident when setting fault handler.

Submitted by: Dominik Ermel <der@semihalf.com>
Obtained from: Semihalf
Sponsored by: Cavium
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D5719
rm64/copyinout.S
15e11f0a03013ee1bae4d9be9d13810adb32e629 23-Mar-2016 wma <wma@FreeBSD.org> ARM64 copyinout improvements

The first of set of patches.
Use wider load/stores when aligned buffer is being copied.

In a simple test:
dd if=/dev/zero of=/dev/null bs=1M count=1024
the performance jumped from 410MB/s up to 3.6GB/s.

TODO:
- better handling of unaligned buffers (WiP)
- implement similar mechanism to bzero

Submitted by: Dominik Ermel <der@semihalf.com>
Obtained from: Semihalf
Sponsored by: Cavium
Reviewed by: kib, andrew, emaste
Differential Revision: https://reviews.freebsd.org/D5664
rm64/copyinout.S
acaab8fb107cfe1d09dafbf09733594df2459be1 22-Mar-2016 andrew <andrew@FreeBSD.org> Use the saved program state register to detect when an exception frame is
from userpsace. Previously we could have triggered a panic by trying to
jump to a kernel address from userland as the trap handling code thought we
received an ast in kernel mode.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
nclude/cpu.h
47415357accafefd4b215d76208e4eff03614d1d 18-Mar-2016 andrew <andrew@FreeBSD.org> Move the opt_ files to be included first so their definitions can be used
from within all further included files.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/nexus.c
d5c75c2b10120df7a70de80520c5118e79d0f6c7 18-Mar-2016 andrew <andrew@FreeBSD.org> Rename COUNT_IPI to INTR_IPI_COUNT to reduce the diff with intrng.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/mp_machdep.c
nclude/smp.h
162f59a4a041e369446c4ff049862dad6fce886d 18-Mar-2016 andrew <andrew@FreeBSD.org> Reduce the diff with intrng by renaming similar functions. This is a noop,
but will help move to use the common interrupt handling code later.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
rm64/exception.S
rm64/intr_machdep.c
rm64/nexus.c
nclude/intr.h
df8c050ab1d3eb81d8114fedbd9e3372c58bfce6 18-Mar-2016 andrew <andrew@FreeBSD.org> Remove the invalid L0_BLOCK definition. ARMv8 doesn't support block
translation in the level 0 descriptor.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
nclude/pte.h
ca92c5bf0943ec24b4a2a0e0b943e40322d9faa4 14-Mar-2016 wma <wma@FreeBSD.org> pmap arm64: fixing pmap_invalidate_range

It seems that if range within one page is given this page will not be
invalidated at all. Clean it up.

Submitted by: Dominik Ermel <der@semihalf.com>
Obtained from: Semihalf
Sponsored by: Cavium
Reviewed by: wma, zbb
Approved by: cognet (mentor)
Differential Revision: https://reviews.freebsd.org/D5569
rm64/pmap.c
e641458f7012aa8c7ad2bc4f3ea9cf9ee880e17e 09-Mar-2016 jhb <jhb@FreeBSD.org> Fix reporting of the CloudABI ABI in kdump.

- Advertise the word size for CloudABI ABIs via the SV_LP64 flag. All of
the other ABIs include either SV_ILP32 or SV_LP64.
- Fix kdump to not assume a 32-bit ABI if the ABI flags field is non-zero
but SV_LP64 isn't set. Instead, only assume a 32-bit ABI if SV_ILP32 is
set and fallback to the unknown value of "00" if neither SV_LP64 nor
SV_ILP32 is set.

Reviewed by: kib, ed
Differential Revision: https://reviews.freebsd.org/D5560
loudabi64/cloudabi64_sysvec.c
983c6646122b6c7333bdf217e6def382ea04744e 02-Mar-2016 bz <bz@FreeBSD.org> Force re-routing PCI interrupts (this is for legacy INTx not MSI).
Need this for gem5, but was not needed on real hadrware (yet) as it
was always MSI.

Reviewed by: andrew, jhb
Discovered by: andrew
Sponsored by: DARPA/AFRL
Differential Revision: https://reviews.freebsd.org/D5494
nclude/param.h
486a6ad7e085b98f5fa71ca32860e1da0a8f5e7e 02-Mar-2016 wma <wma@FreeBSD.org> Improve ThunderX PEM driver to work on pass2 revision

Things changed:
* do not allocate 4GB of SLI space, because it's the waste of
system resources. Allocate only small portions when needed.
* provide own implementation of activate_resource which performs
address translation between PCI bus and host PA address space.
This is temporary solution, should be replaced by bus_map_resource
once implemented.

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: jhb
Differential revision: https://reviews.freebsd.org/D5294
avium/thunder_pcie_common.c
avium/thunder_pcie_common.h
avium/thunder_pcie_pem.c
0a7f0b77a3997aceb1642341e8efa2b7013bf74b 01-Mar-2016 wma <wma@FreeBSD.org> Get memory ranges from FDT if no EFI API is available on ARM64

Obtained from: Semihalf
Submitted by: Michal Stanek <mst@semihalf.com>
Sponsored by: Annapurna Labs
Approved by: cognet (mentor)
Reviewed by: andrew, wma
Differential revision: https://reviews.freebsd.org/D5408
rm64/machdep.c
ad7622cfab408f2eb3eb50f016ab3383cb0f5990 01-Mar-2016 wma <wma@FreeBSD.org> Enable SRE_EL2 on ARM64

Enable system register access for EL2. Alpine-V2 is
the first device requiring this to be enabled.
It is also in-sync with Linux initialization code,
and compatible with Alpine-V2 uboot requirements.

Obtained from: Semihalf
Submitted by: Michal Stanek <mst@semihalf.com>
Sponsored by: Annapurna Labs
Approved by: cognet (mentor)
Reviewed by: wma
Differential revision: https://reviews.freebsd.org/D5394
rm64/locore.S
nclude/armreg.h
8076bc06d9994f207d949c72cb92b3919d1a3f37 01-Mar-2016 wma <wma@FreeBSD.org> Add uart 8250 device to GENERIC arm64 configuration

Obtained from: Semihalf
Submitted by: Michal Stanek <mst@semihalf.com>
Sponsored by: Annapurna Labs
Approved by: cognet (mentor)
Reviewed by: zbb, wma
Differential revision: https://reviews.freebsd.org/D5406
onf/GENERIC
23e52c351253c7260d6757e98af0854e074ad60f 01-Mar-2016 jhibbits <jhibbits@FreeBSD.org> Correct the memory rman ranges to be to BUS_SPACE_MAXADDR

Summary:
As part of the migration of rman_res_t to be typed to uintmax_t, memory ranges
must be clamped appropriately for the bus, to prevent completely bogus addresses
from being used.

This is extracted from D4544.

Reviewed By: cem
Sponsored by: Alex Perez/Inertial Computing
Differential Revision: https://reviews.freebsd.org/D5134
rm64/nexus.c
ac0c41dc12dbe4f4bfbab46621fb4fa119241aae 26-Feb-2016 wma <wma@FreeBSD.org> Restore ThunderX Pass1.1 PCI changes removed by r295962

If Enhanced Allocation is not used, we can't allocate any random
range. All internal devices have hardcoded place where they can
be located within PCI address space. Fortunately, we can read
this value from BAR.

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: zbb
Differential revision: https://reviews.freebsd.org/D5455
avium/thunder_pcie_common.c
avium/thunder_pcie_common.h
avium/thunder_pcie_fdt.c
7d1ae5faf2be44738f09aea3cb84ad678894491f 24-Feb-2016 wma <wma@FreeBSD.org> Make pci_host_generic and thunderx_pci common
* provided OFW interface for pci_host_generic (for handling devices which are present in DTS under the PCI node)
* removed support for internal PCI from arm64/cavium
* cleaned up and made most of the code common

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: zbb
Differential revision: https://reviews.freebsd.org/D5261
avium/thunder_pcie.c
avium/thunder_pcie_common.c
avium/thunder_pcie_common.h
avium/thunder_pcie_fdt.c
avium/thunder_pcie_pem.c
avium/thunder_pcie_pem.h
avium/thunder_pcie_pem_fdt.c
e960072e8386ff9aa2445eddf84771b5d9df544f 22-Feb-2016 wma <wma@FreeBSD.org> Add Intel 10Gb support to ARM64 GENERIC kernel config

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: zbb
Differential revision: https://reviews.freebsd.org/D5347
onf/GENERIC
ad68cf93b11d0db8c7664ce8557cc38db6aa7468 22-Feb-2016 skra <skra@FreeBSD.org> As <machine/vmparam.h> is included from <vm/vm_param.h>, there is no
need to include it explicitly when <vm/vm_param.h> is already included.

Suggested by: alc
Reviewed by: alc
Differential Revision: https://reviews.freebsd.org/D5379
rm64/minidump_machdep.c
rm64/trap.c
812447f90a35cc600130f698908186c552c028a2 22-Feb-2016 skra <skra@FreeBSD.org> As <machine/param.h> is included from <sys/param.h>, there is no need
to include it explicitly when <sys/param.h> is already included.

Reviewed by: alc, kib
Differential Revision: https://reviews.freebsd.org/D5378
rm64/debug_monitor.c
f4b6499ab587e531fac6a6bd8273de3a755eed65 22-Feb-2016 skra <skra@FreeBSD.org> As <machine/pmap.h> is included from <vm/pmap.h>, there is no need to
include it explicitly when <vm/pmap.h> is already included.

Reviewed by: alc, kib
Differential Revision: https://reviews.freebsd.org/D5373
rm64/minidump_machdep.c
loudabi64/cloudabi64_sysvec.c
f8385663eeba2c10cd82a51c0a461a4f932458a3 20-Feb-2016 jhibbits <jhibbits@FreeBSD.org> Introduce a RMAN_IS_DEFAULT_RANGE() macro, and use it.

This simplifies checking for default resource range for bus_alloc_resource(),
and improves readability.

This is part of, and related to, the migration of rman_res_t from u_long to
uintmax_t.

Discussed with: jhb
Suggested by: marcel
rm64/gic_fdt.c
rm64/gic_v3_fdt.c
rm64/nexus.c
avium/thunder_pcie.c
avium/thunder_pcie_fdt.c
avium/thunder_pcie_pem.c
d2a1177be6963874bcf8d15330e4ea74c36d6823 18-Feb-2016 zbb <zbb@FreeBSD.org> Introduce bus_get_bus_tag() method

Provide bus_get_bus_tag() for sparc64, powerpc, arm, arm64 and mips
nexus and its children in order to return a platform specific default tag.

This is required to ensure generic correctness of the bus_space tag.
It is especially needed for arches where child bus tag does not match
the parent bus tag. This solves the problem with ppc architecture
where the PCI bus tag differs from parent bus tag which is big-endian.

This commit is a part of the following patch:
https://reviews.freebsd.org/D4879

Submitted by: Marcin Mazurek <mma@semihalf.com>
Obtained from: Semihalf
Sponsored by: Annapurna Labs
Reviewed by: jhibbits, mmel
Differential Revision: https://reviews.freebsd.org/D4879
rm64/nexus.c
607785ea5a20b76ef6040a64af9b904fc9fa1a32 18-Feb-2016 wma <wma@FreeBSD.org> Fix ThunderX external PEM bus offset

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: zbb
Differential revision: https://reviews.freebsd.org/D5293
rm64/gic_v3_its.c
19710a8b475c3e518b72fb0053a31a5c7fa2e49e 17-Feb-2016 skra <skra@FreeBSD.org> Remove pd_prot and pd_cache members from struct arm_devmap_entry.
The struct is used for definition of static device mappings which
should always have same protection and attributes.
nclude/devmap.h
05274f8a8b8da8cd3adf7aa6d8a877992dbc44d2 16-Feb-2016 andrew <andrew@FreeBSD.org> Allow callers of OF_decode_addr to get the size of the found mapping. This
will allow for code that uses the old fdt_get_range and fdt_regsize
functions to find a range, map it, access, then unmap to replace this, up
to and including the map, with a call to OF_decode_addr.

As this function should only be used in the early boot code the unmap is
mostly do document we no longer need the mapping as it's a no-op, at least
on arm.

Reviewed by: jhibbits
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D5258
rm64/ofw_machdep.c
a48186faeaf8c55b183253e73decf5baa6f17597 16-Feb-2016 zbb <zbb@FreeBSD.org> Support PEM that is not a PCI endpoint on ThunderX

Some chip revisions don't have their external PCIe buses
behind the internal bridge. Add support for FDT-configurable
PEMs but keep ability for PCIe enumeration.

Reviewed by: andrew, wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5285
avium/thunder_pcie_common.h
avium/thunder_pcie_pem.c
avium/thunder_pcie_pem.h
avium/thunder_pcie_pem_fdt.c
9305b5748d57bfc77ebc4868162f0fedcc0d887e 12-Feb-2016 andrew <andrew@FreeBSD.org> Only update curthread and curpcb after we have finished using the old
values.

If switching from a thread that used floating-point registers to a thread
that is still running, but holding the blocked_lock lock we would switch
the curthread to the new (running) thread, then call critical_enter. This
will non-atomically increment td_critnest, and later call critical_exit to
non-atomically decrement this value.

This can happen at the same time as the new thread is still running on the
old core, also calling these functions. In this case there will be a race
between these non-atomic operations. This can be an issue as we could loose
one of these operations leading to the value to not return to zero.

If, later on, we then hit a data abort we check if the td_critnest is zero.
If this check fails we will panic the kernel.

This has been observed when running pcmstat on a Cavium ThunderX. The pcm
thread will use the blocked_lock lock and there is a high chance userspace
will use the floating-point registers. When, later on, pmcstat triggers a
data abort we will hit this panic.

The fix is to update these values after storing the floating-point state.
This means we use the correct curthread while storing the state so it will
not be an issue that the changes to td_critnest are non-atomic.

Sponsored by: ABT Systems Ltd
rm64/swtch.S
5d0577881561110fd5459b9f25120154e6198faf 11-Feb-2016 zbb <zbb@FreeBSD.org> Support interrupts binding in GICv3 and ITS

- Add MOVI command and routine for the LPI migration
- Allow to search for the ITS device descriptor using
not only devID but also LPI number.
- Bind SPIs in the Distributor
- Don't bind its_dev to collection. Keep track of the collection
IDs for each LPI.

Reviewed by: wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5231
rm64/gic_v3.c
rm64/gic_v3_its.c
rm64/gic_v3_var.h
1187007450deeee9710b4ecd076942618e5dbde5 11-Feb-2016 zbb <zbb@FreeBSD.org> Implement finer locking in ITS

- Change locks' names to be more suitable
- Don't use blocking mutex. Lock only basic operations such
as lists or bitmaps modifications.

Reviewed by: wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5230
rm64/gic_v3_its.c
rm64/gic_v3_var.h
b8738668ec859266c7baace1ffc3e08ec7a7eff2 11-Feb-2016 zbb <zbb@FreeBSD.org> Initially bind all interrupts to the boot CPU when using GICv3

This should be done by routing all interrupts to CPU0,
different assignment will be induced by either interrupts
shuffling or bus_bind_intr().

Reviewed by: wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5229
rm64/gic_v3.c
rm64/gic_v3_its.c
0740653d54119f91b101f35a01f9ee2a4fdec95b 11-Feb-2016 zbb <zbb@FreeBSD.org> Call pmc_hook() correctly in the ARM64 interrupt handler

pmc_hook() was called only in case of the stray interrupt but should
rather be called on each interrupt. Move in to the arm_cpu_intr()
handler, out of the critical section too.

Reviewed by: br
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5161
rm64/intr_machdep.c
95137856d119ee0d525eb022f97b0b53749fdf8a 11-Feb-2016 zbb <zbb@FreeBSD.org> Introduce bus_bind_intr method for ARM64

It can be used to bind specific interrupt to a particular CPU.
Requires PIC support for interrupts binding.

Reviewed by: wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5122
rm64/intr_machdep.c
rm64/nexus.c
rm64/pic_if.m
nclude/intr.h
45b4fa372ea10b8f0881e1f930cfe8e6ece95c1d 11-Feb-2016 zbb <zbb@FreeBSD.org> Fix bugs in interrupts allocation on ARM64

Separate interrupt descriptors lookup from allocation. It was possible
to perform config on non-existing interrupt simply by allocating spurious
descriptor.
Must lock the interrupt descriptors table lookup to avoid mismatches.
This ought to prevent trouble while setting up new interrupt
and dispatching existing one.
Use spin mutex rather than sleep mutex. This is mainly due to lock in
arm_dispatch_intr.
This should be eventually changed to a lock-less solution without
walking through a linked list on each interrupt.

Reviewed by: andrew, wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5121
rm64/intr_machdep.c
95c8add1f2e9d0cef6da55a3fc9985db68f23e21 11-Feb-2016 zbb <zbb@FreeBSD.org> Minor clean-ups for ARM64 GICv3 and GIC drivers

GICv3:
- move ICC_SGI1R_EL1 definitions to armreg.h and use proper system
register's names
GIC:
- remove unused functions

Reviewed by: andrew
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5119
rm64/gic.c
rm64/gic_v3.c
rm64/gic_v3_reg.h
nclude/armreg.h
a33f5ef2d9ebef260c055337bf94134b2b5fd1e9 11-Feb-2016 wma <wma@FreeBSD.org> ARM64 disassembler: support for LDR instructions

Implemented disassembly for a whole bunch of
various ldr instructions.

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: zbb
Differential revision: https://reviews.freebsd.org/D5217
rm64/disassem.c
bfa435406f12bc3d084f6fd8dce69e81a24c9042 10-Feb-2016 andrew <andrew@FreeBSD.org> Revert an arm64 change that sneaked in with r295464.
rm64/swtch.S
4cf4fddec8c3df6f92a4c842133d17613c459575 10-Feb-2016 andrew <andrew@FreeBSD.org> Update of the Allwinner drivers to:
* Use the Linux compat string
* Use EARLY_DRIVER_MODULE to attach at the right time
* Add a generic A10 kernel config file
* A20 now use generic_timer
* Add two new dts files for Olimex boards
* Update our custom DTS file for A10 and A20 to use the same compatible
property names as the vendor ones.

Submitted by: Emmanuel Vadot <manu@bidouilliste.com>
Differential Revision: https://reviews.freebsd.org/D4792
rm64/swtch.S
b3c4f0ddbf7454fa0fadf3225bb6b722d4fbbc4a 09-Feb-2016 glebius <glebius@FreeBSD.org> Include sys/_task.h into uma_int.h, so that taskqueue.h isn't a
requirement for uma_int.h.

Suggested by: jhb
rm64/uma_machdep.c
rm64/vm_machdep.c
6b818b637142586b9543f3a199b15896650af307 09-Feb-2016 andrew <andrew@FreeBSD.org> Use designated initialisers for the db disassembler interface to help with
finding the struct when searching for one of its members.
rm64/db_disasm.c
db0cfe84022c1064861d082e313951d82bffc4e7 09-Feb-2016 wma <wma@FreeBSD.org> Ignore invalid page descriptors in ARM64 pmap_mincore

Prevent the function from null-pointer-dereference when unexisting
mapping is being processed.

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: zbb, cognet
Differential revision: https://reviews.freebsd.org/D5228
rm64/pmap.c
a718ec05809255deb93481e75282825de5dc21b5 05-Feb-2016 andrew <andrew@FreeBSD.org> Implement kdb_cpu_sync_icache on arm64.

Sponsored by: ABT Systems Ltd
nclude/kdb.h
327df1a84adbc8d73c704c55a2b3ceaacb6cdd96 04-Feb-2016 andrew <andrew@FreeBSD.org> Enable checking of the stack alignment. The stack should be aligned to a
16-byte value. With this the hardware will check if a memory access uses
an incorrectly aligned stack pointer as the base address.

Sponsored by: ABT Systems Ltd
rm64/locore.S
30a404b9164af3ef3de2f8f38e0050f6b71fb1aa 04-Feb-2016 br <br@FreeBSD.org> Fix build.
rm64/vm_machdep.c
e807599e3ffdfd1caaff4c43f3e2a3b7b6289440 04-Feb-2016 glebius <glebius@FreeBSD.org> Fix build.
rm64/uma_machdep.c
1ef2c8adf6388662de306ad64df5252c02e3bfbd 03-Feb-2016 andrew <andrew@FreeBSD.org> Handle a misaligned stack pointer exception from userspace. The exception
still needs to be enabled, but this will help with testing.

Sponsored by: ABT Systems Ltd
rm64/trap.c
5b58d7a3f84952d4d0e45ed7d2501aa4c1a28f2e 02-Feb-2016 andrew <andrew@FreeBSD.org> Increase the size of PHYS_AVAIL_SIZE to allow firmware to provide a large
number of physical memory locations we can access. This is the case on
some HiKey boards that may have UEFI reserved memory dispersed through the
physical space.

Sponsored by: ABT Systems Ltd
nclude/pmap.h
700dcbc1a2bc36e3a116192b615faf238f4e6e72 02-Feb-2016 andrew <andrew@FreeBSD.org> Ensure we don't overflow the phys_avail array. Some firmware may provide
more memory locations than we have space to record.

Sponsored by: ABT Systems Ltd
rm64/pmap.c
7c81e8b6526c1c0b426d113b839b2f72ef058929 02-Feb-2016 andrew <andrew@FreeBSD.org> Increase the space we use after the kernel to 8MiB. On 2GiB HiKey board we
would try to access data past this point stopping the boot.

Sponsored by: ABT Systems Ltd
rm64/locore.S
80defbca6b1280bd85e38b31455b6e3af0158dd0 02-Feb-2016 andrew <andrew@FreeBSD.org> Only look for the ranges property when we have children. This fixes booting
on systems with a gicv2, but no PCIe so no gicv2m.

Sponsored by: ABT Systems Ltd
rm64/gic_fdt.c
f0c954c605dca38f4db001f41011e1ae332e2408 02-Feb-2016 andrew <andrew@FreeBSD.org> Implement single stepping on arm64. We need to set the single step bits in
the processor and debug state registers. A flag has been added to the pcb
to tell us when to enable single stepping for a given thread.

Reviewed by: kib
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D4730
rm64/genassym.c
rm64/machdep.c
rm64/mp_machdep.c
rm64/swtch.S
rm64/trap.c
nclude/armreg.h
nclude/pcb.h
86ce64705e75081226b33569e8335f94947a0514 02-Feb-2016 andrew <andrew@FreeBSD.org> Correctly handle the case where copystr(9) is given a string longer than
the passed in length. In this case we need to return ENAMETOOLONG.
rm64/copystr.c
06c93e5cef7669f0580e7cd320df054124e95f35 29-Jan-2016 zbb <zbb@FreeBSD.org> Fix sending IPI to all CPUs on ARM64

There is no explanation why IPI ID is incremented here by "16".
This should have been removed in r285533 but somehow survived.

Reviewed by: wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5120
rm64/intr_machdep.c
047ea0e83b66f348bad2128bff3bc3e763acb3c3 29-Jan-2016 wma <wma@FreeBSD.org> Framework for ARM64 instruction disassembler

Provide an easy to use framework for ARM64 DDB disassembler.
This commit does not contain full list of instruction opcodes.

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: zbb, andrew, cognet
Differential revision: https://reviews.freebsd.org/D5114
rm64/db_disasm.c
rm64/disassem.c
nclude/armreg.h
nclude/disassem.h
792baee6b7e046d5fba07463a8c505674df55600 28-Jan-2016 zbb <zbb@FreeBSD.org> Fix VNIC enumeration after r294993 and r294990

ofw_bus_get_node() must be tested against negative values since
missing parent bus method will result in calling the default method
which simply returns (-1): sys/dev/ofw/ofw_bus_if.m
This was lost in the review process.

Obtained from: Semihalf
Sponsored by: Cavium
avium/thunder_pcie_fdt.c
096bc4d6c4a620f1253b7ca74b9dc02cf3baf51d 28-Jan-2016 zbb <zbb@FreeBSD.org> Add FDT bus capabilities to ThunderX PCI driver

New ThunderX firmware incorporates modified DTB that presents
different device hierarchy. In the new device tree, MDIO
devices are below two additional buses that oddly hang on
PCI bridge.

Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5069
avium/thunder_pcie.c
avium/thunder_pcie_common.c
avium/thunder_pcie_common.h
avium/thunder_pcie_fdt.c
0f04b1d51e13d3825e8e209609a2d691453625f9 28-Jan-2016 zbb <zbb@FreeBSD.org> Correct alloc_ and release_resource methods in thunder_pcie driver

- Avoid using BUS_ macros as bus_generic_ functions should be used instead.
- Fix mistaken device_t pointers in thunder_pcie_alloc_resource.
Should use dev->parent method and allocate resource for child device

Reviewed by: wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5068
avium/thunder_pcie.c
a4a7f925b592154694c3994b89ac838eb07592cf 28-Jan-2016 zbb <zbb@FreeBSD.org> Divide ThunderX PCIe driver to general and FDT part

- Separate FDT and general PCIe driver parts
- Drop some irrelevant printfs that cannot be displayed in
FDT attach
- Move ranges parsing to FDT portion of PCIe code

Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5067
avium/thunder_pcie.c
avium/thunder_pcie_common.c
avium/thunder_pcie_common.h
avium/thunder_pcie_fdt.c
d54bf9a884664360a49f9470547ec8310a002a9b 28-Jan-2016 wma <wma@FreeBSD.org> Fix mutex