FreeBSD/src 349222stable 11, stable/11/sys/vm uma_core.c

MFC r348764: Allow UMA hash tables to expand faster then 2x in 20 seconds.

ZFS ABD allocates tons of 4KB chunks via UMA, requiring huge hash tables.
With initial hash table size of only 32 elements it takes ~20 expansions
or ~400 seconds to adapt to handling 220GB ZFS ARC.  During that time not
only the hash table is highly inefficient, but also each of those expan-
sions takes significant time with the lock held, blocking operation.

On my test system with 256GB of RAM and ZFS pool of 28 HDDs this change
reduces time needed to first time read 240GB from ~300-400s, during which
system is quite busy and unresponsive, to only ~150s with light CPU load
and just 5 sub-second CPU spikes to expand the hash table.

FreeBSD/src 349221stable 12, stable/12/sys/vm uma_core.c

MFC r348764: Allow UMA hash tables to expand faster then 2x in 20 seconds.

ZFS ABD allocates tons of 4KB chunks via UMA, requiring huge hash tables.
With initial hash table size of only 32 elements it takes ~20 expansions
or ~400 seconds to adapt to handling 220GB ZFS ARC.  During that time not
only the hash table is highly inefficient, but also each of those expan-
sions takes significant time with the lock held, blocking operation.

On my test system with 256GB of RAM and ZFS pool of 28 HDDs this change
reduces time needed to first time read 240GB from ~300-400s, during which
system is quite busy and unresponsive, to only ~150s with light CPU load
and just 5 sub-second CPU spikes to expand the hash table.

FreeBSD/src 349220head/share/man/man9 sleep.9 sleepqueue.9, head/sys/kern subr_sleepqueue.c kern_synch.c

Add wakeup_any(), cheaper wakeup_one() for taskqueue(9).

wakeup_one() and underlying sleepq_signal() spend additional time trying
to be fair, waking thread with highest priority, sleeping longest time.
But in case of taskqueue there are many absolutely identical threads, and
any fairness between them is quite pointless.  It makes even worse, since
round-robin wakeups not only make previous CPU affinity in scheduler quite
useless, but also hide from user chance to see CPU bottlenecks, when
sequential workload with one request at a time looks evenly distributed
between multiple threads.

This change adds new SLEEPQ_UNFAIR flag to sleepq_signal(), making it wakeup
thread that went to sleep last, but no longer in context switch (to avoid
immediate spinning on the thread lock).  On top of that new wakeup_any()
function is added, equivalent to wakeup_one(), but setting the flag.
On top of that taskqueue(9) is switchied to wakeup_any() to wakeup its
threads.

As result, on 72-core Xeon v4 machine sequential ZFS write to 12 ZVOLs
with 16KB block size spend 34% less time in wakeup_any() and descendants
then it was spending in wakeup_one(), and total write throughput increased
by ~10% with the same as before CPU usage.

Reviewed by:    markj, mmacy
MFC after:      2 weeks

    [2 lines not shown]

FreeBSD/src 349219stable 12, stable/12/share/man/man3 Makefile

MFC r349148:
Add some missing MLINKs for tree(3).

FreeBSD/src 349218head/sys/vm vm_page.c

Group vm_page_activate()'s definition with other related functions.

No functional change intended.

MFC after:      3 days
DeltaFile
+29-29head/sys/vm/vm_page.c
+29-291 files

FreeBSD/src 349217head/stand/libsa/zfs zfsimpl.c

Tell loader to ignore newer features enabled on the root pool.

There are many new features in ZoF. Most, if not all, do not effect read only usage.
Encryption in particular is enabled at the pool level but used at the dataset level.
The loader obviously will not be able to boot if the boot dataset is encrypted, but
should not care if some other dataset in the root pool is encrypted.

Reviewed by:    allanjude
MFC after:      1 week

FreeBSD/src 349216stable 11, stable/11/sys/cddl/contrib/opensolaris/uts/common/fs/zfs arc.c

MFC r348772: Restore ARC MFU/MRU pressure

Submitted by:   Slawa Olhovchenkov <slw at zxy.spb.ru>
Sponsored by:   Integros [integros.com]

FreeBSD/src 349215stable 12, stable/12/sys/cddl/contrib/opensolaris/uts/common/fs/zfs arc.c

MFC r348772: Restore ARC MFU/MRU pressure

Submitted by:   Slawa Olhovchenkov <slw at zxy.spb.ru>
Sponsored by:   Integros [integros.com]

FreeBSD/src 349214stable 11, stable/11/sys/dev/amdgpio amdgpio.c

MFC r348228: amdgpio: fix reading status of input pins

FreeBSD/src 349213stable 12, stable/12/sys/dev/amdgpio amdgpio.c

MFC r348228: amdgpio: fix reading status of input pins

FreeBSD/src 349212stable 11, stable/11/sys/dev/amdgpio amdgpio.c

FC r348227: amdgpio: remove new line symbols from pin names

FreeBSD/src 349211stable 12, stable/12/sys/dev/amdgpio amdgpio.c

MFC r348227: amdgpio: remove new line symbols from pin names

FreeBSD/src 349210stable 11, stable/11/sys/dev/mrsas mrsas.c

MFC r348159: add mrsas_shutdown method

Sponsored by:   Panzura

FreeBSD/src 349209stable 12, stable/12/sys/dev/mrsas mrsas.c

MFC r348159: add mrsas_shutdown method

Sponsored by:   Panzura

FreeBSD/src 349208stable 11, stable/11/share/man/man4 gpioled.4

MFC r348153-r348155: gpioled: add a new hint for initial state

hint.gpioled.%d.state determines the initial state of the LED when the
driver takes control over it:
  0 - the LED is off
  1 - the LED is on
 -1 - the LED is kept as it was

While here, add a module version declaration.

FreeBSD/src 349207stable 12, stable/12/sys/dev/usb usbdevs

MFC r348152: Add USB ID for CP2112

FreeBSD/src 349206stable 11, stable/11/sys/dev/usb usbdevs

MFC r348152: Add USB ID for CP2112

FreeBSD/src 349205stable 11, stable/11/sys/dev/intpm intpm.c

MFC r345411: intpm: change translation of HBA error status to smbus(4) errors

PIIX4_SMBHSTSTAT_ERR can be set for several reasons that, unfortunately,
cannot be distinguished, but the most typical case is a missing or hung
slave (SMB_ENOACK).

PIIX4_SMBHSTSTAT_FAIL means failed or killed / aborted transaction, so
it's previous mapping to SMB_ENOACK was not ideal.

After this change an smb(4) access to a missing slave results in ENXIO
rather than EIO.  To me, that seems to be more appropriate.

FreeBSD/src 349204stable 11, stable/11/cddl/contrib/opensolaris/cmd/zpool zpool.8

MFC r344360,r344361: zpool.8: document -D flag for zpool status

Also, sort zpool status flags in the same order as in illumos manual.

FreeBSD/src 349203stable 11, stable/11/cddl/contrib/opensolaris/lib/libzpool/common taskq.c

MFC r344359: fix userland illumos taskq code to pass relative timeout to cv_timedwait

Sponsored by:   Panzura

FreeBSD/src 349202head/share/mk bsd.sys.mk

Follow-up r349065: Fix .TARGET flag ambiguity with PROGS which broke MK_TESTS.

X-MFC-With:     r349065
Sponsored by:   DellEMC

FreeBSD/src 349201head/stand/efi/libefi efinet.c

efinet: Defer exclusively opening the network handles

Don't commit to exclusive access to the network device handle by
efinet until the loader has decided to load something through the
network. This allows for the possibility of other users of the
network device.

Submitted by:   scottph
Reviewed by:    tsoome, emaste
Tested by:      tsoome, bcran
Differential Revision:  https://reviews.freebsd.org/D20642

FreeBSD/src 349200releng/12.0 UPDATING, releng/12.0/sys/conf newvers.sh

Add UPDATING entries for today's release. Bump patch version.

Also fix the previous UPDATING entry. MDS was an SA, not an EN.

Approved by:    so

FreeBSD/src 349199releng/12.0/sys/netinet/tcp_stacks rack.c tcp_rack.h

Add ability to limit split map entries to prevent resource
exhaustion in non-default RACK TCP stack.

Approved by:    so
Security:        FreeBSD-SA-19:08.rack
Security:        CVE-2019-5599

FreeBSD/src 349198releng/12.0/sys/net if.c if_var.h

Fix incorrect locking in networking stack.

Approved by:    so
Security:       FreeBSD-EN-19:11.net

FreeBSD/src 349197stable 12, stable/12/sys/netinet/tcp_stacks rack.c tcp_rack.h

MFC r349192:
  Add the ability to limit how much the code will fragment the RACK send map
  in response to SACKs. The default behavior is unchanged; however, the
  limit can be activated by changing the new net.inet.tcp.rack.split_limit
  sysctl.

Approved by:    so (gordon)
Security:       CVE-2019-5599

FreeBSD/src 349196head/usr.sbin/bhyve rfb.c

Make zlib encoding messages idempotent.

Otherwise duplicate messages can trigger a reinitialization of the
compression stream while the update thread is running.  Also ensure
that the stream is initialized before the update thread may attempt
to use it.

PR:            238333
Reviewed by:    cem, rgrimes
MFC after:      3 days
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D20673

FreeBSD/src 349195head/sys/geom geom_dump.c geom_disk.c, head/sys/geom/eli g_eli.c

Use sbuf_cat() in GEOM confxml generation.

When it comes to megabytes of text, difference between sbuf_printf() and
sbuf_cat() becomes substantial.

MFC after:      2 weeks
Sponsored by:   iXsystems, Inc.

FreeBSD/src 349194releng/11.3/release/pkg_repos release-dvd.conf

Update the pkg.conf used to populate the dvd1.iso to use
the release_3 package set.

Approved by:    re (implicit)
Sponsored by:   The FreeBSD Foundation

FreeBSD/src 349193stable 12, stable/12/sys/kern makesyscalls.sh

MFC r347228: makesyscalls: use @generated tag in generated files

Multiple tools use @generated to identify generated files (for example,
in a review Phabricator will by default hide diffs in generated files).
Use the @generated tag in makesyscalls.sh as we've done for other
generated files.

Sponsored by:   The FreeBSD Foundation

FreeBSD/src 349192head/sys/netinet/tcp_stacks rack.c tcp_rack.h

Add the ability to limit how much the code will fragment the RACK send map
in response to SACKs. The default behavior is unchanged; however, the limit
can be activated by changing the new net.inet.tcp.rack.split_limit sysctl.

Submitted by:   Peter Lei <peterlei at netflix.com>
Reported by:    jtl
Reviewed by:    lstewart (earlier version)
Security:       CVE-2019-5599

FreeBSD/src 349191stable 11, stable/11/usr.bin/vtfontcvt vtfontcvt.c

vtfontcvt: whitespace and other cleanup

MFC r343842: vtfontcvt: whitespace cleanup

MFC r348653: vtfontcvt: use VFNT_MAP_{NORMAL|BOL}_RH symbolic constants

MFC r348656: vtfontcvt: unwrap a line per style(9)

PR:            205707
Submitted by:   Dmitry Wagin

FreeBSD/src 349190head/sys/kern subr_sbuf.c

Fix typo in r349178.

Reported by:    ae
MFC after:      1 week

FreeBSD/src 349189stable 12, stable/12/usr.bin/vtfontcvt vtfontcvt.c

vtfontcvt: whitespace and other cleanup

MFC r343842: vtfontcvt: whitespace cleanup

MFC r348653: vtfontcvt: use VFNT_MAP_{NORMAL|BOL}_RH symbolic constants

MFC r348656: vtfontcvt: unwrap a line per style(9)

PR:            205707
Submitted by:   Dmitry Wagin

FreeBSD/src 349188head/stand/ofw/libofw ofw_console.c

[PPC] Fix loader input with newer QEMU versions

At least since version 4.0.0, QEMU became bug-compatible with PowerVM's
vty, by inserting a \0 after every \r. As this confuses loader's
interpreter and as a \0 coming from the console doesn't seem reasonable,
it's now being filtered at OFW console input.

Reviewed by:    jhibbits
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D20676

FreeBSD/src 349187head/share/misc bsd-family-tree

Whitespace

FreeBSD/src 349186head/sys/net iflib.c

V_ip6_forwarding and V_ipforwarding have been defined in ip6_var.h /
ip_var.h since at least 2008, so make use of those definitions here.

MFC after:      3 days

FreeBSD/src 349185head/sys/net iflib.c

Evaluating htons() at compile time is more efficient than doing ntohs()
at runtime.  This change removes a dependency on a barrel shifter pass
before branch resolution, while reducing the instruction stream size
by 9 bytes on amd64.

MFC after:      3 days

FreeBSD/src 349184head/sys/amd64/vmm/intel vtd.c

Implement VT-d capability detection on chipsets that have multiple
translation units with differing capabilities

From the author via Bugzilla:
---
When an attempt is made to passthrough a PCI device to a bhyve VM
(causing initialisation of IOMMU) on certain Intel chipsets using
VT-d the PCI bus stops working entirely. This issue occurs on the
E3-1275 v5 processor on C236 chipset and has also been encountered
by others on the forums with different hardware in the Skylake
series.

The chipset has two VT-d translation units. The issue is caused by
an attempt to use the VT-d device-IOTLB capability that is
supported by only the first unit for devices attached to the
second unit which lacks that capability. Only the capabilities of
the first unit are checked and are assumed to be the same for all
units.

Attached is a patch to rectify this issue by determining which
unit is responsible for the device being added to a domain and
then checking that unit's device-IOTLB capability. In addition to
this a few fixes have been made to other instances where the first
unit's capabilities are assumed for all units for domains they
share. In these cases a mutual set of capabilities is determined.

    [19 lines not shown]

FreeBSD/src 349183head/sys/arm64/arm64 pmap.c

Correct an error in r349122.  pmap_unwire() should update the pmap's wired
count, not its resident count.

X-MFC with:     r349122

FreeBSD/src 349182stable 12 11, stable/11/sys/contrib/ipfilter/netinet mlfk_ipl.c

MFC r348985:

Enclose a long multi-line single conditional statement in braces to
improve legibility and aesthetics.

FreeBSD/src 349181releng/11.3/sys/dev/ixl ixl_pf_main.c i40e_common.c

MFS r349163: ixl(4)/ixlv(4): Update Intel XL710 PF and VF drivers to ixl-1.11.9 and 
ixlv-1.5.8

Update the legacy (non-iflib) drivers in stable/11 with recent changes from the
Intel out-of-tree version.

Major changes:

- Support for new BASE-T device with additional link speeds (2.5G and 5G) and EEE
- Additional I2C access methods backported from ixl-iflib
- FW LLDP Agent control with sysctl added for X722 devices (this already
  existed for 710 devices)
- MAC/VLAN filters handling has been refactored
- Building and loading if_ixlv as a KLD has been fixed

This commit is not from CURRENT since the driver in 12/13 uses iflib, and the decision was
made to not use iflib in FreeBSD 11 releases.

Submitted by:   Krzysztof Galazka <krzysztof.galazka at intel.com>
Approved by:    re@ (gjb@)
Sponsored by:   Intel Corporation
Differential Revision:  https://reviews.freebsd.org/D20290

FreeBSD/src 349180stable/12/usr.sbin/jail command.c

Unmount filesystems on jail removal with "-f", to get around a situation
where the jail root vnode reference is stopping the filesystem from
unmounting, when the jail is removed by still exists in a dying state.

PR:            238517
Reported by:    matthias at harz.de

FreeBSD/src 349179head/share/mk bsd.dep.mk

Rework r349061: Don't apply guessed dependencies if there is a custom target.

This is still targeting bin/sh cyclic dependency issues.  Only apply
guessed dependencies that are explicitly set for an object (which
gnu/lib/cc/cc_tools needs) and if no custom target exists with its
own dependencies.

This was manifesting as a missing yacc.h in usr.bin/mkesdb_static when
built without -j (or -B). No actual yacc.h dependency ordering was
defined but with -j it got lucky and built fine.

Before r349061 the behavior was different for META_MODE but that logic
difference isn't needed.

X-MFC-With:     r349061
Sponsored by:   DellEMC

FreeBSD/src 349178head/sys/geom geom_kern.c, head/sys/kern imgact_elf.c subr_sbuf.c

Optimize kern.geom.conf* sysctls.

On large systems those sysctls may generate megabytes of output.  Before
this change sbuf(9) code was resizing buffer by 4KB each time many times,
generating tons of TLB shootdowns.  Unfortunately in this case existing
sbuf_new_for_sysctl() mechanism, supposed to help with this issue, is not
applicable, since all the sbuf writes are done in different kernel thread.

This change improves situation in two ways:
 - on first sysctl call, not providing any output buffer, it sets special
sbuf drain function, just counting the data and so not needing big buffer;
 - on second sysctl call it uses as initial buffer size value saved on
previous call, so that in most cases there will be no reallocation, unless
GEOM topology changed significantly.

MFC after:      1 week
Sponsored by:   iXsystems, Inc.

FreeBSD/src 349177head/share/misc bsd-family-tree

Mark NetBSD branch points
NetBSD 7.0 was a separate branch, subsequent 8.x releases did not emerge from
this branch.
Clean up minor visual nits, centre OpenBSD listing on the B, DragonFly
listings on the y.

FreeBSD/src 349176head/sys/dev/random fortuna.c

random(4): Fix a regression in short AES mode reads

In r349154, random device reads of size < 16 bytes (AES block size) were
accidentally broken to loop forever.  Correct the loop condition for small
reads.

Reported by:    pho
Reviewed by:    delphij
Approved by:    secteam(delphij)
Differential Revision:  https://reviews.freebsd.org/D20686

FreeBSD/src 349175head/usr.sbin/bhyve pci_virtio_net.c

bhyve: vtnet: fix locking on receive

The vsc_rx_ready and the RX virtqueue is protected by the rx_mtx lock.
However, pci_vtnet_ping_rxq() (currently called only once after each
device reset) accesses those without acquiring the lock.

Reviewed by:    markj
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D20609

FreeBSD/src 349174head/sys/dev/pwm pwmc.c

Handle labels specified with hints even on FDT systems.  Hints are the
easiest thing for a user to control (via loader.conf or kenv+kldload), so
handle them in addition to any label specified via the FDT data.

FreeBSD/src 349173stable 12, stable/12/contrib/elftoolchain/elfcopy sections.c ascii.c

MFC r348657:
elfcopy: Use libelftc's string table routines to build .shstrtab.

PR:     234949