PC-BSD/trueos dcb91d8stand/common module.c, stand/efi/include efiip.h

Merge pull request #425 from trueos/trueos12-update-20191024

Trueos12 update 20191024

PC-BSD/trueos 1927d8bstand/common module.c, stand/efi/include efiip.h

Merge pull request #424 from trueos/fbsd12-update-20191024

Fbsd12 update 20191024

PC-BSD/trueos 25803abcontrib/llvm/lib/CodeGen/SelectionDAG DAGCombiner.cpp, contrib/llvm/lib/Target/ARM ARMInstrMVE.td

Merge pull request #423 from trueos/fbsdm-update-20191024

Fbsdm update 20191024

PC-BSD/trueos 5459bbccontrib/llvm/lib/CodeGen/SelectionDAG DAGCombiner.cpp, contrib/llvm/lib/Target/ARM ARMInstrMVE.td

Merge pull request #422 from trueos/fbsd-head-20191024

Fbsd head 20191024

PC-BSD/trueos bd7580fstand/common module.c, stand/efi/include efiip.h

Merge branch 'stable/12' of https://github.com/freebsd/freebsd into 
trueos12-update-20191024

Signed-off-by: Martin Wilke <miwi at ixsystems.com>

PC-BSD/trueos b96ee26stand/common module.c, stand/efi/include efiip.h

Merge branch 'stable/12' of https://github.com/freebsd/freebsd into fbsd12-update-20191024

Signed-off-by: Martin Wilke <miwi at ixsystems.com>

PC-BSD/trueos c034024contrib/llvm/lib/CodeGen/SelectionDAG DAGCombiner.cpp, contrib/llvm/lib/Target/ARM ARMInstrMVE.td

Merge branch 'master' of https://github.com/freebsd/freebsd into fbsdm-update-20191024

Signed-off-by: Martin Wilke <miwi at ixsystems.com>

PC-BSD/trueos e1d1295contrib/llvm/lib/CodeGen/SelectionDAG DAGCombiner.cpp, contrib/llvm/lib/Target/ARM ARMInstrMVE.td

Merge branch 'master' of https://github.com/freebsd/freebsd into fbsd-head-20191024

Signed-off-by: Martin Wilke <miwi at ixsystems.com>

PC-BSD/trueos f730bbbsys/vm swap_pager.c

MFC r353681:
swapon_check_swzone(): use already calculated static variables.

PC-BSD/trueos f651cbcsys/netinet/tcp_stacks bbr.c

Fix a small bug in bbr when running under a VM. Basically what
happens is we are more delayed in the pacer calling in so
we remove the stack from the pacer and recalculate how
much time is left after all data has been acknowledged. However
the comparision was backwards so we end up with a negative
value in the last_pacing_delay time which causes us to
add in a huge value to the next pacing time thus stalling
the connection.

Reported by:    vm2.finance at gmail.com

PC-BSD/trueos db3b0afstand/forth support.4th

MFC r352420: loader_4th: scan_buffer can leave empty string on stack

When the file processing is done, we will have string with lenght 0 in stack
and we will attempt to
allocate 0 bytes.

PC-BSD/trueos 6769936stand/efi/libefi efinet.c, stand/libsa net.h

MFC r349928: Allow efi loader to get network params from uboot

Summary:
efi loader does not work with static network parameters. It always uses
BOOTP/DHCP and also uses RARP as a fallback.  Problems with DHCP servers can
cause the loader to fail to populate network parameters.

PC-BSD/trueos b6d3bcfstand/forth menu.4th menu.rc, stand/lua color.lua menu.lua

MFC r352599, r352601: loader interp color reset fixes

r352599:
loader_4th: menu items need to reset color attribute, not switch to white

Forth menu kernel and BE entries, instead of resetting the color attribute,
are switching to white color.

r352601:
loader_lua: lua color changes should end with reset

The color change should have reset sequence, not switch to white.

PC-BSD/trueos 6fd94c0stand/efi/loader copy.c

MFC r349471, r351166: Tweak EFI_STAGING_SIZE

r349471:
Increase EFI_STAGING_SIZE to 100MB on x64

To avoid failures when the large 18MB nvidia.ko module is being loaded,
increase EFI_STAGING_SIZE from 64MB to 100MB on x64 systems.
Leave the other platforms at 64MB.

r351166:
Reduce size of EFI_STAGING_SIZE to 32 on arm

Reduce the size of the EFI_STAGING area we allocate on arm to 32. On arm SBC
such as the NanoPi-NEOLTS the staging area allocation will fail on the 256MB
model with a staging size of 64.

PC-BSD/trueos fa0e2d9stand/efi/include efidevp.h, stand/efi/include/Protocol Http.h Ip4Config2.h

MFC EFI HTTP Support: r349349, r349395, r349404, r349564-r349566, r349613,
r350444

r349349:
loader: add HTTP support using UEFI

Add support for an HTTP "network filesystem" using the UEFI's HTTP
stack.

This also supports HTTPS, but TianoCore EDK2 implementations currently
crash while fetching loader files.
Only IPv4 is supported at the moment. IPv6 support is planned for a
follow-up changeset.

Note that we include some headers from the TianoCore EDK II project in
stand/efi/include/Protocol verbatim, including links to the license instead
of including the full text because that's their preferred way of
communicating it, despite not being normal FreeBSD project practice.

r349395:
Disconnect EFI HTTP support

The EFI HTTP code has been causing boot failures for people, so disable it
while a fix is being worked on.


    [33 lines not shown]

PC-BSD/trueos 76347a5stand loader.mk, stand/common vdisk.c help.common

MFC r346759: loader: implement map-vdisk and unmap-vdisk commands

illumos update: https://www.illumos.org/issues/10598

Add map-vdisk and unmap-vdisk commands to create virtual disk interface on
top of file. This will allow to use disk image from file system to load and
start the kernel.

By mapping file, we create vdiskX device, the device will be listed by lsdev
[-v] and can be accessed directly as ls vdisk0p1:/path or can be used as
value for currdev variable.

vdisk strategy function does not use bcache as we have bcache used with
backing file. vdisk can be unmapped when all consumers have closed the open
files.

In first iteration we do not support the zfs images because zfs pools do
keep the device open (there is no "zpool export" mechanism). Adding zfs
support is relatively simple, we just need to run zfs disk probe after
mapping is done.

PC-BSD/trueos 8cb8834stand/common disk.c disk.h, stand/efi/loader main.c

MFC r345477, r346675, r346984, r348748

r345477:
Distinguish between "no partition" and "choose best partition" with a
constant.

The values of the d_slice and d_partition fields of a disk_devdesc have a
few values with special meanings in the disk_open() routine. Through various
evolutions of the loader code over time, a d_partition value of -1 has
meant both "use the first ufs partition found in the bsd label" and "don't
open a bsd partition at all, open the raw slice."

This defines a new special value of -2 to mean open the raw slice, and it
gives symbolic names to all the special values used in d_slice and
d_partition, and adjusts all existing uses of those fields to use the new
constants.

The phab review for this timed out without being accepted, but I'm still
citing it below because there is useful commentary there.

r346675:
Restore the ability to open a raw disk or partition in loader(8).

The disk_open() function searches for "the best partition" when slice and
partition information is not provided as part of the device name.  As of

    [47 lines not shown]

PC-BSD/trueos 92b9f22sys/powerpc/include resource.h, sys/powerpc/mpc85xx pci_mpc85xx.c

powerpc/booke: Simplify the MPC85XX PCIe root complex driver

Summary:
Due to bugs in the enumeration code, fsl_pcib_init() was not configuring
sub-bridges properly, so devices hanging off a separate bridge would not
be found.  Since the generic PCI code already supports probing child
buses, just delete this code and initialize only the device itself,
letting the generic code handle all the additional probing and
initializing.

This also deletes setup for some PCI peripherals found on some MPC85XX
evaluation boards.  The code can be resurrected if needed, but overly
complicated this code in the first place.

Reviewed by:    bdragon
Differential Revision:  https://reviews.freebsd.org/D22050

PC-BSD/trueos 9f2086fstand/efi/loader/arch/arm exec.c, stand/efi/loader/arch/i386 efimd.c elf32_freebsd.c

MFC r344560, r344718

r344560:
stand: Remove unused i386 EFI MD bits

r328169 removed the copy of bootinfo that would've made this somewhat
functional. However, this is irrelevant- earlier work in r292338 was done to
exit boot services in the MI bi_load() rather than having N copies of the
GetMemoryMap/ExitBootServices dance.

i386 never quite caught up to that; ldr_enter was still being called but
the prereq for that, ldr_bootinfo, was no longer. As a consequence, this
ExitBootServices() was being called with a mapkey=0, clearly bogus, and
reportedly breaking the boot in some instances.

r344718:
EFI: don't call printf after ExitBootServices, since it uses Boot Services

ExitBootServices terminates all boot services including console access.
Attempting to call printf afterwards can result in a crash, depending on the
implementation.

Move any printf statements to before we call bi_load, and remove any that
depend on calling bi_load first.

PC-BSD/trueos 3e9ca4bsys/kern kern_mib.c

MFC r349344: Add missing include of sys/boot.h

This change was dropped out in a rebase and I didn't catch that before
I committed.

PC-BSD/trueos 1a0321dstand/common interp_forth.c

MFC r341276:
When handling CMD_CRIT error set command_errmsg to NULL after we dump it
out, so that it does not result in error message printed twice.

OK load doodoo
can't find 'doodoo'
can't find 'doodoo'
OK

PC-BSD/trueos 75282fastand/libsa Makefile crc32_libkern.c

MFC (proactively; not required yet) r339673: Fix stand/ build after r339671.

ffs_subr.c requires calculate_crc32c() from libkern.  Unfortunately we
cannot just add libkern/crc32.c to libstand because crc32.o is already
compiled from contrib/zlib/crc32.c. Use the include trick to rename
the source.

Note that libstand also provides crc32.c which seems to be unused.

PC-BSD/trueos 3e7eb92stand/libsa ip.c udp.c

MFC r339970: Remove unnecessary include from libstand.

PC-BSD/trueos dfc5571stand/i386 Makefile.inc

MFC r340834: Disable build-id in i386 binary boot components

A user may enable build-id for all builds by adding
LDFLAGS=-Wl,--build-id=sha1 to /etc/make.conf.  In this case the build-id
note ends added up to mbr and pmbr's .text, which makes it too large (it
ends up being 532 bytes). To avoid this explicitly turn off build-id for
these components.

PC-BSD/trueos 2c6d50cstand/efi/loader main.c

MFC r353697: Allow loader.efi to identify non-standard boot setup

PATH_BOOTABLE_TOKEN can be set to a non-standard
path that identifies a device as bootable.

PC-BSD/trueos 2d3ef10stand/efi/boot1 boot1.c

MFC r353544: boot1.efi: provide generic exit() and stub getchar()

PC-BSD/trueos f392e73stand/libsa zalloc.c

MFC r352548: loader: fix typo in zalloc.

PC-BSD/trueos fed52a6stand defs.mk, stand/i386/loader Makefile

MFC r352032: loader: --gc-sections needs sections to work with

--gc-sections is not really useful unless we generate sections with
-ffunction-sections -fdata-sections

While there, i386/loader would win from --gc-sections too.

PC-BSD/trueos a85147c

MFC r352037, r352145: Remove empty zfsloader directories
DeltaFile
+0-00 files

PC-BSD/trueos aa6a2bestand/efi/libefi efi_console.c

MFC r351695-r351696

r351695:
loader.efi: some systems do not translate scan code 0x8 to backspace

Add scancode translation for backspace.

r351696:
loader.efi: use and prefer coninex interface

Add support for EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL.

PC-BSD/trueos 3ea969estand/common paths.h, stand/efi/loader main.c

MFC r349343: Use a common kernel path between loader and kernel

PC-BSD/trueos 1cc1515stand/efi/libefi efinet.c

MFC r349201: efinet: Defer exclusively opening the network handles

Don't commit to exclusive access to the network device handle by
efinet until the loader has decided to load something through the
network. This allows for the possibility of other users of the
network device.

PC-BSD/trueos c6af607stand/libsa/zfs zfsimpl.c

MFC r349217: Tell loader to ignore newer features enabled on the root pool.

There are many new features in ZoF. Most, if not all, do not effect read
only usage.
Encryption in particular is enabled at the pool level but used at the
dataset level.
The loader obviously will not be able to boot if the boot dataset is
encrypted, but
should not care if some other dataset in the root pool is encrypted.

PC-BSD/trueos 5b19b78stand/efi/boot1 boot1.c proto.c, stand/efi/gptboot proto.c drv.h

MFC r348658-r348659, r348674-r348675, r348678, r348760, r348766, r348768,
r348811-r348812, r348814, r349008

r348658:
Introduce efi_devpath_same_disk

This is like efi_devpath_match, but allows differing device media
paths. Those just specify the partition information.

r348659:
Use newly minted efi_devpath_same_disk() instead of
efi_devpath_match(). This fixes a regression in r347193.

r348674:
Don't shadow a global zfsmount variable.

r348675:
ufs_module.c can't currently be compiled with -Wcast-align, but the
code is safe enough. Turn off the warning for now until I can find the
right construct to silence it in the code.

r348678:
Eliminate unused uuid parameters from gptread and gptread_table.  We
only need it for the gptfind() function, where it's used.


    [41 lines not shown]

PC-BSD/trueos 787f459stand/common interp_simple.c

MFC r348499: Fix comment parsing in interp_simple.c

loader.rc has comment lines without a trailing space, which get
interpreted as commands. Avoid this by only matching against the
backslash character.

PC-BSD/trueos e19c62fstand/libsa/zfs zfsimpl.c

MFC r348381: Unexpand be32dec().

PC-BSD/trueos d316a65stand/libsa/zfs zfsimpl.c

MFC r348352, r348354

r348352:
loader: zfs_alloc and zfs_free should use panic

The zfs alloc and free code print out the error and get stuck in infinite
loop; use panic() instead.

r348354:
loader: malloc+memset is calloc in spa_create

Replace malloc + memset pair with calloc.

PC-BSD/trueos c8bb502stand/common module.c, stand/fdt fdt_loader_cmd.c fdt_platform.h

MFC r348194, r348196, r348204, r348207: loader/fdt pnpmatch

r348194:
loader: Add pnp functions for autoloading modules based on linker.hints

This adds some new commands to loader :

- pnpmatch
   This takes a pnpinfo string as argument and tries to find a kernel module
   associated with it. -v and -d option are available and are the same as in
   devmatch (v is verbose, d dumps the hints).
- pnpload
   This takes a pnpinfo string as argument and tries to load a kernel module
   associated with it.
- pnpautoload
   This will attempt to load every kernel module for each buses. Each buses are
   probed, the probe function will generate pnpinfo string and load kernel module
   associated with it if it exists.

Only simplebus for FDT system is implemented for now.
Since we need the dtb and overlays to be applied before searching the tree
fdt_devmatch_next will load and apply the dtb + overlays.

All the pnp parsing code comes from devmatch and is the same at 99%.


    [8 lines not shown]

PC-BSD/trueos 065b960stand/common part.c disk.c, stand/i386/libi386 biosdisk.c vidconsole.c

MFC r345066, r347219-r347220, r347223, r347388-r347389, r347391, r347393,
r347553, r348040

r345066:
stand: Improve some debugging experience

Some of these files using <FOO>_DEBUG defined a DEBUG() macro to serve as a
debug-printf. -DDEBUG is useful to enable some debugging output across
multiple ELF/common parts, so switch the DEBUG-as-printf macros over to
something more like DPRINTF that is more commonly used for this kind of
thing and less likely to conflict.

userboot/elf64_freebsd debugging also assumed %llx for uint64; use PRIx64
instead.

r347219:
loader: use safer DPRINTF body for non-debug case

r347220:
loader: bcache code does not need to check argument for free()

r347223:
command_bcache() does not use argv

Therefore mark argv __unused.

    [42 lines not shown]

PC-BSD/trueos 346d905stand/efi/boot1 boot1.c Makefile, stand/efi/include efilib.h

MFC r346702-r346704, r346879-r346881, r347023, r347059-r347062,
r347193-r347194, r347201

r346702:
Add efi_freebsd_delenv

Add a wrapper around efi_delenv akin to efi_freebsd_getenv and
efi_getenv.

r346703:
Move initialization of the block device handles earlier (we're just
snagging them from UEFI BIOS). Call the device type init routines
earlier as well, as they don't depend on how the console is
setup. This will allow us to read files earlier in boot, so any rare
error messages that this might move only to the EFI console will be an
acceptable price to pay. Also tweak the order of has_kbd so it resides
next to the rest of the console code. It needs to be after we initialize
the buffer cache.

r346704:
Add the proper range of years for Netflix's copyright on this
file. Note that I wrote it.

r346879:
Read in and parse /efi/freebsd/loader.env from the boot device's

    [87 lines not shown]

PC-BSD/trueos e575056stand/i386/zfsboot zfsboot.c

MFC r346969: zfsboot: to detect disk size, use GPT information first

If we do have GPT on disk, read the disk size from it and do not
call int13.

Since int13 does report bogus informatiopn too often, rather trust the
partition table. We are using the same strategy with loader.

PC-BSD/trueos bf084e1stand/efi/loader autoload.c, stand/fdt fdt_loader_cmd.c fdt_platform.h

MFC r346701: loader: fdt: Add fdt_is_setup function

When efi_autoload is called it will call fdt_setup_fdtp which setup the
dtb and overlays. If a user already loaded at dtb or overlays or just
printed the efi provided dtb, this will re-setup everything and also
re-applying the overlays.
Test that everything is setup before doing it again.

PC-BSD/trueos 4c74867stand/efi/include efidevp.h efilib.h, stand/efi/libefi devpath.c efienv.c

MFC r346345-r346346, r346353, r346407-r346409, r346430, r346573, r346575

r346345:
Add a more generic efi_setenv function.

efi_setenv allows any UEFI variable to be set.

r346346:
Add efi_delenv

Add an interface to remove / delete UEFI variables.

r346353:
Minor tweak to the debug

Make it clear we're loading from UFS.

r346407:
Add define for CONST.

Newer interfaces take CONST parameters, so define CONST to minimize
differences between our headers and the standards docs.

r346408:
Add UEFI definitions related to converting string to DEVICE_PATH

    [46 lines not shown]

PC-BSD/trueos c246f8fstand/common module.c, stand/efi/loader main.c

MFC r345998-r346002, r346007-r346008: various loader improvements

r345998:
loader: malloc+bzero is calloc

Replace malloc+bzero in module.c with calloc.

r345999:
loader: file_addmodule should check for memory allocation

strdup() can return NULL.

r346000:
loader: remove pointer checks before free() in module.c

free() does check for NULL argument, remove duplicate checks.

r346001:
loader: file_addmetadata() should check for memory allocation

malloc() can return NULL.

r346002:
loader: mod_loadkld() error: we previously assumed 'last_file' could be null


    [15 lines not shown]

PC-BSD/trueos b6c75ffstand/common load_elf.c

MFC r345330: loader: fix loading of kernels with . in path

The loader indended to search the kernel file name (only) for . but
instead searched the entire path, so paths like
"boot/test.elfv2/kernel" would not work.

PC-BSD/trueos 1d9dc30stand/common module.c

MFC r344892: stand/common/module: Apply style(9)
DeltaFile
+725-724stand/common/module.c
+725-7241 files

PC-BSD/trueos f8264b9stand/common part.c

MFC r344268: loader: ptable_close() should check its argument

PC-BSD/trueos a6d07bbstand/i386/libfirewire firewire.c

MFC r342865: biospci_write_config args were backwards

biospci_write_config args swapped length and value to write. Some
hardware coped just fine, while other hardware had issues.

PR:            155441

PC-BSD/trueos b92f67cstand/efi/include efiip.h efitcp.h, stand/efi/libefi env.c

MFC r342054-r342055, r342742: loader diagnostics

r342054:
Print an error message in efi_main.c if we can't allocate memory for the
heap

With the default Qemu parameters, only 128MB RAM gets given to a VM. This
causes
the loader to be unable to allocate the 64MB it needs for the heap. This
change
makes the cause of the error more obvious.

r342055:
Cast error message in efi_main.c to CHAR16* to avoid build error

r342742:
loader.efi: efi variable rework and lsefi command added

This update does add diag and debug capabilities to interpret the efi
variables, configuration and protocols (lsefi).

The side effect is that we add/update bunch of related headers.

PC-BSD/trueos d95525estand defs.mk

MFC r341433: Move inclusion of src.opts.mk later.

src.opts.mk includes bsd.own.mk. This in turn defines CTFCONVERT_CMD
depending on the MK_CTF value. We then set MK_CTF to no, which has no
real effect. The solution is to set all the MK_foo values before
including src.opts.mk.

This should stop the cdboot binary from exploding in size for releases built
WITH_CTF=yes in src.conf.
DeltaFile
+8-4stand/defs.mk
+8-41 files