Displaying 1 50 of 4,978 commits (0.009s)

ZFS on Linux — config deb.am tgz.am

Fix config issues: frame size and headers

1. With various (debug and/or tracing?) kernel options enabled it's
possible for 'struct inode' and 'struct super_block' to exceed the
default frame size, leaving errors like this in config.log:

build/conftest.c:116:1: error: the frame size of 1048 bytes is larger
than 1024 bytes [-Werror=frame-larger-than=]

Fix this by removing the frame size warning for config checks

2. Without the correct headers included, it's possible for declarations
to be missed, leaving errors like this in the config.log:

build/conftest.c:131:14: error: ‘struct nameidata’ declared inside
parameter list [-Werror]

Fix this by adding appropriate headers.

Note: Both these issues can result in silent config failures because
the compile failure is taken to mean "this option is not supported by
this kernel" rather than "there's something wrong with the config
test". This can lead to something merely annoying (compile failures) to
something potentially serious (miscompiled or misused kernel primitives
or functions). E.g. the fixes included here resulted in these

    [9 lines not shown]
Delta File
+4 -4 config/deb.am
+2 -2 config/tgz.am
+1 -1 config/user-libssl.m4
+1 -1 config/kernel-get-link.m4
+1 -1 config/kernel-vm_node_stat.m4
+1 -1 config/kernel.m4
+5 -1 5 files not shown
+15 -11 11 files

ZFS on Linux — module/lua llimits.h

Address objtool check failures in lua module

The use of void __attribute__((noreturn)) in kernel builds
was causing lots of warnings if CONFIG_STACK_VALIDATION
is active. For now we just remove this attribute to achieve
clean builds for the Lua module. There was no significant
increase in the time to run the full channel_program ZTS tests.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Don Brady <don.brady at delphix.com>
Closes #7173
Delta File
+3 -1 module/lua/llimits.h
+3 -1 1 file

ZFS on Linux — man/man8 zinject.8

Clarify zinject(8) explanation of -e

Error injection of EIO or ENXIO simply sets the zio's io_error value,
rather than preventing the read or write from occurring.  This is
important information as it affects how the probes must be used.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Olaf Faaland <faaland1 at llnl.gov>
Closes #7172
Delta File
+3 -0 man/man8/zinject.8
+3 -0 1 file

ZFS on Linux — . configure.ac, cmd/zdb zdb.c

Project Quota on ZFS

Project quota is a new ZFS system space/object usage accounting
and enforcement mechanism. Similar as user/group quota, project
quota is another dimension of system quota. It bases on the new
object attribute - project ID.

Project ID is a numerical value to indicate to which project an
object belongs. An object only can belong to one project though
you (the object owner or privileged user) can change the object
project ID via 'chattr -p' or 'zfs project [-s] -p' explicitly.
The object also can inherit the project ID from its parent when
created if the parent has the project inherit flag (that can be
set via 'chattr +P' or 'zfs project -s [-p]').

By accounting the spaces/objects belong to the same project, we
can know how many spaces/objects used by the project. And if we
set the upper limit then we can control the spaces/objects that
are consumed by such project. It is useful when multiple groups
and users cooperate for the same project, or a user/group needs
to participate in multiple projects.

Support the following commands and functionalities:

zfs set projectquota at project

    [43 lines not shown]
Delta File
+318 -12 cmd/zfs/zfs_main.c
+297 -30 module/zfs/zfs_vnops.c
+252 -70 module/zfs/zfs_vfsops.c
+295 -0 cmd/zfs/zfs_project.c
+212 -0 man/man8/zfs.8
+186 -3 module/zfs/sa.c
+2,959 -165 76 files not shown
+4,519 -280 82 files

ZFS on Linux — cmd/zfs zfs_main.c, lib/libzfs libzfs_sendrecv.c

'zfs receive' fails with "dataset is busy"

Receiving an incremental stream after an interrupted "zfs receive -s"
fails with the message "dataset is busy": this is because we still have
the hidden clone ../%recv from the resumable receive.

Improve the error message suggesting the existence of a partially
complete resumable stream from "zfs receive -s" which can be either
aborted ("zfs receive -A") or resumed ("zfs send -t").

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7129
Closes #7154

ZFS on Linux — contrib/initramfs Makefile.am, contrib/initramfs/conf.d zfs

contrib/initramfs: add missing conf.d/zfs

When upgrading from the distribution-provided zfs-initramfs package on
root-on-zfs Ubuntu and Debian the system may fail to boot: this change
adds the missing initramfs configuration file.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Richard Laager <rlaager at wiktel.com>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7158

ZFS on Linux — module/zfs mmp.c

mmp should use a fixed tag for spa_config locks

mmp_write_uberblock() and mmp_write_done() should the same tag
for spa_config_locks.

Reviewed-by: Olaf Faaland <faaland1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Sanjeev Bagewadi <sanjeev.bagewadi at gmail.com>
Closes #6530
Closes #7155
Delta File
+3 -2 module/zfs/mmp.c
+3 -2 1 file

ZFS on Linux — include/sys zap_leaf.h, module/zfs zfs_vnops.c zap_micro.c

Handle zap_add() failures in mixed case mode

With "casesensitivity=mixed", zap_add() could fail when the number of
files/directories with the same name (varying in case) exceed the
capacity of the leaf node of a Fatzap. This results in a ASSERT()
failure as zfs_link_create() does not expect zap_add() to fail. The fix
is to handle these failures and rollback the transactions.

Reviewed by: Matt Ahrens <mahrens at delphix.com>
Reviewed-by: Chunwei Chen <david.chen at nutanix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Sanjeev Bagewadi <sanjeev.bagewadi at gmail.com>
Closes #7011
Closes #7054

ZFS on Linux — include/linux vfs_compat.h

Linux 4.11 compat: avoid refcount_t name conflict

Related to commit 4859fe796, when directly using the kernel's
refcount functions in kernel compatibility code do not map
refcount_t to zfs_refcount_t.  This leads to a type mismatch.

Longer term we should consider renaming refcount_t to
zfs_refcount_t in the zfs code base.

Reviewed-by: Olaf Faaland <faaland1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Chunwei Chen <david.chen at nutanix.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7148
Delta File
+6 -0 include/linux/vfs_compat.h
+6 -0 1 file

ZFS on Linux — config kernel-inode-set-iversion.m4 kernel.m4, include/linux vfs_compat.h

Linux 4.16 compat: inode_set_iversion()

A new interface was added to manipulate the version field of an
inode.  Add a inode_set_iversion() wrapper for older kernels and
use the new interface when available.

The i_version field was dropped from the trace point due to the
switch to an atomic64_t i_version type.

Reviewed-by: Olaf Faaland <faaland1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Chunwei Chen <david.chen at nutanix.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7148

ZFS on Linux — include/sys zio.h, module/zfs zio.c

OpenZFS 8857 - zio_remove_child() panic due to already destroyed parent zio

PROBLEM
=======
It's possible for a parent zio to complete even though it has children
which have not completed. This can result in the following panic:
    > $C
    ffffff01809128c0 vpanic()
    ffffff01809128e0 mutex_panic+0x58(fffffffffb94c904, ffffff597dde7f80)
    ffffff0180912950 mutex_vector_enter+0x347(ffffff597dde7f80)
    ffffff01809129b0 zio_remove_child+0x50(ffffff597dde7c58, ffffff32bd901ac0,
    ffffff3373370908)
    ffffff0180912a40 zio_done+0x390(ffffff32bd901ac0)
    ffffff0180912a70 zio_execute+0x78(ffffff32bd901ac0)
    ffffff0180912b30 taskq_thread+0x2d0(ffffff33bae44140)
    ffffff0180912b40 thread_start+8()
    > ::status
    debugging crash dump vmcore.2 (64-bit) from batfs0390
    operating system: 5.11 joyent_20170911T171900Z (i86pc)
    image uuid: (not set)
    panic message: mutex_enter: bad mutex, lp=ffffff597dde7f80
    owner=ffffff3c59b39480 thread=ffffff0180912c40
    dump content: kernel pages only
The problem is that dbuf_prefetch along with l2arc can create a zio tree
which confuses the parent zio and allows it to complete with while children

    [52 lines not shown]
Delta File
+29 -20 module/zfs/zio.c
+11 -0 include/sys/zio.h
+40 -20 2 files

ZFS on Linux — cmd/zfs zfs_main.c, include libzfs_core.h

OpenZFS 8677 - Open-Context Channel Programs

Authored by: Serapheim Dimitropoulos <serapheim at delphix.com>
Reviewed by: Matt Ahrens <mahrens at delphix.com>
Reviewed by: Chris Williamson <chris.williamson at delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov at delphix.com>
Approved by: Robert Mustacchi <rm at joyent.com>
Ported-by: Don Brady <don.brady at delphix.com>

We want to be able to run channel programs outside of synching
context. This would greatly improve performance for channel programs
that just gather information, as they won't have to wait for synching
context anymore.

=== What is implemented?

This feature introduces the following:
- A new command line flag in "zfs program" to specify our intention
  to run in open context. (The -n option)
- A new flag/option within the channel program ioctl which selects
  the context.
- Appropriate error handling whenever we try a channel program in
  open-context that contains zfs.sync* expressions.
- Documentation for the new feature in the manual pages.


    [32 lines not shown]

ZFS on Linux — include/sys zfs_ioctl.h, module/zfs zfs_ioctl.c dsl_destroy.c

OpenZFS 8604 - Simplify snapshots unmounting code

Authored by: Serapheim Dimitropoulos <serapheim at delphix.com>
Reviewed by: Matt Ahrens <mahrens at delphix.com>
Reviewed by: George Wilson <george.wilson at delphix.com>
Reviewed by: Andy Stormont <astormont at racktopsystems.com>
Approved by: Robert Mustacchi <rm at joyent.com>
Ported-by: Don Brady <don.brady at delphix.com>

Every time we want to unmount a snapshot (happens during snapshot
deletion or renaming) we unnecessarily iterate through all the
mountpoints in the VFS layer (see zfs_get_vfs).

The current patch completely gets rid of that code and changes
the approach while keeping the behavior of that code path the
same. Specifically, it puts a hold on the dataset/snapshot and
gets its vfs resource reference directly, instead of linearly
searching for it. If that reference exists we attempt to amount
it.

With the above change, it became obvious that the nvlist
manipulations that we do (add_boolean and add_nvlist) take a
significant amount of time ensuring uniqueness of every new
element even though they don't have too. Thus, we updated the
patch so those nvlists are not trying to enforce the uniqueness

    [8 lines not shown]

ZFS on Linux — lib/libzpool Makefile.am, module/lua lundump.c lbitlib.c

Increase code coverage for Lua libraries

Add test coverage for lua libraries
Remove dead code in Lua implementation

Signed-off-by: Don Brady <don.brady at delphix.com>

ZFS on Linux — tests/runfiles linux.run, tests/zfs-tests/tests/functional/channel_program/synctask_core tst.list_user_props.ksh Makefile.am

Add basic functional tests for zcp user properties

Signed-off-by: Don Brady <don.brady at delphix.com>

ZFS on Linux — include/sys dsl_dataset.h zcp.h, lib/libzfs_core libzfs_core.c

OpenZFS 8600 - ZFS channel programs - snapshot

Authored by: Chris Williamson <chris.williamson at delphix.com>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed by: John Kennedy <john.kennedy at delphix.com>
Reviewed by: Brad Lewis <brad.lewis at delphix.com>
Approved by: Robert Mustacchi <rm at joyent.com>
Ported-by: Don Brady <don.brady at delphix.com>

ZFS channel programs should be able to create snapshots.
In addition to the base snapshot functionality, this entails extra
logic to handle edge cases which were formerly not possible, such as
creating then destroying a snapshot in the same transaction sync.

OpenZFS-issue: https://www.illumos.org/issues/8600
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/68089b8b

ZFS on Linux — include/sys dsl_dataset.h, man/man8 zfs-program.8

OpenZFS 8592 - ZFS channel programs - rollback

Authored by: Brad Lewis <brad.lewis at delphix.com>
Reviewed by: Chris Williamson <chris.williamson at delphix.com>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Approved by: Robert Mustacchi <rm at joyent.com>
Ported-by: Don Brady <don.brady at delphix.com>

ZFS channel programs should be able to perform a rollback.

OpenZFS-issue: https://www.illumos.org/issues/8592
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/d46b5ed6

ZFS on Linux — man/man8 zfs-program.8, module/zfs zcp.c

OpenZFS 8605 - zfs channel programs fix zfs.exists

Authored by: Chris Williamson <chris.williamson at delphix.com>
Reviewed by: Paul Dagnelie <pcd at delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel at delphix.com>
Reviewed by: Matt Ahrens <mahrens at delphix.com>
Approved by: Robert Mustacchi <rm at joyent.com>
Ported-by: Don Brady <don.brady at delphix.com>

zfs.exists() in channel programs doesn't return any result, and should
have a man page entry. This patch corrects zfs.exists so that it
returns a value indicating if the dataset exists or not. It also adds
documentation about it in the man page.

OpenZFS-issue: https://www.illumos.org/issues/8605
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1e85e111

ZFS on Linux — . configure.ac, cmd/zfs zfs_main.c

OpenZFS 7431 - ZFS Channel Programs

Authored by: Chris Williamson <chris.williamson at delphix.com>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed by: George Wilson <george.wilson at delphix.com>
Reviewed by: John Kennedy <john.kennedy at delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel at delphix.com>
Approved by: Garrett D'Amore <garrett at damore.org>
Ported-by: Don Brady <don.brady at delphix.com>
Ported-by: John Kennedy <john.kennedy at delphix.com>

OpenZFS-issue: https://www.illumos.org/issues/7431
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/dfc11533

Porting Notes:
* The CLI long option arguments for '-t' and '-m' don't parse on linux
* Switched from kmem_alloc to vmem_alloc in zcp_lua_alloc
* Lua implementation is built as its own module (zlua.ko)
* Lua headers consumed directly by zfs code moved to 'include/sys/lua/'
* There is no native setjmp/longjump available in stock Linux kernel.
  Brought over implementations from illumos and FreeBSD
* The get_temporary_prop() was adapted due to VFS platform differences
* Use of inline functions in lua parser to reduce stack usage per C call
* Skip some ZFS Test Suite ZCP tests on sparc64 to avoid stack overflow
Delta File
+1,643 -0 module/lua/lparser.c
+1,357 -0 module/zfs/zcp.c
+1,346 -0 module/lua/lapi.c
+1,218 -0 module/lua/lgc.c
+1,037 -0 module/lua/lstrlib.c
+960 -0 module/lua/lvm.c
+19,494 -272 173 files not shown
+27,055 -272 179 files

ZFS on Linux — contrib/dracut/90zfs zfs-load-key.sh.in

Only run pre-mount hook zfs-load-key on systemd

Reviewed-by: Kash Pande <kash at tripleback.net>
Reviewed-by: bunder2015 <omfgbunder at gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Matthew Thode <mthode at mthode.org>
Closes #7136
Closes #7140

ZFS on Linux — . Makefile.am, tests/zfs-tests/tests/functional/casenorm casenorm.kshlib

Verify ZFS Test Suite scripts executability

This change adds a make target 'testscheck' which verifies all ksh test
scripts, part of the ZFS Test Suite, have their executable bit set:
this should avoid adding new test scripts that cannot be executed by
the test-runner.py which would result in the following warning message:

   Warning: Test removed from TestGroup because it failed verification.

This also verifies both *.cfg and *.kshlib files used in the ZFS Test
Suite are not executable.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7131
Delta File
+9 -1 Makefile.am
+0 -0 tests/zfs-tests/tests/functional/casenorm/casenorm.kshlib
+0 -0 tests/zfs-tests/tests/functional/cli_root/zpool_events/zpool_events.kshlib
+0 -0 tests/zfs-tests/tests/functional/events/events_common.kshlib
+0 -0 tests/zfs-tests/tests/functional/rsend/send_encrypted_files.ksh
+0 -0 tests/zfs-tests/tests/functional/zvol/zvol_misc/zvol_misc_common.kshlib
+0 -0 tests/zfs-tests/tests/functional/cli_root/zpool_reopen/zpool_reopen.cfg
+9 -1 7 files

ZFS on Linux — man/man5 zfs-module-parameters.5, module/zfs arc.c

Remove deprecated zfs_arc_p_aggressive_disable

zfs_arc_p_aggressive_disable is no more. This PR removes docs
and module parameters for zfs_arc_p_aggressive_disable.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Richard Elling <Richard.Elling at RichardElling.com>
Closes #7135

ZFS on Linux — contrib/dracut/90zfs zfs-load-key.sh .gitignore

Add zfs-load-key.sh to .gitignore

The generated zfs-load-key.sh file should have been added to
the .gitignore file as part of commit 7da8f8d8.  And the
generated file should not be included in the repo.

Reviewed-by: Matthew Thode <prometheanfire at gentoo.org>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7134

ZFS on Linux — cmd/zpool zpool_main.c

Fix `zpool status` overflow on fast scrubs

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: Richard Elling <Richard.Elling at RichardElling.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: DHE <git at dehacked.net>
Closes #7133
Delta File
+1 -1 cmd/zpool/zpool_main.c
+1 -1 1 file

ZFS on Linux — config zfs-build.m4

Fix default libdir for Debian/Ubuntu

The distribution provided architecture specific RPM macro files
for x86_64 and other architectures on Debian/Ubuntu specify the
wrong default libdir install location.  When building deb packages
override _lib with the correct location.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Reviewed by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7083
Closes #7101
Delta File
+13 -0 config/zfs-build.m4
+13 -0 1 file

ZFS on Linux — man/man5 zfs-module-parameters.5, module/zfs arc.c

Adjust ARC prefetch tunables to match docs

Currently, the ARC exposes 2 tunables (zfs_arc_min_prefetch_ms
and zfs_arc_min_prescient_prefetch_ms) which are documented
to be specified in milliseconds. However, the code actually
uses the values as though they were in seconds. This patch
adjusts the code to match the names and documentation of the
tunables.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #7126

ZFS on Linux — cmd/ztest ztest.c

Set persistent ztest failure mode

In order to reliably detect deadlocks in the create and import
path ztest should set the failure mode property.  This ensures
that the pool is always using the correct failmode behavior.

Removed insidious use of local variable in MAXFAULTS macro.

Converted VERIFY() to VERIFY0() where appropriate.

Reviewed-by: Tim Chase <tim at chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7111
Delta File
+19 -17 cmd/ztest/ztest.c
+19 -17 1 file

ZFS on Linux — module/zfs qat_compress.c

Bug fix in qat_compress.c for vmalloc addr check

Remove the unused vmalloc address check, and function mem_to_page
will handle the non-vmalloc address when map it to a physical
address.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Weigang Li <weigang.li at intel.com>
Closes #7125
Delta File
+0 -4 module/zfs/qat_compress.c
+0 -4 1 file

ZFS on Linux — module/zfs dsl_crypt.c

Fix hash_lock / keystore.sk_dk_lock lock inversion

The keystore.sk_dk_lock should not be held while performing I/O.
Drop the lock when reading from disk and update the code so
they the first successful caller adds the key.

Improve error handling in spa_keystore_create_mapping_impl().

Reviewed by: Thomas Caputi <tcaputi at datto.com>
Reviewed-by: RageLtMan <rageltman at sempervictus>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7112
Closes #7115
Delta File
+39 -48 module/zfs/dsl_crypt.c
+39 -48 1 file

ZFS on Linux — rpm/generic zfs.spec.in

Fix systemd_ RPM macros usage on Debian-based distributions

Debian-based distributions do not seem to provide RPM macros for
dealing with systemd pre- and post- (un)install actions: this results
in errors when installing or upgrading .deb packages because the
resulting control scripts contain the following unresolved macros:

 * %systemd_post
 * %systemd_preun
 * %systemd_postun

Fix this by providing default values for postinstall, preuninstall and
postuninstall scripts when these macros are not defined.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7074
Closes #7100
Delta File
+20 -1 rpm/generic/zfs.spec.in
+20 -1 1 file

ZFS on Linux — cmd/zdb zdb.c, tests/runfiles linux.run

Fix zdb -ed on objset for exported pool

zdb -ed on objset for exported pool would failed with:
  failed to own dataset 'qq/fs0': No such file or directory

The reason is that zdb pass objset name to spa_import, it uses that
name to create a spa. Later, when dmu_objset_own tries to lookup the spa
using real pool name, it can't find one.

We fix this by make sure we pass pool name rather than objset name to
spa_import.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: Chunwei Chen <david.chen at nutanix.com>
Closes #7099
Closes #6464

ZFS on Linux — cmd/zdb zdb.c

Fix zdb -E segfault

SPA_MAXBLOCKSIZE is too large for stack.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: Chunwei Chen <david.chen at nutanix.com>
Closes #7099
Delta File
+4 -1 cmd/zdb/zdb.c
+4 -1 1 file

ZFS on Linux — cmd/zdb zdb.c, man/man8 zdb.8

Fix zdb -R decompression

There are some issues in the zdb -R decompression implementation.

The first is that ZLE can easily decompress non-ZLE streams. So we add
ZDB_NO_ZLE env to make zdb skip ZLE.

The second is the random bytes appended to pabd, pbuf2 stuff. This serve
no purpose at all, those bytes shouldn't be read during decompression
anyway. Instead, we randomize lbuf2, so that we can make sure
decompression fill exactly to lsize by bcmp lbuf and lbuf2.

The last one is the condition to detect fail is wrong.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: Chunwei Chen <david.chen at nutanix.com>
Closes #7099
Closes #4984
Delta File
+18 -20 cmd/zdb/zdb.c
+3 -1 man/man8/zdb.8
+21 -21 2 files

ZFS on Linux — cmd/zdb zdb.c

Fix racy assignment of zcb.zcb_haderrors

zcb_haderrors will be modified in zdb_blkptr_done, which is
asynchronous. So we must move this assignment after zio_wait.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: Chunwei Chen <david.chen at nutanix.com>
Closes #7099
Delta File
+8 -2 cmd/zdb/zdb.c
+8 -2 1 file

ZFS on Linux — module/zfs zle.c

Fix zle_decompress out of bound access

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu at gmail.com>
Signed-off-by: Chunwei Chen <david.chen at nutanix.com>
Closes #7099
Delta File
+4 -0 module/zfs/zle.c
+4 -0 1 file

ZFS on Linux — include/sys dmu_objset.h, module/zfs dmu_objset.c zil.c

Change os->os_next_write_raw to work per txg

Currently, os_next_write_raw is a single boolean used for determining
whether or not the next call to dmu_objset_sync() should write out
the objset_phys_t as a raw buffer. Since the boolean is not associated
with a txg, the work simply happens during the next txg, which is not
necessarily the correct one. In the current implementation this issue
was misdiagnosed, resulting in a small hack in dmu_objset_sync() which
seemed to resolve the problem.

This patch changes os_next_write_raw to be an array of booleans, one
for each txg in TXG_OFF and removes the hack.

Reviewed-by: Jorgen Lundman <lundman at lundman.net>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #6864

ZFS on Linux — module/icp Makefile.in, module/icp/asm-x86_64/aes aes_intel.S

Change movaps to movups in AES-NI code

Currently, the ICP contains accelerated assembly code to be
used specifically on CPUs with AES-NI enabled. This code
makes heavy use of the movaps instruction which assumes that
it will be provided aes keys that are 16 byte aligned. This
assumption seems to hold on Illumos, but on Linux some kernel
options such as 'slub_debug=P' will violate it. This patch
changes all instances of this instruction to movups which is
the same except that it can handle unaligned memory.

This patch also adds a few flags which were accidentally never
given to the assembly compiler, resulting in objtool warnings.

Reviewed by: Gvozden Neskovic <neskovic at gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Nathaniel R. Lewis <linux.robotdude at gmail.com>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #7065
Closes #7108

ZFS on Linux — tests/zfs-tests/tests/functional/acl acl_common.kshlib

OpenZFS 8965 - zfs_acl_ls_001_pos fails due to no longer supported grep regex

The test used \> to detect the end of a string, but this no longer works,
so use $ which works as well since the string ends the line anyway.

Authored by: John Wren Kennedy <john.kennedy at delphix.com>
Reviewed by: Akash Ayare <aayare at delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov at delphix.com>
Reviewed by: Yuri Pankov <yuripv at icloud.com>
Reviewed by: Igor Kozhukhov <igor at dilos.org>
Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Approved by: Dan McDonald <danmcd at joyent.com>
Ported-by: Giuseppe Di Natale <dinatale2 at llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/8965
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/cb1204e444
Closes #7145

ZFS on Linux — module/zfs dsl_scan.c

Fix txg_sync_thread hang in scan_exec_io()

When scn->scn_maxinflight_bytes has not been initialized it's
possible to hang on the condition variable in scan_exec_io().
This issue was uncovered by ztest and is only possible when
deduplication is enabled through the following call path.

  txg_sync_thread()
    spa_sync()
      ddt_sync_table()
        ddt_sync_entry()
          dsl_scan_ddt_entry()
            dsl_scan_scrub_cb()
              dsl_scan_enqueuei()
                scan_exec_io()
                  cv_wait()

Resolve the issue by always initializing scn_maxinflight_bytes
to a reasonable minimum value.  This value will be recalculated
in dsl_scan_sync() to pick up changes to zfs_scan_vdev_limit
and the addition/removal of vdevs.

Reviewed-by: Tom Caputi <tcaputi at datto.com>
Reviewed by: George Melikov <mail at gmelikov.ru>

    [2 lines not shown]
Delta File
+13 -2 module/zfs/dsl_scan.c
+13 -2 1 file

ZFS on Linux — etc/systemd/system zfs-import-cache.service.in zfs-import-scan.service.in

remove pools without a bootfs from BOOTFS variable

Use the same method used in zfs-load-key.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Llewelyn Trahaearn <WoefulDerelict at GMail.com>
Signed-off-by: Matthew Thode <mthode at mthode.org>
Closes #7089

ZFS on Linux — lib/libzfs libzfs_sendrecv.c, tests/zfs-tests/tests/functional/cli_root/zfs_receive receive-o-x_props_override.ksh

Fix 'zfs receive -o' when used with '-e|-d'

When used in conjunction with one of '-e' or '-d' zfs receive options
none of the properties requested to be set (-o) are actually applied:
this is caused by a wrong assumption made about the toplevel dataset
in zfs_receive_one().

Fix this by correctly detecting the toplevel dataset.

Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7088

ZFS on Linux — module/zfs dmu_traverse.c

Fix zdb -c traverse stop on damaged objset root

If a corruption happens to be on a root block of an objset, zdb -c will
not correctly report the error, and it will not traverse the datasets
that come after. This is because traverse_visitbp, which does the
callback and reset error for TRAVERSE_HARD, is skipped when traversing
zil is failed in traverse_impl.

Here's example of what 'zdb -eLcc' command looks like on a pool with
damaged objset root:

== before patch:

Traversing all blocks to verify checksums ...

Error counts:

        errno  count
block traversal size 379392 != alloc 33987072 (unreachable 33607680)

        bp count:             172
        ganged count:           0
        bp logical:       1678336      avg:   9757
        bp physical:       130560      avg:    759     compression:  12.85
        bp allocated:      379392      avg:   2205     compression:   4.42

    [35 lines not shown]
Delta File
+14 -6 module/zfs/dmu_traverse.c
+14 -6 1 file

ZFS on Linux — include/sys trace_dmu.h

tx_waited -> tx_dirty_delayed in trace_dmu.h

This change was missed in 0735ecb33485e91a78357a274e47c2782858d8b9.

Reviewed-by: Fabian Grünbichler <f.gruenbichler at proxmox.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: András Korn <korn-github.com at elan.rulez.org>
Closes #7096
Delta File
+6 -5 include/sys/trace_dmu.h
+6 -5 1 file

ZFS on Linux — man/man5 zpool-features.5

Fix zpool-features(5) large_block inconsistency

Large_blocks feature activation was not consistent with man page,
which erroneously stated that the feature was active when the
recordsize was increased past the stock 128KB.  It actually
becomes active when data is written to the dataset.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: bunder2015 <omfgbunder at gmail.com>
Closes #6275
Closes #7093
Delta File
+2 -2 man/man5/zpool-features.5
+2 -2 1 file

ZFS on Linux — cmd/zdb zdb.c, cmd/zfs zfs_main.c

Fix style issues in man pages and commands help

* Remove 'zfs snap' from zfs help message (OpenZFS sync)
* Update zfs(8) to suggest 'snap' can be used as an alias for 'snapshot'
* Enforce 80 columns limit in help messages
* Remove zfs_disable_dup_eviction from zfs-module-parameters(5)
* Expose zfs_scan_max_ext_gap as a kernel module parameter.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tom Caputi <tcaputi at datto.com>
Signed-off-by: loli10K <ezomori.nozomu at gmail.com>
Closes #7087

ZFS on Linux — tests/zfs-tests/include commands.cfg, tests/zfs-tests/tests/functional/rsend send-cD.ksh

OpenZFS 9004 - Some ZFS tests used files removed with 32 bit kernel

Authored by: John Wren Kennedy <john.kennedy at delphix.com>
Reviewed by: Igor Kozhukhov <igor at dilos.org>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed by: George Melikov <mail at gmelikov.ru>
Approved by: Dan McDonald <danmcd at joyent.com>
Ported-by: Giuseppe Di Natale <dinatale2 at llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/9004
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/fafe9b241f
Closes #7149

ZFS on Linux — . configure.ac, cmd/dbufstat dbufstat.py

Add dbuf hash and dbuf cache kstats

Introduce kstats about the dbuf hash and dbuf cache
to make it easier to inspect state. This should help
with debugging and understanding of these portions
of the codebase.

Correct format of dbuf kstat file.

Introduce a dbc column to dbufs kstat to indicate if
a dbuf is in the dbuf cache.

Introduce field filtering in the dbufstat python script.

Introduce a no header option to the dbufstat python script.

Introduce a test case to test basic mru->mfu list movement
in the ARC.

Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Closes #6906

ZFS on Linux — lib/libspl/include libgen.h, lib/libzfs libzfs_import.c

zpool import -d to specify device path

When we know which devices have the pool we are looking for, sometime
it's better if we can directly pass those device paths to zpool import
instead of letting it to search through all unrelated stuff, which might
take a lot of time if you have hundreds of disks.

This patch allows option -d <dev_path> to zpool import. You can have
multiple pairs of -d <dev_path>, and zpool import will only search
through those devices. For example:

    zpool import -d /dev/sda -d /dev/sdb

Reviewed-by: Tony Hutter <hutter2 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Chunwei Chen <david.chen at nutanix.com>
Closes #7077

ZFS on Linux — contrib/initramfs README.initramfs.markdown

Update README.initramfs.markdown

Fix several typos and grammar.

Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Reviewed by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Arno van Wyk <avw1987 at users.noreply.github.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #7080

ZFS on Linux — module/zfs zfs_fm.c

Revert "Remove wrong ASSERT in annotate_ecksum"

This reverts commit 093911f1945b5c164a45bb077103283dafdcae0c.

Reviewed-by: Chunwei Chen <tuxoko at gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Closes #7079
Delta File
+2 -2 module/zfs/zfs_fm.c
+2 -2 1 file