Displaying 1 50 of 11,528 commits (0.008s)

OpenZFS on OS X — module/zfs vdev_initialize.c space_map.c

Remove warnings with ASSERTV()

OpenZFS on OS X — module/zfs zfs_vfsops.c

Remove printf that can cause panic
Delta File
+0 -10 module/zfs/zfs_vfsops.c
+0 -10 1 file

OpenZFS on OS X — module/spl spl-osx.c

Wait for current_proc() to be setup

On very early boots, current_proc() returns NULL, and we call
taskq_thread_create() which calls spl_cv_wait()->msleep().
_sleep() in kernel will use current_proc() to update stats and
we panic with page fault.
Delta File
+5 -0 module/spl/spl-osx.c
+5 -0 1 file

OpenZFS on OS X — module/spl spl-taskq.c

mojave debug

print if p or p->p_stats is NULL or not-NULL.

Skip the first cv_wait that we know we die in.
Delta File
+14 -0 module/spl/spl-taskq.c
+14 -0 1 file

OpenZFS on OS X — module/spl spl-osx.c

Wait for current_proc() to be setup

On very early boots, current_proc() returns NULL, and we call
taskq_thread_create() which calls spl_cv_wait()->msleep().
_sleep() in kernel will use current_proc() to update stats and
we panic with page fault.
Delta File
+5 -0 module/spl/spl-osx.c
+5 -0 1 file

OpenZFS on OS X — module/spl/KernelExports zfs.exports

test

fs.exports

OpenZFS on OS X — module/zfs arc.c

Refactor arc_hdr_realloc_crypt()

The arc_hdr_realloc_crypt() function is responsible for converting
a "full" arc header to an extended "crypt" header and visa versa.
This code was originally written with a bcopy() so that any new
members added to arc headers would automatically be included
without requiring a code change. However, in practice this (along
with small differences in kmem_cache implementations between
various platforms) has caused a number of hard-to-find problems in
ports to other operating systems. This patch solves this problem
by making all member copies explicit and adding ASSERTs for fields
that cannot be set during the transfer. It also manually resets the
old header after the reallocation is finished so it can be properly
reallocated and reused.

Signed-off-by: Tom Caputi <tcaputi at datto.com>
Delta File
+61 -8 module/zfs/arc.c
+61 -8 1 file

OpenZFS on OS X — module/zfs arc.c

Revert "Temporary: fix arc_hdr_realloc_crypt() to zero released hdr"

This reverts commit d282603ac41fec0e1d47a21b0542f44c350390cb.

Replaced with proper fix:

Refactor arc_hdr_realloc_crypt()
Delta File
+0 -6 module/zfs/arc.c
+0 -6 1 file

OpenZFS on OS X — module/zfs dmu_send.c

Correct missing os_raw_receive

Causing ASSERT/VERIFY when receiving raw
Delta File
+1 -0 module/zfs/dmu_send.c
+1 -0 1 file

OpenZFS on OS X — module/zfs arc.c

Temporary: fix arc_hdr_realloc_crypt() to zero released hdr

This is a temporary commit until the proper fix will come from upstream
Delta File
+6 -0 module/zfs/arc.c
+6 -0 1 file

OpenZFS on OS X — lib/libzfs libzfs_crypto.c, module/zfs dsl_crypt.c

Separate the error code for already unloaded key

Connected to the future commit: Adopt pyzfs from ClusterHQ
from ZOL (85ce3f4fd1).

OpenZFS on OS X — include/sys zfs_znode.h, module/zfs zfs_znode.c

hack: test znode_move defrag performance

Shadow struct vnode to allow us to swap v_data fields over, and
test zfs_znode_move() feature of spl-kmem.

kstat.unix.kmem_cache.zfs_znode_cache.move_callbacks: 2804
kstat.unix.kmem_cache.zfs_znode_cache.move_yes: 2306
kstat.unix.kmem_cache.zfs_znode_cache.move_no: 0
kstat.unix.kmem_cache.zfs_znode_cache.move_later: 497
kstat.unix.kmem_cache.zfs_znode_cache.move_dont_need: 0
kstat.unix.kmem_cache.zfs_znode_cache.move_dont_know: 1
kstat.unix.kmem_cache.zfs_znode_cache.move_hunt_found: 0
kstat.unix.kmem_cache.zfs_znode_cache.move_slabs_freed: 694
kstat.unix.kmem_cache.zfs_znode_cache.move_reclaimable: 5
Delta File
+79 -11 module/zfs/zfs_znode.c
+3 -1 include/sys/zfs_znode.h
+82 -12 2 files

OpenZFS on OS X — include/sys vnode.h, module/spl spl-vnode.c

hack: test znode_move defrag performance

OpenZFS on OS X — module/zfs dbuf.c dmu_send.c

Fix coverity defects: CID 176037

CID 176037: Uninitialized scalar variable

This patch fixes an uninitialized variable defect caught by
coverity and introduced in 69830602

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Delta File
+4 -3 module/zfs/dbuf.c
+1 -1 module/zfs/dmu_send.c
+5 -4 2 files

OpenZFS on OS X — include/sys dmu.h, module/zfs dbuf.c dsl_dataset.c

Add ASSERT to debug encryption key mapping issues

This patch simply adds an ASSERT that confirms that the last
decrypting reference on a dataset waits until the dataset is
no longer dirty. This should help to debug issues where the
ZIO layer cannot find encryption keys after a dataset has been
disowned.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>

OpenZFS on OS X — module/zfs dbuf.c arc.c

Raw receive fix and encrypted objset security fix

This patch fixes two problems with the encryption code. First, the
current code does not correctly prohibit the DMU from updating
dn_maxblkid during object truncation within a raw receive. This
usually only causes issues when the truncating DRR_FREE record is
aggregated with DRR_FREE records later in the receive, so it is
relatively hard to hit.

Second, this patch fixes a security issue where reading blocks
within an encrypted object did not guarantee that the dnode block
itself had ever been verified against its MAC. Usually the
verification happened anyway when the bonus buffer was read, but
some use cases (notably zvols) might never perform the check.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Delta File
+103 -33 module/zfs/dbuf.c
+7 -3 module/zfs/arc.c
+6 -1 module/zfs/dnode_sync.c
+116 -37 3 files

OpenZFS on OS X — include/sys dbuf.h dmu.h, module/zfs dmu.c dmu_send.c

assertion in arc_release() during encrypted receive

In the existing code, when doing a raw (encrypted) zfs receive,
we call arc_convert_to_raw() from open context. This creates a
race condition between arc_release()/arc_change_state() and
writing out the block from syncing context (arc_write_ready/done()).

This change makes it so that when we are doing a raw (encrypted)
zfs receive, we save the crypt parameters (salt, iv, mac) of dnode
blocks in the dbuf_dirty_record_t, and call arc_convert_to_raw()
from syncing context when writing out the block of dnodes.

Additionally, we can eliminate dr_raw and associated setters, and
instead know that dnode blocks are always raw when doing a zfs
receive (see the new field os_raw_receive).

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tom Caputi <tcaputi at datto.com>
Signed-off-by: Matthew Ahrens <mahrens at delphix.com>

OpenZFS on OS X — include/sys dmu.h, module/zfs dmu_send.c dmu.c

Raw DRR_OBJECT records must write raw data

b1d21733 made it possible for empty metadnode blocks to be
compressed to a hole, fixing a bug that would cause invalid
metadnode MACs when a send stream attempted to free objects
and allowing the blocks to be reclaimed when they were no
longer needed. However, this patch also introduced a race
condition; if a txg sync occurred after a DRR_OBJECT_RANGE
record was received but before any objects were added, the
metadnode block would be compressed to a hole and lose all
of its encryption parameters. This would cause subsequent
DRR_OBJECT records to fail when they attempted to write
their data into an unencrypted block. This patch defers the
DRR_OBJECT_RANGE handling to receive_object() so that the
encryption parameters are set with each object that is
 written into that block.

Reviewed-by: Kash Pande <kash at tripleback.net>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Delta File
+46 -39 module/zfs/dmu_send.c
+22 -12 module/zfs/dmu.c
+3 -2 include/sys/dmu.h
+71 -53 3 files

OpenZFS on OS X — module/zfs dbuf.c

many threads directly evicting dbufs makes a computer slow

On a low-latency pool (RAM disk, SSD, L2 backed spinny
disk, or even a big hot ARC) with many many snapshots
running multiple concurrent
"zfs list -r -o name,creation -t snap pool"
thrashes the dbuf cache, and decompressing the
metadata blocks causes the kernel to eat all the available
CPU, kiling userland responsiveness and depressing
I/O throughput.

Keep a simple atomic count of the number of threads
directly evicting from the dbuf, and introduce an
IODelay(1) -- which does a microsecond of REP;NOP aka PAUSE,
yielding to the other hyperthread -- if the number of such
threads is less than half of max_ncpus.   For any directly
evicting thread above that threshold, do an IOSleep(1),
which deschedules the thread entirely for approximately a
millisecond, letting other threads (including the dbuf
eviction thread) make progress.
Delta File
+42 -0 module/zfs/dbuf.c
+42 -0 1 file

OpenZFS on OS X — module/zfs metaslab.c

Missing mutex_init(mc_fastwrite_lock)
Delta File
+1 -0 module/zfs/metaslab.c
+1 -0 1 file

OpenZFS on OS X — module/zfs spa_misc.c

Missing mutex_init(spa_feat_stats_lock)
Delta File
+1 -0 module/zfs/spa_misc.c
+1 -0 1 file

OpenZFS on OS X — include/sys zfs_ioctl.h

Reserve DMU_BACKUP_FEATURE for ZSTD

Reserve bit 25 for the ZSTD compression feature from FreeBSD.

Signed-off-by: Allan Jude <allanjude at freebsd.org>
Delta File
+1 -0 include/sys/zfs_ioctl.h
+1 -0 1 file

OpenZFS on OS X — module/zfs arc.c

Update the correct abd in l2arc_read_done()

This patch fixes an issue where l2arc_read_done() would always
write data to b_pabd, even if raw encrypted data was requested.
This only occured in cases where the L2ARC device had a different
ashift than the main pool.

Signed-off-by: Tom Caputi <tcaputi at datto.com>
Delta File
+12 -9 module/zfs/arc.c
+12 -9 1 file

OpenZFS on OS X — include/sys dmu.h, module/zfs dmu_send.c dmu.c

Raw receive functions must not decrypt data

This patch fixes a small bug found where receive_spill() sometimes
attempted to decrypt spill blocks when doing a raw receive. In
addition, this patch fixes another small issue in arc_buf_fill()'s
error handling where a decryption failure (which could be caused by
the first bug) would attempt to set the arc header's IO_ERROR flag
without holding the header's lock.

Signed-off-by: Tom Caputi <tcaputi at datto.com>

OpenZFS on OS X — config always-arch.m4

Config File fix for generic case

When the platform can not be identified, assume once again a generic
platform, not x86_64. This probably won't happen since OS X only runs
on intel platforms, but is cleaner either way.
Delta File
+1 -1 config/always-arch.m4
+1 -1 1 file

OpenZFS on OS X — cmd/zinject zinject.c, include/sys zio_crypt.h dsl_crypt.h

Add support for decryption faults in zinject

This patch adds the ability for zinject to trigger decryption
and authentication faults in the ZIO and ARC layers. This
functionality is exposed via the new "decrypt" error type, which
may be provided for "data" object types.

This patch also refactors some of the core encryption / decryption
functions so that they have consistent prototypes, handle errors
consistently, and do not have unused arguments.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Delta File
+47 -95 module/zfs/arc.c
+21 -12 module/zfs/dsl_crypt.c
+26 -6 cmd/zinject/zinject.c
+31 -1 module/zfs/zio_inject.c
+16 -5 module/zfs/zio.c
+8 -8 module/zfs/zio_crypt.c
+17 -11 7 files not shown
+166 -138 13 files

OpenZFS on OS X — lib/libspl zmount.c

Decrease noise of mount
Delta File
+3 -1 lib/libspl/zmount.c
+3 -1 1 file

OpenZFS on OS X — module/zfs arc.c

Correct minor differences in arc.c to upstream crypto changes
Delta File
+19 -11 module/zfs/arc.c
+19 -11 1 file

OpenZFS on OS X — module/zfs arc.c dmu_diff.c, tests/zfs-tests/runfiles osx.run.in

Fix issues found with zfs diff

Two deadlocks / ASSERT failures were introduced in a2c2ed1b which
would occur whenever arc_buf_fill() failed to decrypt a block of
data. This occurred because the call to arc_buf_destroy() which
was responsible for cleaning up the newly created buffer would
attempt to take out the hdr lock that it was already holding. This
was resolved by calling the underlying functions directly without
retaking the lock.

In addition, the dmu_diff() code did not properly ensure that keys
were loaded and mapped before begining dataset traversal. It turns
out that this code does not need to look at any encrypted values,
so the code was altered to perform raw IO only.

Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>

OpenZFS on OS X — cmd/zfs zfs_main.c, lib/libzfs libzfs_dataset.c

9512 zfs remap poolname at snapname coredumps

Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed by: John Wren Kennedy <john.kennedy at delphix.com>
Reviewed by: Sara Hartse <sara.hartse at delphix.com>
Approved by: Matt Ahrens <mahrens at delphix.com>

Only filesystems and volumes are valid 'zfs remap' parameters: when
passed a snapshot name zfs_remap_indirects() does not handle the
EINVAL returned from libzfs_core, which results in failing an assertion
and consequently crashing.

OpenZFS on OS X — man/man8 zpool.8

9521 Add checkpoint field in the default list of the zpool-list man page

Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro at delphix.com>
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed by: kpande <github at tripleback.net>
Approved by: Dan McDonald <danmcd at joyent.com>
Delta File
+2 -2 man/man8/zpool.8
+2 -2 1 file

OpenZFS on OS X — cmd/zstreamdump zstreamdump.c, module/zfs dmu_send.c

Fix issues with raw sends of spill blocks

This patch fixes 2 issues in how spill blocks are processed during
raw sends. The first problem is that compressed spill blocks were
using the logical length rather than the physical length to
determine how much data to dump into the send stream. The second
issue is a typo that caused the spill record's object number to be
used where the objset's ID number was required. Both issues have
been corrected, and the payload_size is now printed in zstreamdump
for future debugging.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>

OpenZFS on OS X — include/sys dnode.h dmu_impl.h, module/zfs dmu_objset.c dnode.c

Fix race in dnode_check_slots_free()

Currently, dnode_check_slots_free() works by checking dn->dn_type
in the dnode to determine if the dnode is reclaimable. However,
there is a small window of time between dnode_free_sync() in the
first call to dsl_dataset_sync() and when the useraccounting code
is run when the type is set DMU_OT_NONE, but the dnode is not yet
evictable, leading to crashes. This patch adds the ability for
dnodes to track which txg they were last dirtied in and adds a
check for this before performing the reclaim.

This patch also corrects several instances when dn_dirty_link was
treated as a list_node_t when it is technically a multilist_node_t.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>

OpenZFS on OS X — lib/libzfs libzfs_dataset.c

Correct swapped keylocation error messages

This patch corrects a small issue where two error messages
in the code that checks for invalid keylocations were
swapped.

Reviewed by: Matt Ahrens <mahrens at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Delta File
+4 -4 lib/libzfs/libzfs_dataset.c
+4 -4 1 file

OpenZFS on OS X — include/sys dmu_send.h, module/zfs dmu_send.c

9487 Free objects when receiving full stream as clone
Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Approved by: Dan McDonald <danmcd at joyent.com>
Delta File
+58 -0 module/zfs/dmu_send.c
+1 -0 include/sys/dmu_send.h
+59 -0 2 files

OpenZFS on OS X — module/zfs vdev.c

9479 fix wrong format specifier for vdev_id

Reviewed by: Matt Ahrens <matt at delphix.com>
Approved by: Robert Mustacchi <rm at joyent.com>
Delta File
+1 -1 module/zfs/vdev.c
+1 -1 1 file

OpenZFS on OS X — config always-arch.m4, lib/libicp Makefile.am

Use ASM sha / aes code on Mac OS X

- Re-add commented-out code in the build files.
- Remove ELF specific direcives .type and .size and change the name mangling
  to include a leading underscore.
  https://sourceware.org/binutils/docs-2.22/as/Type.html#Type
  https://sourceware.org/binutils/docs-2.22/as/Size.html#Size
- Following https://github.com/apple/darwin-xnu/blob/master/osfmk/i386/asm.h,
  change the .align values to be a bit shift instead of the actual alignment
  value. This seems to be different as in the assembler used on linux.
- Replace [] with () since "brackets expression not supported on this target".
- Enable compilation of the asm code in the kext. This mainly required removing
  specialized floating point support handling since this isn't apparently needed
  in the OS X kernel. (At least I couldn't find any documentation about it).
  If it is actually needed, this change can cause lots of trouble in userland
  code that gets the xmm registers clobbered.

OpenZFS on OS X — module/zfs ZFSDatasetProxy.cpp

Add missing methods
Delta File
+15 -0 module/zfs/ZFSDatasetProxy.cpp
+15 -0 1 file

OpenZFS on OS X — include/sys ZFSDataset.h ZFSDatasetScheme.h, module/zfs ZFSDatasetScheme.cpp ZFSDataset.cpp

Compile fixes for older OsX

The i116lite commit introduced some compatibility regression with
previous OsX versions.

OpenZFS on OS X — cmd/zfs zfs_main.c, tests/zfs-tests/tests/functional/cli_root/zfs_mount zfs_mount_encrypted.ksh

Make encrypted "zfs mount -a" failures consistent

Currently, "zfs mount -a" will print a warning and fail to mount
any encrypted datasets that do not have a key loaded. This patch
makes the behavior of this failure consistent with other failure
modes ("zfs mount -a" will silently continue, explict "zfs mount"
will print a message and return an error code.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>

OpenZFS on OS X — include/sys kmem.h taskq.h, module/spl spl-kmem.c spl-taskq.c

9018 Replace kmem_cache_reap_now() with kmem_cache_reap_soon()

    Reviewed by: Bryan Cantrill <bryan at joyent.com>
    Reviewed by: Dan McDonald <danmcd at joyent.com>
    Reviewed by: Matthew Ahrens <mahrens at delphix.com>
    Reviewed by: Yuri Pankov <yuripv at yuripv.net>

OpenZFS on OS X — module/zfs vdev_removal.c

9434 Speculative prefetch is blocked by device removal code

Reviewed by: Matthew Ahrens <mahrens at delphix.com>
Reviewed by: Prashanth Sreenivasa <pks at delphix.com>
Approved by: Matt Ahrens <mahrens at delphix.com>

Device removal code does not set spa_indirect_vdevs_loaded for pools
that never experienced device removal.  At least one visual consequence
of it is completely blocked speculative prefetcher.  This patch sets
the variable in such situations.
Delta File
+1 -0 module/zfs/vdev_removal.c
+1 -0 1 file

OpenZFS on OS X — . META

Tag zfs-1.7.3

META file updated.
Delta File
+1 -1 META
+1 -1 1 file

OpenZFS on OS X — . META

Tag spl-1.7.3

META file updated.
Delta File
+1 -1 META
+1 -1 1 file

OpenZFS on OS X — module/zfs arc.c

Remove ASSERT() in l2arc_apply_transforms()

The ASSERT was erroneously copied from the next section of code.
The buffer's size should be expanded from "psize" to "asize"
if necessary.

Reviewed-by: Tom Caputi <tcaputi at datto.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tim Chase <tim at chase2k.com>
Delta File
+4 -4 module/zfs/arc.c
+4 -4 1 file

OpenZFS on OS X — include/sys zio.h spa.h, module/zfs arc.c dbuf.c

Decryption error handling improvements

Currently, the decryption and block authentication code in
the ZIO / ARC layers is a bit inconsistent with regards to
the ereports that are produces and the error codes that are
passed to calling functions. This patch ensures that all of
these errors (which begin as ECKSUM) are converted to EIO
before they leave the ZIO or ARC layer and that ereports
are correctly generated on each decryption / authentication
failure.

In addition, this patch fixes a bug in zio_decrypt() where
ECKSUM never gets written to zio->io_error.

Reviewed by: Matt Ahrens <matt at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Delta File
+35 -4 module/zfs/arc.c
+14 -6 module/zfs/dbuf.c
+5 -4 include/sys/zio.h
+5 -4 module/zfs/zfs_fm.c
+5 -1 module/zfs/dmu_objset.c
+5 -1 module/zfs/dmu_send.c
+4 -3 3 files not shown
+73 -23 9 files

OpenZFS on OS X — module/zfs dbuf.c

Encrypted dnode blocks should be prefetched raw

Encrypted dnode blocks are always initially read as raw data and
converted to decrypted data when an encrypted bonus buffer is
needed. This allows the DMU to be used for things like fetching
the DMU master node without requiring keys to be loaded. However,
dbuf_issue_final_prefetch() does not currently read the data as
raw. The end result of this is that prefetched dnode blocks are
read twice from disk: once decrypted and then again as raw data.
This patch corrects the issue by adding the flag when appropriate.

Reviewed by: Matt Ahrens <matt at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Delta File
+7 -2 module/zfs/dbuf.c
+7 -2 1 file

OpenZFS on OS X — module/zfs zfs_vnops.c

Correct minor warning
Delta File
+2 -0 module/zfs/zfs_vnops.c
+2 -0 1 file

OpenZFS on OS X — tests/zfs-tests/include default.cfg.in, tests/zfs-tests/runfiles osx.run.in

Correct new zfstest files to run

These files require further fixes, in particular mdb use in functions like
set_condense_delay, should be call to sysctl instead. In addition those
new values may require to be added to kstat.

OpenZFS on OS X — tests/zfs-tests/include default.cfg.in, tests/zfs-tests/runfiles osx.run.in

Correct new zfstest files to run

These files require further fixes, in particular mdb use in functions like
set_condense_delay, should be call to sysctl instead. In addition those
new values may require to be added to kstat.