OpenZFS on OS X/zfs 486e5b1. META

zfs-1.9.3
DeltaFile
+1-1META
+1-11 files

OpenZFS on OS X/spl 6472461. META

spl-1.9.3
DeltaFile
+1-1META
+1-11 files

OpenZFS on OS X/zfs efa1fe4lib/libzfs libzfs_mount.c

Temporarily disable the volume custom icon code

OpenZFS on OS X/spl 008d227include/sys stropts.h

Compiple fixes for 10.10

OpenZFS on OS X/zfs 9dea79czfs_bundle Makefile.am

zfs_bundle should pick the correct mount_zfs

OpenZFS on OS X/zfs 3741cbcmodule/zfs spa_config.c

Do not spa_write_cachefile() on boot.

If we call spa_write_cachefile() very early in boot process it will
panic. Use similar tests to illumos and wait for rootdir to be set.

OpenZFS on OS X/zfs 9b67175cmd/zfs Makefile.am

Produce static mount_zfs

based off cmd/zfs/ used to mount root on boot.
DeltaFile
+10-4cmd/zfs/Makefile.am
+10-41 files

OpenZFS on OS X/zfs 81e5cf3. configure.ac, cmd Makefile.am

Remove mount_zfs and use cmd/zfs

OsX uses cmd/zfs named "mount_zfs" to do manual mounts. We also need
to make a static compile of cmd/zfs that is not codesigned for
boot support.

OpenZFS on OS X/zfs eb7f648module/zfs ldi_iokit.cpp ldi_vnode.c

Revert "Release mutex before iokit calls"

This reverts commit 5ffe3d6cefe6da0dde7bbc3987be085106aa5845.

One side-effect of this is that if you try to import the same pool
twice at the same time, we can panic in spa_add(avl_add(exists)).

Unfortunately, even though we try to let go of spa_namespace_lock
here, it is still holding spa_config().

This means this commit would try to grab spa_namespace_lock again while
holding spa_config. A different thread may be in B->A case:

spa_open_common(hold spa_namespace_lock -> spa_config_generate( grab spa_config

This unfurtunately means the original ZVOL deadlock problem is probably
back, but we'll have to find a different solution to that issue.

OpenZFS on OS X/zfs 4889d27module/zfs zfs_vnops_osx.c

Revert "Disable decmpfs decompression"

This reverts commit 37ef7e553a3f62e885fa9d8b2c62adf16061168b.

The cause of the issue has been resolved with spl:
https://github.com/openzfsonosx/spl/commit/e1134e3380ed63ebfd38888562751356e2f365c1

OpenZFS on OS X/spl e1134e3module/spl spl-vnode.c, module/spl/KernelExports zfs.exports

10.15 changed struct decmpfs

Causes memory corruption in unrelated memory areas.

Currently the situation is decmpfs cnodes:

.. 10.11  | 10.12  -  10.14 | 10.15 ..
no alloc  | can use alloc   | struct changed
use struct| same struct     | use alloc

But we make a hard switch at 10.11 - 10.12 to keep it simple. At
older versions, including 10.11, we have a shadow copy of
struct decmpfs, and use internal kmem_alloc.
At 10.12 and newer versions, we call decmpfs_cnode_alloc() and
do not need to know struct size.

Wrong 'struct decmpfs' on 10.15 (Catalina) caused the value
0x0000200721000000 to be written after the allocated memory, due to final
struct member being "lck_rw_t" and initialising the lock would
cause the issue.

OpenZFS on OS X/zfs 37ef7e5module/zfs zfs_vnops_osx.c

Disable decmpfs decompression

Under Catalina, calling _decmpfs_file_is_compressed()
appears to write 0x0000200721000000 into a random memory
location. Disabled until a solution can be found.

OpenZFS on OS X/zfs 28d37abmodule/zfs zfs_ioctl.c

Temporarily remove zfs_ioc_log_history

It appears zfs_ioc_log_history() is the cause of random memory
corruption on (at least) Catalina.

Function stubbed out until cause is known and fixed.

OpenZFS on OS X/zfs d76946emodule/nvpair nvpair.c, module/zfs fm.c zfs_ioctl.c

hardcore events verify

OpenZFS on OS X/zfs 5c37cffmodule/zfs zfs_vnops_osx.c dsl_prop.c

Add _PC_MIN_HOLE_SIZE to pathconf.

Frequently used in Catalina onwards.

OpenZFS on OS X/zfs 41d0e7amodule/zfs zfs_ctldir.c

Check returncode of VN_HOLD()

Don't release vp in error case, caller handles

OpenZFS on OS X/zfs f59d4betests/zfs-tests/runfiles osx.run.in, tests/zfs-tests/tests/functional/cli_root/zpool_import zpool_import.kshlib zpool_import_errata3.ksh

zfs-test fixes

OpenZFS on OS X/zfs 94cfe2ecmd/fsck_zfs Makefile.am, cmd/mount_zfs Makefile.am

Compile fixes for Catalina / Xcode 11

Rename zfs.util -> zfs_util
Add fsck_zfs to zfs_bundle
Add mount_zfs to zfs_bundle
Add newfs_zfs to zfs_bundle

OpenZFS on OS X/spl 45a67cainclude/osx mutex.h, include/sys sysmacros.h types.h

Compile fixes for Catalina / Xcode 11

OpenZFS on OS X/zfs 86a28e3module/zfs zfs_vnops_osx.c

Catalina compatibility fixes

OpenZFS on OS X/zfs f7d25d5cmd/zfs zfs_project.c zfs_main.c, module/zfs sa.c zfs_vfsops.c

Add userobj, groupobj and project quotas

Project quota currently does nothing on OsX.

OpenZFS on OS X/zfs 28cef2finclude zfeature_common.h, include/sys dsl_dataset.h

Add types to featureflags in zfs

The boolean featureflags in use thus far in ZFS are extremely useful,
but because they take advantage of the zap layer, more interesting data
than just a true/false value can be stored in a featureflag. In redacted
send/receive, this is used to store the list of redaction snapshots for
a redacted dataset.

This change adds the ability for ZFS to store types other than a boolean
in a featureflag. The only other implemented type is a uint64_t array.
It also modifies the interfaces around dataset features to accomodate
the new capabilities, and adds a few new functions to increase
encapsulation.

This functionality will be used by the Redacted Send/Receive feature.

Reviewed-by: Matthew Ahrens <mahrens at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Paul Dagnelie <pcd at delphix.com>
Closes #7981

OpenZFS on OS X/zfs 15ef0eecmd/zfs zfs_main.c, include/sys zfs_vfsops.h

two

OpenZFS on OS X/zfs 31e497acmd/zfs zfs_main.c, include/sys zfs_vfsops.h dnode.h

start

OpenZFS on OS X/zfs ee975c0. META

zfs-1.9.2
DeltaFile
+1-1META
+1-11 files

OpenZFS on OS X/spl 5298e88. META

spl-1.9.2
DeltaFile
+1-1META
+1-11 files

OpenZFS on OS X/zfs 2c5fb9dmodule/zcommon zfs_fletcher.c, module/zfs vdev_raidz_math.c

Remove AVX512* vectorised functions

Tested to cause panic on modern CPUs.

OpenZFS on OS X/zfs 1a708falib/libzfs libzfs_dataset.c

zfs rename abort

Merging with ZOL missed some cleanup code avoid core dumps.

OpenZFS on OS X/zfs 3a45fe4. META

zfs-1.9.1
DeltaFile
+1-1META
+1-11 files

OpenZFS on OS X/spl d7a26c0. META

spl-1.9.1
DeltaFile
+1-1META
+1-11 files

OpenZFS on OS X/zfs df73a34module/zfs arc.c

Silence debug print
DeltaFile
+1-1module/zfs/arc.c
+1-11 files

OpenZFS on OS X/spl dab6cc4module/spl spl-condvar.c

spl_cv_timedwait() timeout of zero blocks.

sec==0, nsec==0 is not a poll request in XNU kernel. Ensure it
never is all zero, preferring sleep over fast-return.

OpenZFS on OS X/zfs fbe9d96module/zfs vdev_queue.c

Use ZOL's vdev_queue_max_async_writes with guards

vdev_queue_max_writes was throwing assertions when writing
to raidz/raidz2/raidz3 vdevs because it was miscalculating
writes.

Use the ZOL version of the function edited to add an
ASSERT and a MAX to catch and avoid a division-by-zero KP,
which was the motivation for diverging from the code in
the upstream(s).

All the assertions had stacks like:

                  zfs`vdev_queue_io+0x1b3
                  zfs`zio_vdev_io_start+0x540
                  zfs`zio_nowait+0x2b2
                  zfs`vdev_raidz_io_start+0x658
                  zfs`zio_vdev_io_start+0x755
                  zfs`zio_nowait+0x2b2
                  zfs`vdev_mirror_io_start+0x419
                  zfs`zio_vdev_io_start+0x755
                  zfs`__zio_execute+0x1b2
                  spl`taskq_thread+0x237
                  kernel`call_continuation+0x17


    [7 lines not shown]

OpenZFS on OS X/spl dc70a3einclude/sys simd_x86.h

Compile fixes for pre-10.13

OpenZFS on OS X/spl d5e5938. META

spl-1.9.1-rc1
DeltaFile
+2-2META
+2-21 files

OpenZFS on OS X/spl 37d47c6module/spl spl-condvar.c

Clean up spl_cv_timedwait()

Handle partial second delays as well, even though hires timer should be
used for that.
DeltaFile
+14-11module/spl/spl-condvar.c
+14-111 files

OpenZFS on OS X/zfs 1d59f7e. META

zfs-1.9.1-rc1
DeltaFile
+2-2META
+2-21 files

OpenZFS on OS X/spl 7fb322d. META

spl-1.9.1-rc1
DeltaFile
+2-2META
+2-21 files

OpenZFS on OS X/spl c1ee357module/spl spl-condvar.c

Clean up spl_cv_timedwait()

Handle partial second delays as well, even though hires timer should be
used for that.
DeltaFile
+14-11module/spl/spl-condvar.c
+14-111 files

OpenZFS on OS X/zfs 219bb65module/zfs bqueue.c

Temporary work around to mutex starvation

While bqueue_enqueue() is sitting in cv_wait_sig() waiting for the mutex
to be returned; bqueue_dequeue() can spin so fast it never releases the
mutex long enough. We deliberately drop the mutex in each loop.

This is suggesting we have a thread starvation problem, possibly with
thread priority levels being incorrect. Further research required.

OpenZFS on OS X/zfs 6df9bb5module/zfs bqueue.c

Temporary work around to mutex starvation

While bqueue_enqueue() is sitting in cv_wait_sig() waiting for the mutex
to be returned; bqueue_dequeue() can spin so fast it never releases the
mutex long enough. We deliberately drop the mutex in each loop.

This is suggesting we have a thread starvation problem, possibly with
thread priority levels being incorrect. Further research required.

OpenZFS on OS X/zfs 18d41ffmodule/zfs dmu_send.c

hack slash
DeltaFile
+21-7module/zfs/dmu_send.c
+21-71 files

OpenZFS on OS X/zfs 1867705module/zfs abd.c

ABD changes for vectorized RAIDZ

This is a distilled version of ZoL's
https://github.com/zfsonlinux/zfs/commit/a206522c4fd31f03f14ba174d6159b72acfae0a9

* add abd_get_offset_size() interface
* abd_iter_map(): fix calculation of iter_mapsize
* add abd_raidz_gen_iterate() and abd_raidz_rec_iterate()

We had some minor differences in the size calculations causing issues in
certain corner cases.
DeltaFile
+10-6module/zfs/abd.c
+10-61 files

OpenZFS on OS X/zfs 3ca683emodule/zfs dmu_send.c

Pick HAVE_LARGE_STACK path on OsX

OpenZFS on OS X/zfs 1a5cd97module/zfs bqueue.c

Call cv_signal() with mutex held

In bqueue_dequeue(), call cv_signal() with bq_lock held.
Re-enable rsend_009_pos to test the fix.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Boris Protopopov <boris.protopopov at actifio.com>
Closes #5887

OpenZFS on OS X/zfs 3859c2einclude/sys zfs_ioctl.h, module/zfs dmu_send.c dmu_recv.c

Revert "Fix send/recv lost spill block"

This reverts commit 8d7a713e91f7f4594c63e9c851fe926faee3d6d5.

OpenZFS on OS X/zfs d6e954dmodule/zfs zio.c

Enable raw writes to perform dedup with verification

Reviewed by: Dan Kimmel <dan.kimmel at delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed by: David Quigley <david.quigley at intel.com>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Issue #5078
DeltaFile
+46-24module/zfs/zio.c
+46-241 files

OpenZFS on OS X/zfs 2be5466module/zfs zio.c

Enable raw writes to perform dedup with verification

Reviewed by: Dan Kimmel <dan.kimmel at delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed by: David Quigley <david.quigley at intel.com>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Issue #5078
DeltaFile
+47-7module/zfs/zio.c
+47-71 files

OpenZFS on OS X/zfs 99992e3tests/config commands-osx.m4

Missing commands-osx.m4

OpenZFS on OS X/zfs bfe2903module/zfs zvolIO.cpp zil.c

Correct ASSERT for --enable-debug case