Use a variable for mtree that includes the DEFAULT_MTREE_KEYWORDS, in
order to avoid missing the important "time" keyword or passing an
unimplemented "nlink" keyword.
Unskip the "failing" tests, and comment out the specific failing checks,
these will be addressed (or at least discussed) in a different revision.
For MS-DOS tests, use an even value timestamp, as precision is 2s.
For ZFS tests, use import_image instead of mount_image consistently.
Reviewed by: bnovkov, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D54429
As stated in the manual page:
-F is almost certainly not the option you are looking for. To
create an image from a list of files in an mtree format manifest,
specify it as the last argument on the command line, not as the
argument to -F.
This change does exactly that. Also bug #192839 has already been fixed.
Reviewed by: ngie
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D54428
Cleanup and remove default atf_check flags for clarity. The following
two lines are equivalent:
atf_check $cmd
atf_check -s exit:0 -e empty -o empty $cmd
Update the links to the reference documents.
Remove the D_flag_cleanup function, as common_cleanup() for these
particular set of tests does two things:
1. Unmount the md(4) device.
2. Destroy the md(4) device.
Essentially, one should only call common_cleanup() if the test body
invokes mount_image(). This is not the case for D_flag_body().
No functional changes intended.
Reviewed by: ngie
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D54427
The test makefs_zfs_tests:file_extend is timing out on ci.freebsd.org.
Double the default timeout to allow it to finish.
Reported by: Jenkins
Reviewed by: asomers
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D54425
The test makefs_zfs_tests:compression is timing out on ci.freebsd.org.
Double the default timeout to allow it to finish.
While here, check if the file exists before cleaning up, otherwise, cat
may fail.
Reported by: Jenkins
Reviewed by: asomers
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D54424
Use the script PID as a pool GUID. This way, tests running in parallel
will have pool GUIDs that won't collide, and the tests no longer need to
be serialized.
MFC after: 1 month
makefs currently does not implement compression for ZFS datasets, as
doing so seems somewhat fraught with compatibility issues. As a result,
the root dataset has compression disabled, and all others inherit from
that.
However, it may be useful to enable compression for new files once the
generated pool is actually imported. Thus, implement a per-data
compression option. By default, compression is inherited from the
parent dataset and disabled on the root dataset.
Add a regression test.
PR: 288241
MFC after: 1 month
Add tests for the -T flag to each makefs backend. This includes tests
for both mtree and directory scan options.
PR: 285630
Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation
Reviewed by: markj, emaste, kevans, jlduran
Differential Revision: https://reviews.freebsd.org/D49492
Previously, anything other than a regular file, directory or symlink
would cause makefs to exit with an assertion failure. Make it a bit
more resilient to user error: print a warning and skip the file. Add a
regression test wherein we create an image from a devfs mount.
PR: 283583
MFC after: 2 weeks
Previously we could create cd9660 images with duplicate short (level 2)
names.
cd9660_level2_convert_filename used a 30-character limit (for files and
directories), not including the '.' separator. cd9660_rename_filename
used a 31-character limit, including the '.'. Directory names 31
characters or longer (without '.') were shortened to 30 characters, and
if a collision occurred cd9660_rename_filename uniquified them starting
with the 31st character. Unfortunately the directory record's name_len
was already set, so the unique part of the name was stripped off.
Directories are up to 31 d-characters (i.e., A-Z 0-9 and _); there is no
provision for a '.' in a directory name. Increase the name length limit
to 31 for directories, and exclude '.'s.
This name mapping and deduplication code is still fragile and convoluted
and would beenfit from a more holistic effort.
PR: 283238, 283112
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D48251
By default, OpenZFS will perform metadata verification of the most
recent TXGs, but this can be very slow since all data in a pool
generated by makefs was logically written in a single transaction.
Avoid triggering this verification by default, but add an option to
restore the previous behaviour and enable it in regression test cases.
Reported by: cperciva
Tested by: cperciva (previous version)
MFC after: 2 weeks
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
This allows one to take a staged directory tree and create a file
consisting of a ZFS pool with one or more datasets that contain the
contents of the directory tree. This is useful for creating virtual
machine images without using the kernel to create a pool; "zpool create"
requires root privileges and currently is not permitted in jails.
makefs -t zfs also provides reproducible images by using a fixed seed
for pseudo-random number generation, used for generating GUIDs and hash
salts. makefs -t zfs requires relatively little by way of machine
resources.
The "zpool_reguid" rc.conf setting can be used to ask a FreeBSD guest to
generate a unique pool GUID upon first boot.
A small number of pool and dataset properties are supported. The pool
is backed by a single disk vdev. Data is always checksummed using
Fletcher-4, no redundant copies are made, and no compression is used.
The manual page documents supported pool and filesystem properties.
The implementation uses a few pieces of ZFS support from with the boot
loader, especially definitions for various on-disk structures, but is
otherwise standalone and in particular doesn't depend on OpenZFS.
This feature should be treated as experimental for now, i.e., important
data shouldn't be trusted to a makefs-created pool, and the command-line
interface is subject to change.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35248
When a new FS image is created we need to calculate how much space each
file is going to consume.
Fix two bugs in that logic:
1) Count the space needed for indirect blocks for large files.
1) Normally the trailing data of a file is written to a block of frag
size, 4 kB by default.
However for files that use indirect blocks a full block is allocated,
32kB by default. Take that into account.
Adjust size calculations to match what is done in ffs_mkfs routine:
* Depending on the UFS version the superblock is stored at a different
offset. Take that into account.
* Add the cylinder group block size.
* All of the above has to be aligned to the block size.
Finally, Remove "ncg" variable. It's always 1 and it was used to
multiply stuff.
PR: 229929
Reviewed by: mckusick
MFC after: 2 weeks
Sponsored by: Semihalf
Submitted by: Kornel Dulęba <mindal@semihalf.com>
Differential Revision: https://reviews.freebsd.org/D35131
Differential Revision: https://reviews.freebsd.org/D35132
When processing mtree(5) MANIFEST files, makefs(8) previously threw an
error if it encountered an entry whose "time" attribute contained a
non-zero subsecond component (e.g. time=1551620152.987220000).
Update the handling logic to properly assign the subsecond component if
built with nanosecond support, or silently discard it otherwise.
Also, re-enable the time attribute for the kyua tests.
PR: 194703
Submitted by: Mitchell Horne <mhorne063@gmail.com>
Differential Revision: https://reviews.freebsd.org/D19627
Fix two failing makefs test cases by adding "-M 1m", which was already used
for every other FFS test case. Add a new test case for the underlying
issue: with no -M, -m, or -s options, makefs can underestimate image size.
PR: 229929
Reported by: Jenkins
MFC after: 2 weeks
This extends the set in r316028 to allow all tests to pass or be skipped
on a system without cd9660 support.
A better approach using tar is possible, but this works today.
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D10516
dumpfs prints a harmless warning message (via ufs_disk_fillout(3) and
getfsfile(3)), when /etc/fstab does not exist. We can ignore it.
PR: 220165
Reported by: gjb
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
after r298107
Summary of changes:
- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)
Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.
MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
In some cases the test system might not have mount_cd9660(8). Don't
implicitly rely on it while testing cd9660 support; explicitly rely
on it
MFC after: 1 week
Reported by: mjohnston
Sponsored by: EMC / Isilon Storage Division
Verify the filesystem type using dumpfs. Add preliminary support
for NetBSD (needs to be validated)
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
- Parameterize the mtree keywords as $DEFAULT_MTREE_KEYWORDS
- Test with the extra mtree keywords, `mode,gid,uid`.
- Add a note about mtrees with time support not working with makefs right now
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
-- -o isolevel=1 currently fails because of path comparison issues,
so mark it as an expected failure.
-- -o isolevel=3 is not implemented, so expect it to fail as an out
of bounds value [*].
PR: 203645
MFC after: 1 week
X-MFC with: r290264
Sponsored by: EMC / Isilon Storage Division
