The default shell for root has been changed to sh(1) followup changing
in release images sh(1) the shell for the "freebsd" user.
MFC After: 1 week
Reviewed by: manu, emaste (re)
Approved by: manu, emaste (re)
Differential Revision: https://reviews.freebsd.org/D54602
This is required for MSI support on GCE ARM64 instances which is
prerequisite to gve(4) not panicking at boot, and nvme(4) also has
a real sad time without interrupts. Tested on a variety of c4a VMs.
This is meant to be a temporary hack; long term fix would be to
check for the hypervisor and quirk gve(4) device with
PCI_QUIRK_ENABLE_MSI_VM.
PR: kern/292081
MFC after: 1 week
Removes hw.vtnet.mq_disable=1.
This workaround was originally introduced nearly a decade ago to
address stability issues on KVM that have long since been resolved
in both the FreeBSD driver and the GCE hypervisor. Removing this
allows network interrupts to scale across multiple vCPUs.
Tested on n2-highcpu-16 VM with 15.0-RELEASE and confirmed multiple
queue pairs active and interrupts handling across cores.
PR: kern/292081
MFC after: 1 day
The sed command was missing the ${DESTDIR} prefix, meaning it was
attempting to modify the build host's /etc/rc.d/growfs instead of
the target image's script. Tested in an arm64 builder that builds
as non-root.
PR: kern/292081
MFC after: 1 day
The packages for 15.0-RELEASE built without the bug fix needed to make
files created via @sample get properly listed in METALOG. Fix the
cloudware which contain @sample-using packages by adding the necessary
files to METALOG manually.
This should be reverted after the next full package build, and live on
only in releng/15.0.
Reviewed by: markj
MFC after: immediately (15.0-RC2)
Differential Revision: https://reviews.freebsd.org/D53797
We ship these in order to comply with GCE Marketplace rules about
providing source code and licenses for all the software we ship as
part of images.
Reviewed by: markj
MFC after: immediately (15.0-RC2)
Differential Revision: https://reviews.freebsd.org/D53796
These were forgotten during the METALOGization process earlier.
Reviewed by: markj
MFC after: immediately (for 15.0-RC2)
Differential Revision: https://reviews.freebsd.org/D53795
Paths all need to start with "./" because that's what newfs wants.
Fixes: e0c41af925 ("vmimage.subr: Enable FreeBSD-base repo if pkgbase")
MFC after: immediately
When installing "extra" packages (aka those built from the ports tree),
we record everything being installed in METALOG.pkg; the contents of
that file is appended to METALOG before we generate the filesystem.
There are two cases when files recorded in METALOG.pkg will no longer
exist by the time we create the final disk image:
1. If a pkg bug results in false dependencies being installed which
are later removed by "pkg autoremove", and
2. If the pkg we build and install from /usr/ports is older than the
pkg on pkg.freebsd.org, and pkg gets upgraded automatically as part of
installing extra packages.
The ultimate issue in both cases is that there's no mechanism for
removing entries from METALOG when we run 'pkg delete'.
Address this build breakage by checking, line by line, if filesystem
objects mentioned in METALOG.pkg exist before appending them to METALOG.
Fixes: 6a13aeac3c ("vmimage.subr: pkg autoremove after pkg install")
MFC after: immediately (needed for 15.0-RC1)
Running 'pkg autoremove' without -y results in VM image builds failing
when (bogusly installed) packages are removed.
Fixes: 6a13aeac3c ("vmimage.subr: pkg autoremove after pkg install")
MFC after: immediately (needed for 15.0-RC1)
When creating a VM image using pkgbase, create a configuration file in
/usr/local/etc/pkg/repos/FreeBSD.conf which enables the FreeBSD-base
repository. (This repository is defined in /etc/pkg/FreeBSD.conf as
being disabled by default.)
Reported by: Mark Millard
MFC after: immediately (needed for 15.0-RC1)
We were doing this in vm_extra_install_packages but VM images without
any extra packages installed would not get this installed. This
results in a pkgbase system which thinks it doesn't have any packages
installed (even though all the files are right there).
Add a "metalog_add_data ./var/db/pkg/local.sqlite" call to the pkgbase
install code path, and make the call from vm_extra_install_packages
conditional on !PKGBASE.
Reported by: Michael Dexter
MFC after: immediately (needed for 15.0-RC1)
We want to fetch distfiles, regardless of whether they contain known
vulnerabilities or we're building images for a different version of
FreeBSD.
Reviewed by: ivy
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D53569
In order to comply with the require that GCE images must include their
source code, we fetch distfiles for all of the packages installed into
GCE images. This fails for obvious reasons for packages with an origin
of base/*; filter those out to generate the list to fetch.
Reviewed by: ivy
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D53568
A bug in pkg, which somehow only surfaced as a consequence of pkgbase,
results in pkg install sometimes pulling in false dependencies. This
problem might be limited to cases when the lib32 pkgbase packages are
not installed. In the case of EC2 "small" images, installing the
ebsnvme-id package results in binutils, gcc12-devel, gmp, indexinfo,
liblz4, mpc, mpfr, and zstd packages being installed.
These false dependencies are however not recorded as dependencies --
at some level pkg does understand that they're not needed -- so running
pkg autoremove immediately after pkg install cleans them up.
Note: This does not remove lines from METALOG corresponding to these
packages, and makefs emits an error when it attempts to create the
filesystem but cannot find the files listed in METALOG -- but makefs
does seem to complete normally despite the error messages.
This change should be reverted once the pkg issue has been located and
fixed.
Reviewed by: ivy
MFC after: 3 days
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D53543
This defaults to plain "pkg", but being able to override it is useful
when testing pkg itself.
Reviewed by: cperciva
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D53307
We only need to check for unMETALOGed directories and sort the METALOG
file if we're using it, i.e. if we're doing a NO_ROOT build. This
non-NO_ROOT builds by no longer bogusly writing to /METALOG*.
We only need to add databases (spwd.db etc) to METALOG if we're doing
a pkgbase-enabled NO_ROOT build; but we should always do this before
creating the filesystem, not only if we installed extra packages (in
vm_extra_install_packages, where that code was erroneously placed).
This fixes non-cloud VM images, which in 15.0-BETA2 shipped without
password databases.
Reviewed by: ivy
MFC after: 3 days
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D53194
Theoretically METALOG should include everything which needs to go
into disk images; unfortunately there are still a few bugs which
are resulting in directories not being listed -- and if METALOG
has files in unrecorded directories, the directories end up being
created with 000 permissions.
Oddly enough, systems where / has 000 permissions are not very
usable.
As a temporary hack, compare the staging tree against METALOG and
add entries for any unrecorded directories. This will hopefully
be reverted before 15.0-RELEASE.
Reviewed by: bapt, emaste, ivy
Sponsored by: https://www.patreon.com/cperciva
MFC after: 5 minutes
Differential Revision: https://reviews.freebsd.org/D53153
When creating VM images from pkgbase, the METALOG may not be in order;
in particular, files may be listed before the directories which contain
them. This causes makefs to create directories with 000 permissions.
Interestingly, such VM images boot just fine, since root ignores those
permissions; the first sign of trouble was sshd refusing logins with an
error message which said absolutely nothing about /etc/ having
incorrect permissions or being unable to read files inside it.
Immediately prior to running makefs, sort the METALOG file. While
we're here, uniquify as well; this does not guarantee that we do not
have duplicate paths, but if there are duplicate paths with different
settings something else has gone wrong and we don't really have any
good way of solving the problem anyway.
Reviewed by: ivy
Hint from: imp
MFC after: 3 days
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D53046
The file /usr/local/etc/ssl/cert.pem is not present on "small" and
"builder" AMIs, so we don't need to add it to METALOG.
Fixes: 2b0ffc0ee4 ("EC2: metalog_add missing files from packages")
MFC after: 3 days
Sponsored by: https://www.patreon.com/cperciva
The files
/usr/local/etc/dhclient.conf
/usr/local/etc/ssl/cert.pem
are created in EC2 images by installed packages, but are not being
registered in METALOG. The reason for this is not immediately clear,
but may be related to the handling of "sample" files.
Reviewed by: dch
MFC after: 5 minutes
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D53035
The files /etc/login.conf.db, /etc/passwd, /etc/pwd.db, /etc/spwd.db,
and /var/db/services.db are generated at install time. When VM images
are built using installworld, there is Makefile code which emits the
necessary METALOG lines; but the pkgbase triggers do not do this. This
resulted in VM images which lacked these essential database files.
VM images are currently the only place where this matters, since this
issue is specific to non-root image builds uusing pkgbase. (In some
point in the future we will start building release ISO iamges from
pkgbase, at which point it will be a problem there as well.)
The long-term solution for this is probably to teach the relevant
database-creating tools to emit METALOG lines; but for now, just log
the necessary files from vmimage.subr.
Reviewed by: dch
MFC after: 5 minutes
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D53034
This also installs the pkg key from the current source tree instead of
using the one currently installed on the host.
Reviewed by: dch
MFC after: 1 day
Differential Revision: https://reviews.freebsd.org/D52615
Filtering the list of packages broke with the introduction of package
sets, since we excluded FreeBSD-src.* but still included
FreeBSD-set-src, which transitively included the FreeBSD-src and
FreeBSD-src-sys packages. This could be fixed by excluding package
sets, but that would be fragile, potentially breaking upgrades if
new packages are introduced.
Instead, start with an explicit set of package sets:
base, base-dbg
lib32, lib32-dbg
kernels, kernels-dbg
tests
and filter the package sets; the EC2 "small" and "builder" AMIs
filter out everything except "base" and "kernels".
Note that using FreeBSD-set-kernels may pose a problem in the future
if we start shipping packages for multiple differently-configured
kernels. That will be addressed if and when that problem arises.
Reviewed by: ivy
MFC after: 3 days
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D52922
I excluded lldb from the "small" and "builder" flavours of EC2 AMIs
because on FreeBSD 14.x the lldb binary was 100 MB, as a result of
statically linking in lots of LLVM code; but we now have libprivatellvm
as a shared library and lldb is much smaller so there's no significant
benefit to excluding it.
Reviewed by: ivy
MFC after: 3 days
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D52921
Otherwise pkg complains about unexpected ownership of the root
directory.
MFC after: 1 day
Fixes: 86221f965c ("vmimage.subr: Fix NO_ROOT build in ..._rmcache")
- Don't clean pkg files, they won't be recorded in the manifest anyway.
- Use pw's new metalog mode to create the vagrant user. Note that we do
not need to manually create the home directory, pw will do it, so just
remove that.
- Write metalog entries for the vagrant user's ssh key dir and
authorized keys file
Note, this depends on an updated pw being installed on the host.
Reviewed by: emaste
MFC after: 1 day
Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D52631
We need to tell pkg to record files in METALOG; otherwise when we
create the VM image it's almost empty.
MFC after: 6 hours
Sponsored by: https://www.patreon.com/cperciva
When doing a NO_ROOT build we can't run pkg inside a chroot, for
multiple reasons including not having /dev mounted.
MFC after: 6 hours
Sponsored by: https://www.patreon.com/cperciva
This was previously deprecated and is slated for removal in 15.0.
Users who still need ftpd(8) can install the ftp/freebsd-ftpd port.
Retain the ftp(d) PAM services since other FTP daemons use them.
Update /etc/inetd.conf to point to /usr/local.
Add ftpd to ObsoleteFiles, but do not list configuration files since
users may want to preserve these to use with the freebsd-ftpd port.
There is still some language in the manual referring to ftpd(8)
which is relevant to the port, which has been retained but updated
to reference the port.
MFC after: 3 days
Relnotes: yes
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D52739
VM and cloud images are now built as packaged base systems by default,
matching the default for installation media.
Setting -DNOPKGBASE allows building as non-pkgbase systems.
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D51483
This also indirectly addresses the lack of FreeBSD-lldb package on
the riscv64 platform by relying on that package being part of the
FreeBSD-set-devel set iff it exists.
A subsequent commit will adjust the "notoolchain" OCI container image
to also use a package set, which will indirectly affect the contents
of the "toolchain" container (which is an explicit superset of the
"notoolchain" container).
MFC after: 1 day
Co-authored-by: dch
Add metalog entries for various files. This is a bit incomplete as
pw(8) doesn't yet have support for emitting metalog entries, which we
need since it's used here to create an interactive user.
Reviewed by: emaste
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52456
- Avoid mounting devfs just to list packages and fetch distfiles.
- Add metalog entries.
Reviewed by: cperciva, emaste
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52455
- Add metalog entries.
- Remove a stray entry in the package list.
Reviewed by: cperciva, emaste
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52454
- Avoid chrooting to uninstall pkg, as it will not be installed in the
first place when in non-root mode.
- Add metalog entries.
Reviewed by: cperciva
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52452
- Don't chroot to uninstall pkg since we don't install pkg in the first
place.
- Use pw -R instead of chrooting.
- Add metalog entries.
Reviewed by: cperciva, emaste
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52451
Provide variables, PKG_REPOS_DIR and PKG_REPO_NAME, to allow the user to
configure the source package repository. Configure pkg to emit added
files to a metalog which gets concatenated with the main metalog
immediately before we build the root filesystem.
Reviewed by: cperciva, emaste
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52453
Some services, such as growfs, only run upon the first boot of an image.
The first boot is indicated by the presence of the file /firstboot,
which is unlinked after boot.
Individual cloudware types shouldn't be responsible for creating it. Do
so in a centralized place. Aside from simplifying things, this ensures
that we create a metalog entry for the firstboot file.
Reviewed by: cperciva, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52450
Verify that the target file or directory exists, and add support for an
optional mode. This is required to enable NO_ROOT builds of some
cloudware image types. No functional change intended.
Reviewed by: cperciva, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52449
This "noatime" tweak, reduced the OPNsense constant 50kb - 80kb disk writes that wear down the SSD / NVME.
Signed-off-by: Unicorn9x
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/1834
