kgess-mirror/freebsd-src
1
0
Fork 0
mirror of https://git.freebsd.org/src.git synced 2026-01-11 19:57:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Vendor import of xz 5.8.2 (trimmed)

Browse source
This commit is contained in:
Xin LI 2025-12-21 16:14:28 -08:00
parent 12eff5f0d8
commit 762f11d98d
32 changed files with 2079 additions and 339 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1586
ChangeLog
View file

File diff suppressed because it is too large Load diff

434
THANKS
View file

@ -3,225 +3,243 @@ Thanks
======
Some people have helped more, some less, but nevertheless everyone's help
has been important. :-) In alphabetical order:
- Mark Adler
- Kian-Meng Ang
- H. Peter Anvin
- Jeff Bastian
- Nelson H. F. Beebe
- Karl Beldan
- Karl Berry
- Anders F. Björklund
- Emmanuel Blot
- Melanie Blower
- Alexander Bluhm
- Martin Blumenstingl
- Ben Boeckel
- Jakub Bogusz
has been important. :-)
- Adam Borowski
- Maarten Bosmans
- Roel Bouckaert
- Lukas Braune
- Benjamin Buch
- Trent W. Buck
- Kevin R. Bulgrien
- James Buren
- David Burklund
- Frank Busse
- Daniel Mealha Cabrita
- Milo Casagrande
- Cristiano Ceglia
- Marek Černocký
- Tomer Chachamu
- Vitaly Chikunov
- Antoine Cœur
- Elijah Almeida Coimbra
- Felix Collin
- Ryan Colyer
- Marcus Comstedt
- Vincent Cruz
- Gabi Davar
- Ron Desmond
- İhsan Doğan
- Chris Donawa
- Andrew Dudman
- Markus Duft
- İsmail Dönmez
- Dexter Castor Döpping
- Paul Eggert
- Robert Elz
- Gilles Espinasse
- Denis Excoffier
- Vincent Fazio
- Michael Felt
- Sean Fenian
- Michael Fox
- Andres Freund
- Mike Frysinger
- Collin Funk
- Daniel Richard G.
- Tomasz Gajc
- Bjarni Ingi Gislason
- John Paul Adrian Glaubitz
- Bill Glessner
- Matthew Good
- Michał Górny
- Jason Gorski
- Alexander M. Greenham
- Juan Manuel Guerrero
- Gabriela Gutierrez
- Diederik de Haas
- Jan Terje Hansen
- Tobias Lahrmann Hansen
- Joachim Henke
- Lizandro Heredia
- Christian Hesse
- Vincenzo Innocente
- Peter Ivanov
- Nicholas Jackson
- Sam James
- Hajin Jang
- Hans Jansen
- Jouk Jansen
- Jun I Jin
- Christoph Junghans
- Kiyoshi Kanazawa
- Joona Kannisto
- Per Øyvind Karlsen
- Firas Khalil Khana
- Iouri Kharon
- Kim Jinyeong
- Thomas Klausner
- Richard Koch
- Anton Kochkov
- Harri K. Koskinen
- Ville Koskinen
- Sergey Kosukhin
- Marcin Kowalczyk
- Jan Kratochvil
- Christian Kujau
- Stephan Kulow
- Ilya Kurdyukov
- Peter Lawler
- James M Leddy
- Kelvin Lee
- Vincent Lefevre
- Hin-Tak Leung
- Andraž 'ruskie' Levstik
- Cary Lewis
- Wim Lewis
- Xin Li
- Yifeng Li
- Eric Lindblad
- Lorenzo De Liso
- H.J. Lu
- Bela Lubkin
- Chenxi Mao
- Gregory Margo
- Julien Marrec
- Pierre-Yves Martin
- Ed Maste
- Martin Matuška
- Scott McAllister
- Chris McCrohan
- Derwin McGeary
- Ivan A. Melnikov
- Jim Meyering
- Arkadiusz Miskiewicz
- Nathan Moinvaziri
- Étienne Mollier
- Conley Moorhous
- Dirk Müller
- Rainer Müller
- Andrew Murray
- Rafał Mużyło
- Adam Walling
- Adrien Nader
- Evan Nemerson
- Alexander Neumann
- Hongbo Ni
- Jonathan Nieder
- Asgeir Storesund Nilsen
- Andre Noll
- Ruarí Ødegaard
- Peter O'Gorman
- Dimitri Papadopoulos Orfanos
- Daniel Packard
- Filip Palian
- Peter Pallinger
- Kai Pastor
- Keith Patton
- Rui Paulo
- Igor Pavlov
- Diego Elio Pettenò
- Elbert Pol
- Guiorgy Potskhishvili
- Mikko Pouru
- Frank Prochnow
- Rich Prohaska
- Trần Ngọc Quân
- Pavel Raiskup
- Matthieu Rakotojaona
- Ole André Vadla Ravnås
- Eric S. Raymond
- Robert Readman
- Bernhard Reutner-Fischer
- Markus Rickert
- Cristian Rodríguez
- Jeroen Roovers
- Christian von Roques
- Boud Roukema
- Torsten Rupp
- Stephen Sachs
- Jukka Salmi
- Agostino Sarubbo
- Vijay Sarvepalli
- Alexander Bluhm
- Alexander M. Greenham
- Alexander Neumann
- Alexandre Sauvé
- Benno Schulenberg
- Alexey Tourbin
- Anders F. Björklund
- Andraž 'ruskie' Levstik
- Andre Noll
- Andreas K. Hüttel
- Andreas Müller
- Andreas Schwab
- Eli Schwartz
- Peter Seiderer
- Bhargava Shastry
- Dan Shechter
- Stuart Shelton
- Sebastian Andrzej Siewior
- Andreas Zieringer
- Andrej Skenderija
- Ville Skyttä
- Andres Freund
- Andrew Dudman
- Andrew Murray
- Antoine Cœur
- Anton Kochkov
- Antonio Diaz Diaz
- Arkadiusz Miskiewicz
- Asgeir Storesund Nilsen
- Aziz Chaudhry
- Bela Lubkin
- Ben Boeckel
- Benjamin Buch
- Benno Schulenberg
- Bernhard Reutner-Fischer
- Bert Wesarg
- Bhargava Shastry
- Bill Glessner
- Bjarni Ingi Gislason
- Boud Roukema
- Brad Smith
- Bruce Stark
- Pippijn van Steenhoven
- Tobias Stoeckmann
- Martin Storsjö
- Jonathan Stott
- Dan Stromberg
- Douglas Thor
- Vincent Torri
- Alexey Tourbin
- Paul Townsend
- Mohammed Adnène Trojette
- Orange Tsai
- Taiki Tsunekawa
- Mathieu Vachon
- Maksym Vatsyk
- Loganaden Velvindron
- Patrick J. Volkerding
- Martin Väth
- Adam Walling
- Jeffrey Walton
- Christian Weisgerber
- Dan Weiss
- Bert Wesarg
- Mark Wielaard
- Fredrik Wikstrom
- Jim Wilcoxson
- Ralf Wildenhues
- Cary Lewis
- Charles Wilson
- Lars Wirzenius
- Vincent Wixsom
- Pilorz Wojciech
- Chenxi Mao
- Chien Wong
- Xi Ruoyao
- Chris Donawa
- Chris McCrohan
- Christian Hesse
- Christian Kujau
- Christian von Roques
- Christian Weisgerber
- Christoph Junghans
- Collin Funk
- Conley Moorhous
- Cristian Rodríguez
- Cristiano Ceglia
- Dan Shechter
- Dan Stromberg
- Dan Weiss
- Daniel Leonard
- Daniel Mealha Cabrita
- Daniel Packard
- Daniel Richard G.
- David Burklund
- Denis Excoffier
- Derwin McGeary
- Dexter Castor Döpping
- Diederik de Haas
- Diego Elio Pettenò
- Dimitri Papadopoulos Orfanos
- Dirk Müller
- Douglas Thor
- Ed Maste
- Elbert Pol
- Eli Schwartz
- Elijah Almeida Coimbra
- Émilie Labbé
- Emmanuel Blot
- Eric Lindblad
- Eric S. Raymond
- Étienne Mollier
- Evan Nemerson
- Fangrui Song
- Felix Collin
- Filip Palian
- Firas Khalil Khana
- François Etcheverry
- Frank Busse
- Frank Prochnow
- Fredrik Wikstrom
- Gabi Davar
- Gabriela Gutierrez
- Gilles Espinasse
- Gregory Margo
- Guillaume Outters
- Guiorgy Potskhishvili
- H. Peter Anvin
- Hajin Jang
- Hans Jansen
- Harri K. Koskinen
- Hin-Tak Leung
- H.J. Lu
- Hongbo Ni
- Igor Pavlov
- İhsan Doğan
- Ilya Kurdyukov
- Iouri Kharon
- İsmail Dönmez
- Ivan A. Melnikov
- Jakub Bogusz
- James Buren
- James M Leddy
- Jan Kratochvil
- Jan Terje Hansen
- Jason Gorski
- Jeff Bastian
- Jeffrey Walton
- Jeroen Roovers
- Jim Meyering
- Jim Wilcoxson
- Joachim Henke
- John Paul Adrian Glaubitz
- Jonathan Nieder
- Jonathan Stott
- Joona Kannisto
- Jouk Jansen
- Juan Manuel Guerrero
- Jukka Salmi
- Julien Marrec
- Jun I Jin
- Kai Pastor
- Karl Beldan
- Karl Berry
- Keith Patton
- Kelvin Lee
- Kevin R. Bulgrien
- Kian-Meng Ang
- Kim Jinyeong
- Kirill A. Korinsky
- Kiyoshi Kanazawa
- Lars Wirzenius
- Li Chenggang
- Lizandro Heredia
- Loganaden Velvindron
- Lorenzo De Liso
- Lukas Braune
- Maarten Bosmans
- Maksym Vatsyk
- Marcin Kowalczyk
- Marcus Comstedt
- Marcus Tillmanns
- Marek Černocký
- Mark Adler
- Mark Wielaard
- Markus Duft
- Markus Rickert
- Martin Blumenstingl
- Martin Matuška
- Martin Storsjö
- Martin Väth
- Mathieu Vachon
- Matthew Good
- Matthieu Rakotojaona
- Melanie Blower
- Michael Felt
- Michael Fox
- Michał Górny
- Mike Frysinger
- Mikko Pouru
- Milo Casagrande
- Mohammed Adnène Trojette
- Nathan Moinvaziri
- Nelson H. F. Beebe
- Nicholas Jackson
- Ole André Vadla Ravnås
- Orange Tsai
- Orgad Shaneh
- Patrick J. Volkerding
- Paul Eggert
- Paul Townsend
- Pavel Raiskup
- Per Øyvind Karlsen
- Peter Ivanov
- Peter Lawler
- Peter O'Gorman
- Peter Pallinger
- Peter Seiderer
- Pierre-Yves Martin
- Pilorz Wojciech
- Pippijn van Steenhoven
- Rafał Mużyło
- Rainer Müller
- Ralf Wildenhues
- Rich Prohaska
- Richard Koch
- Richard W.M. Jones
- Robert Elz
- Robert Readman
- Roel Bouckaert
- Ron Desmond
- Ruarí Ødegaard
- Rui Paulo
- Ryan Colyer
- Ryan Young
- Andreas Zieringer
- Sam James
- Scott McAllister
- Sean Fenian
- Sebastian Andrzej Siewior
- Sergey Kosukhin
- Simon Josefsson
- Siteshwar Vashisht
- Steffen Nurpmeso
- Stephan Kulow
- Stephen Sachs
- Stuart Shelton
- Taiki Tsunekawa
- Thomas Klausner
- Tobias Lahrmann Hansen
- Tobias Stoeckmann
- Tomasz Gajc
- Tomer Chachamu
- Torsten Rupp
- Trần Ngọc Quân
- Trent W. Buck
- Victoria Alexia
- Vijay Sarvepalli
- Ville Koskinen
- Ville Skyttä
- Vincent Cruz
- Vincent Fazio
- Vincent Lefevre
- Vincent Torri
- Vincent Wixsom
- Vincenzo Innocente
- Vitaly Chikunov
- Wim Lewis
- Xi Ruoyao
- Xin Li
- Yifeng Li
- 榆柳松 (ZhengSen Wang)
Companies:

8
doxygen/Doxyfile
View file

@ -8,7 +8,7 @@
# - Instead of API docs, docs of XZ Utils internals may be built.
# - Change the output directory for out-of-tree builds.
#
# These options were tested with Doxygen 1.10.0.
# These options were tested with Doxygen 1.9.8 and 1.13.2.
PROJECT_NAME = "liblzma (XZ Utils)"
OUTPUT_DIRECTORY = ../doc
@ -19,6 +19,8 @@ RECURSIVE = YES
OPTIMIZE_OUTPUT_FOR_C = YES
EXTRACT_STATIC = YES
SORT_MEMBER_DOCS = NO
WARN_IF_UNDOCUMENTED = NO
WARN_AS_ERROR = FAIL_ON_WARNINGS
SOURCE_TOOLTIPS = NO
VERBATIM_HEADERS = NO
ALPHABETICAL_INDEX = NO
@ -37,3 +39,7 @@ PREDEFINED = LZMA_API(type)=type \
tuklib_attr_noreturn= \
lzma_attribute(attr)= \
lzma_attr_alloc_size(size)=
# Debian and Ubuntu patch Doxygen so that HAVE_DOT = YES is the default.
# Set HAVE_DOT explicitly to get consistent behavior across distributions.
HAVE_DOT = NO

45
src/common/my_landlock.h
View file

@ -4,6 +4,10 @@
//
/// \file my_landlock.h
/// \brief Linux Landlock sandbox helper functions
///
/// \note This uses static variables to cache the Landlock ABI version.
/// Only one file in an application should include this header.
/// Only one thread should call these functions.
//
// Author: Lasse Collin
//
@ -17,6 +21,7 @@
#include <linux/landlock.h>
#include <sys/syscall.h>
#include <sys/prctl.h>
#include <sys/utsname.h>
/// \brief Initialize Landlock ruleset attributes to forbid everything
@ -32,8 +37,38 @@ my_landlock_ruleset_attr_forbid_all(struct landlock_ruleset_attr *attr)
{
memzero(attr, sizeof(*attr));
const int abi_version = syscall(SYS_landlock_create_ruleset,
// Cache the Landlock ABI version:
// 0 = not checked yet
// -1 = Landlock not supported
// >0 = Landlock ABI version
static int abi_version = 0;
#ifdef LANDLOCK_SCOPE_SIGNAL
// Red Hat Enterprise Linux 9 kernel since 5.14.0-603.el9 (2025-07-30)
// claims ABI version 6 support, but as of 5.14.0-643.el9 (2025-11-22)
// it lacks LANDLOCK_SCOPE_SIGNAL. ABI version 6 was added in upstream
// Linux 6.12 while RHEL 9 has Linux 5.14 with lots of backports.
// We assume that any kernel version 5.14 with ABI version 6 is buggy.
static bool is_rhel9 = false;
#endif
if (abi_version == 0) {
abi_version = syscall(SYS_landlock_create_ruleset,
(void *)NULL, 0, LANDLOCK_CREATE_RULESET_VERSION);
#ifdef LANDLOCK_SCOPE_SIGNAL
if (abi_version == 6) {
static const char rel[] = "5.14.";
const size_t rel_len = sizeof(rel) - 1;
struct utsname un;
if (uname(&un) == 0 && strncmp(
un.release, rel, rel_len) == 0)
is_rhel9 = true;
}
#endif
}
if (abi_version <= 0)
return -1;
@ -109,6 +144,14 @@ my_landlock_ruleset_attr_forbid_all(struct landlock_ruleset_attr *attr)
#endif
FALLTHROUGH;
case 6:
#ifdef LANDLOCK_SCOPE_SIGNAL
if (is_rhel9)
attr->scoped &= ~LANDLOCK_SCOPE_SIGNAL;
#endif
FALLTHROUGH;
default:
// We only know about the features of the ABIs 1-6.
break;

3
src/common/mythread.h
View file

@ -78,7 +78,8 @@ do { \
} while (0)
#if !(defined(_WIN32) && !defined(__CYGWIN__)) && !defined(__wasm__)
#if !(defined(_WIN32) && !defined(__CYGWIN__)) \
&& (!defined(__wasm__) || defined(__EMSCRIPTEN__))
// Use sigprocmask() to set the signal mask in single-threaded programs.
#include <signal.h>

6
src/liblzma/api/lzma/base.h
View file

@ -450,7 +450,9 @@ typedef struct {
* \param opaque lzma_allocator.opaque (see below)
* \param ptr Pointer returned by lzma_allocator.alloc(),
* or when it is set to NULL, a pointer returned
* by the standard malloc().
* by the standard malloc(). In addition, NULL
* is a possible value. The function should do
* nothing when ptr == NULL.
*/
void (LZMA_API_CALL *free)(void *opaque, void *ptr);
@ -561,7 +563,7 @@ typedef struct {
* \brief New seek input position for LZMA_SEEK_NEEDED
*
* When lzma_code() returns LZMA_SEEK_NEEDED, the new input position
* needed by liblzma will be available seek_pos. The value is
* needed by liblzma will be available in seek_pos. The value is
* guaranteed to not exceed the file size that was specified when
* this lzma_stream was initialized.
*

38
src/liblzma/api/lzma/container.h
View file

@ -843,8 +843,7 @@ extern LZMA_API(lzma_ret) lzma_alone_decoder(
/**
* \brief Initialize .lz (lzip) decoder (a foreign file format)
*
* This decoder supports the .lz format version 0 and the unextended .lz
* format version 1:
* This decoder supports the .lz format versions 0 and 1:
*
* - Files in the format version 0 were produced by lzip 1.3 and older.
* Such files aren't common but may be found from file archives
@ -853,28 +852,27 @@ extern LZMA_API(lzma_ret) lzma_alone_decoder(
* support for the format version 0 was removed in lzip 1.18.
*
* - lzip 1.3 added decompression support for .lz format version 1 files.
* Compression support was added in lzip 1.4. In lzip 1.6 the .lz format
* version 1 was extended to support the Sync Flush marker. This extension
* is not supported by liblzma. lzma_code() will return LZMA_DATA_ERROR
* at the location of the Sync Flush marker. In practice files with
* the Sync Flush marker are very rare and thus liblzma can decompress
* almost all .lz files.
* Compression support was added in lzip 1.4.
*
* - lzlib extends version 1 format with the Sync Flush marker. This
* extension is only meant for lzlib use; it's not valid in normal .lz
* files. This extension is not supported by liblzma. lzma_code() will
* return LZMA_DATA_ERROR at the location of the Sync Flush marker.
*
* Just like with lzma_stream_decoder() for .xz files, LZMA_CONCATENATED
* should be used when decompressing normal standalone .lz files.
*
* The .lz format allows putting non-.lz data at the end of a file after at
* least one valid .lz member. That is, one can append custom data at the end
* of a .lz file and the decoder is required to ignore it. In liblzma this
* is relevant only when LZMA_CONCATENATED is used. In that case lzma_code()
* will return LZMA_STREAM_END and leave lzma_stream.next_in pointing to
* the first byte of the non-.lz data. An exception to this is if the first
* 1-3 bytes of the non-.lz data are identical to the .lz magic bytes
* (0x4C, 0x5A, 0x49, 0x50; "LZIP" in US-ASCII). In such a case the 1-3 bytes
* will have been ignored by lzma_code(). If one wishes to locate the non-.lz
* data reliably, one must ensure that the first byte isn't 0x4C. Actually
* one should ensure that none of the first four bytes of trailing data are
* equal to the magic bytes because lzip >= 1.20 requires it by default.
* If LZMA_CONCATENATED is used and there is non-.lz data after at least one
* valid .lz member, lzma_code() leaves lzma_stream.next_in pointing to the
* first byte of the non-.lz data and returns LZMA_STREAM_END. That is, one
* can append custom data at the end of a .lz file and the decoder will
* ignore it. An exception to this is if the first 1-3 bytes of the non-.lz
* data are identical to the .lz magic bytes (0x4C, 0x5A, 0x49, 0x50; "LZIP"
* in US-ASCII). In such a case the 1-3 bytes are consumed by lzma_code().
* If one wishes to locate the non-.lz data reliably, one must ensure that
* the first byte isn't 0x4C. It's best if none of the first four bytes of
* trailing data are equal to the magic bytes because if two or three bytes
* are, lzip >= 1.20 diagnoses it as a corrupt member header by default.
*
* \param strm Pointer to lzma_stream that is at least initialized
* with LZMA_STREAM_INIT.

2
src/liblzma/api/lzma/version.h
View file

@ -22,7 +22,7 @@
#define LZMA_VERSION_MINOR 8
/** \brief Patch version number of the liblzma release. */
#define LZMA_VERSION_PATCH 1
#define LZMA_VERSION_PATCH 2
/**
* \brief Version stability marker

5
src/liblzma/check/crc32_arm64.h
View file

@ -23,7 +23,8 @@
// If both versions are going to be built, we need runtime detection
// to check if the instructions are supported.
#if defined(CRC32_GENERIC) && defined(CRC32_ARCH_OPTIMIZED)
# if defined(HAVE_GETAUXVAL) || defined(HAVE_ELF_AUX_INFO)
# if (defined(HAVE_GETAUXVAL) && defined(HAVE_HWCAP_CRC32)) \
|| defined(HAVE_ELF_AUX_INFO)
# include <sys/auxv.h>
# elif defined(_WIN32)
# include <processthreadsapi.h>
@ -103,7 +104,7 @@ crc32_arch_optimized(const uint8_t *buf, size_t size, uint32_t crc)
static inline bool
is_arch_extension_supported(void)
{
#if defined(HAVE_GETAUXVAL)
#if defined(HAVE_GETAUXVAL) && defined(HAVE_HWCAP_CRC32)
return (getauxval(AT_HWCAP) & HWCAP_CRC32) != 0;
#elif defined(HAVE_ELF_AUX_INFO)

2
src/liblzma/check/crc32_fast.c
View file

@ -2,7 +2,7 @@
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32.c
/// \file crc32_fast.c
/// \brief CRC32 calculation
//
// Authors: Lasse Collin

10
src/liblzma/check/crc64_fast.c
View file

@ -2,7 +2,7 @@
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc64.c
/// \file crc64_fast.c
/// \brief CRC64 calculation
//
// Authors: Lasse Collin
@ -146,14 +146,6 @@ crc64_dispatch(const uint8_t *buf, size_t size, uint64_t crc)
extern LZMA_API(uint64_t)
lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
{
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER) && !defined(__clang__) \
&& defined(_M_IX86) && defined(CRC64_ARCH_OPTIMIZED)
// VS2015-2022 might corrupt the ebx register on 32-bit x86 when
// the CLMUL code is enabled. This hack forces MSVC to store and
// restore ebx. This is only needed here, not in lzma_crc32().
__asm mov ebx, ebx
#endif
#if defined(CRC64_GENERIC) && defined(CRC64_ARCH_OPTIMIZED)
return crc64_func(buf, size, crc);

19
src/liblzma/check/crc_common.h
View file

@ -89,7 +89,8 @@ extern const uint64_t lzma_crc64_table[4][256];
// ARM64
//
// Keep this in sync with changes to crc32_arm64.h
#if defined(_WIN32) || defined(HAVE_GETAUXVAL) \
#if defined(_WIN32) \
|| (defined(HAVE_GETAUXVAL) && defined(HAVE_HWCAP_CRC32)) \
|| defined(HAVE_ELF_AUX_INFO) \
|| (defined(__APPLE__) && defined(HAVE_SYSCTLBYNAME))
# define CRC_ARM64_RUNTIME_DETECTION 1
@ -134,10 +135,20 @@ extern const uint64_t lzma_crc64_table[4][256];
// built and runtime detection is used even if compiler flags
// were set to allow CLMUL unconditionally.
//
// - This doesn't work with MSVC as I don't know how to detect
// the features here.
// - The unconditional use doesn't work with MSVC as I don't know
// how to detect the features here.
//
# if (defined(__SSSE3__) && defined(__SSE4_1__) && defined(__PCLMUL__) \
// Don't enable CLMUL at all on old MSVC that targets 32-bit x86.
// There seems to be a compiler bug that produces broken code
// in optimized (Release) builds. It results in crashing tests.
// It is known that VS 2019 16.11 (MSVC 19.29.30158) is broken
// and that VS 2022 17.13 (MSVC 19.43.34808) works.
# if defined(_MSC_FULL_VER) && _MSC_FULL_VER < 194334808 \
&& !defined(__INTEL_COMPILER) && !defined(__clang__) \
&& defined(_M_IX86)
// Old MSVC targeting 32-bit x86: Don't enable CLMUL at all.
# elif (defined(__SSSE3__) && defined(__SSE4_1__) \
&& defined(__PCLMUL__) \
&& !defined(HAVE_CRC_X86_ASM)) \
|| (defined(__e2k__) && __iset__ >= 6)
# define CRC32_ARCH_OPTIMIZED 1

6
src/liblzma/common/alone_decoder.c
View file

@ -128,8 +128,10 @@ alone_decode(void *coder_ptr, const lzma_allocator *allocator,
lzma_set_ext_size(coder->options, coder->uncompressed_size);
// Calculate the memory usage so that it is ready
// for SEQ_CODER_INIT.
coder->memusage = lzma_lzma_decoder_memusage(&coder->options)
// for SEQ_CODER_INIT. We know that lc/lp/pb are valid
// so we can use the _nocheck variant.
coder->memusage
= lzma_lzma_decoder_memusage_nocheck(&coder->options)
+ LZMA_MEMUSAGE_BASE;
coder->pos = 0;

4
src/liblzma/common/filter_common.c
View file

@ -213,8 +213,8 @@ lzma_filters_copy(const lzma_filter *src, lzma_filter *real_dest,
error:
// Free the options which we have already allocated.
while (i-- > 0)
lzma_free(dest[i].options, allocator);
while (i > 0)
lzma_free(dest[--i].options, allocator);
return ret;
}

3
src/liblzma/common/lzip_decoder.c
View file

@ -212,7 +212,8 @@ lzip_decode(void *coder_ptr, const lzma_allocator *allocator,
coder->options.pb = LZIP_PB;
// Calculate the memory usage.
coder->memusage = lzma_lzma_decoder_memusage(&coder->options)
coder->memusage
= lzma_lzma_decoder_memusage_nocheck(&coder->options)
+ LZMA_MEMUSAGE_BASE;
// Initialization is a separate step because if we return

1
src/liblzma/common/outqueue.h
View file

@ -188,6 +188,7 @@ extern bool lzma_outq_is_readable(const lzma_outq *outq);
/// \brief Read finished data
///
/// \param outq Pointer to an output queue
/// \param allocator lzma_allocator for custom allocator functions
/// \param out Beginning of the output buffer
/// \param out_pos The next byte will be written to
/// out[*out_pos].

79
src/liblzma/common/stream_decoder_mt.c
View file

@ -33,26 +33,6 @@ typedef enum {
} worker_state;
typedef enum {
/// Partial updates (storing of worker thread progress
/// to lzma_outbuf) are disabled.
PARTIAL_DISABLED,
/// Main thread requests partial updates to be enabled but
/// no partial update has been done by the worker thread yet.
///
/// Changing from PARTIAL_DISABLED to PARTIAL_START requires
/// use of the worker-thread mutex. Other transitions don't
/// need a mutex.
PARTIAL_START,
/// Partial updates are enabled and the worker thread has done
/// at least one partial update.
PARTIAL_ENABLED,
} partial_update_mode;
struct worker_thread {
/// Worker state is protected with our mutex.
worker_state state;
@ -104,10 +84,18 @@ struct worker_thread {
/// happen if all worker threads were frequently locking the main
/// mutex to update their outbuf->pos.
///
/// Only when partial_update is something else than PARTIAL_DISABLED,
/// this worker thread will update outbuf->pos after each call to
/// the Block decoder.
partial_update_mode partial_update;
/// When partial_update_enabled is true, this worker thread will
/// update outbuf->pos and outbuf->decoder_in_pos after each call
/// to the Block decoder. This is initially false. Main thread may
/// set this to true.
bool partial_update_enabled;
/// Once the main thread has set partial_updated_enabled = true,
/// we will do the first partial update as soon as we can and
/// set partial_update_started = true. After the first update,
/// we only update if we have made progress. This avoids useless
/// locking of thr->coder->mutex.
bool partial_update_started;
/// Block decoder
lzma_next_coder block_decoder;
@ -335,7 +323,7 @@ worker_enable_partial_update(void *thr_ptr)
struct worker_thread *thr = thr_ptr;
mythread_sync(thr->mutex) {
thr->partial_update = PARTIAL_START;
thr->partial_update_enabled = true;
mythread_cond_signal(&thr->cond);
}
}
@ -346,7 +334,7 @@ worker_decoder(void *thr_ptr)
{
struct worker_thread *thr = thr_ptr;
size_t in_filled;
partial_update_mode partial_update;
bool partial_update_enabled;
lzma_ret ret;
next_loop_lock:
@ -378,14 +366,16 @@ next_loop_unlocked:
thr->progress_out = thr->out_pos;
// If we don't have any new input, wait for a signal from the main
// thread except if partial output has just been enabled. In that
// thread except if partial output has just been enabled
// (partial_update_enabled is true but _started is false). In that
// case we will do one normal run so that the partial output info
// gets passed to the main thread. The call to block_decoder.code()
// is useless but harmless as it can occur only once per Block.
in_filled = thr->in_filled;
partial_update = thr->partial_update;
partial_update_enabled = thr->partial_update_enabled;
if (in_filled == thr->in_pos && partial_update != PARTIAL_START) {
if (in_filled == thr->in_pos && !(partial_update_enabled
&& !thr->partial_update_started)) {
mythread_cond_wait(&thr->cond, &thr->mutex);
goto next_loop_unlocked;
}
@ -407,23 +397,21 @@ next_loop_unlocked:
thr->outbuf->allocated, LZMA_RUN);
if (ret == LZMA_OK) {
if (partial_update != PARTIAL_DISABLED) {
// The main thread uses thr->mutex to change from
// PARTIAL_DISABLED to PARTIAL_START. The main thread
// doesn't care about this variable after that so we
// can safely change it here to PARTIAL_ENABLED
// without a mutex.
thr->partial_update = PARTIAL_ENABLED;
if (partial_update_enabled) {
// Remember that we have done at least one partial
// update. After the first update we won't do updates
// unless we have made progress.
thr->partial_update_started = true;
// The main thread is reading decompressed data
// from thr->outbuf. Tell the main thread about
// our progress.
//
// NOTE: It's possible that we consumed input without
// producing any new output so it's possible that
// only in_pos has changed. In case of PARTIAL_START
// it is possible that neither in_pos nor out_pos has
// changed.
// producing any new output so it's possible that only
// in_pos has changed. If thr->partial_update_started
// was false, it is possible that neither in_pos nor
// out_pos has changed.
mythread_sync(thr->coder->mutex) {
thr->outbuf->pos = thr->out_pos;
thr->outbuf->decoder_in_pos = thr->in_pos;
@ -645,7 +633,8 @@ get_thread(struct lzma_stream_coder *coder, const lzma_allocator *allocator)
coder->thr->progress_in = 0;
coder->thr->progress_out = 0;
coder->thr->partial_update = PARTIAL_DISABLED;
coder->thr->partial_update_enabled = false;
coder->thr->partial_update_started = false;
return LZMA_OK;
}
@ -816,12 +805,12 @@ read_output_and_wait(struct lzma_stream_coder *coder,
// keeps calling lzma_code() without providing more
// input, it will eventually get LZMA_BUF_ERROR.
//
// NOTE: We can read partial_update and in_filled
// without thr->mutex as only the main thread
// NOTE: We can read partial_update_enabled and
// in_filled without thr->mutex as only the main thread
// modifies these variables. decoder_in_pos requires
// coder->mutex which we are already holding.
if (coder->thr != NULL && coder->thr->partial_update
!= PARTIAL_DISABLED) {
if (coder->thr != NULL &&
coder->thr->partial_update_enabled) {
// There is exactly one outbuf in the queue.
assert(coder->thr->outbuf == coder->outq.head);
assert(coder->thr->outbuf == coder->outq.tail);

2
src/liblzma/common/vli_decoder.c
View file

@ -38,7 +38,7 @@ lzma_vli_decode(lzma_vli *restrict vli, size_t *vli_pos,
// Validate the arguments.
if (*vli_pos >= LZMA_VLI_BYTES_MAX
|| (*vli >> (*vli_pos * 7)) != 0)
return LZMA_PROG_ERROR;;
return LZMA_PROG_ERROR;
if (*in_pos >= in_size)
return LZMA_BUF_ERROR;

4
src/liblzma/lzma/lzma_common.h
View file

@ -88,14 +88,14 @@ typedef enum {
#define update_literal_normal(state) \
state = ((state) <= STATE_SHORTREP_LIT_LIT \
? STATE_LIT_LIT \
: (state) - 3);
: (state) - 3)
/// Like update_literal(state) but when it is already known that
/// is_literal_state(state) is false.
#define update_literal_matched(state) \
state = ((state) <= STATE_LIT_SHORTREP \
? (state) - 3 \
: (state) - 6);
: (state) - 6)
/// Indicate that the latest state was a match.
#define update_match(state) \

11
src/liblzma/lzma/lzma_decoder.h
View file

@ -23,6 +23,12 @@ extern lzma_ret lzma_lzma_decoder_init(lzma_next_coder *next,
extern uint64_t lzma_lzma_decoder_memusage(const void *options);
/// Gets memory usage without validating lc/lp/pb. This is used by LZMA2
/// decoder, because raw LZMA2 decoding doesn't need lc/lp/pb. Also
/// alone_decoder.c and lzip_decoder.c use this because there lc/lp/pb
/// are known to be valid.
extern uint64_t lzma_lzma_decoder_memusage_nocheck(const void *options);
extern lzma_ret lzma_lzma_props_decode(
void **options, const lzma_allocator *allocator,
const uint8_t *props, size_t props_size);
@ -42,11 +48,6 @@ extern bool lzma_lzma_lclppb_decode(
extern lzma_ret lzma_lzma_decoder_create(
lzma_lz_decoder *lz, const lzma_allocator *allocator,
const lzma_options_lzma *opt, lzma_lz_options *lz_options);
/// Gets memory usage without validating lc/lp/pb. This is used by LZMA2
/// decoder, because raw LZMA2 decoding doesn't need lc/lp/pb.
extern uint64_t lzma_lzma_decoder_memusage_nocheck(const void *options);
#endif
#endif

6
src/liblzma/rangecoder/range_decoder.h
View file

@ -246,14 +246,14 @@ do { \
#define rc_bit(prob, action0, action1) \
rc_bit_last(prob, \
symbol <<= 1; action0, \
symbol = (symbol << 1) + 1; action1);
symbol = (symbol << 1) + 1; action1)
#define rc_bit_safe(prob, action0, action1, seq) \
rc_bit_last_safe(prob, \
symbol <<= 1; action0, \
symbol = (symbol << 1) + 1; action1, \
seq);
seq)
// Unroll fixed-sized bittree decoding.
//
@ -327,7 +327,7 @@ do { \
#define rc_bit_add_if_1(probs, dest, value_to_add_if_1) \
rc_bit(probs[symbol], \
, \
dest += value_to_add_if_1);
dest += value_to_add_if_1)
// Matched literal

4
src/liblzma/validate_map.sh
View file

@ -86,7 +86,7 @@ export LC_ALL
STATUS=0
cd "$(dirname "$0")"
cd "$(dirname "$0")" || exit 1
# Get the list of symbols that aren't defined in liblzma_generic.map.
SYMS=$(sed -n 's/^extern LZMA_API([^)]*) \([a-z0-9_]*\)(.*$/\1;/p' \
@ -95,7 +95,7 @@ SYMS=$(sed -n 's/^extern LZMA_API([^)]*) \([a-z0-9_]*\)(.*$/\1;/p' \
| grep -Fve "$(sed '/[{}:*]/d;/^$/d;s/^ //' liblzma_generic.map)")
# Check that there are no old alpha or beta versions listed.
VER=$(cd ../.. && sh build-aux/version.sh)
VER=$(cd ../.. && sh build-aux/version.sh) || exit 1
NAMES=
case $VER in
*alpha | *beta)

1
src/xz/args.c
View file

@ -480,7 +480,6 @@ parse_real(args_info *args, int argc, char **argv)
case OPT_FILTERS_HELP:
// This doesn't return.
message_filters_help();
break;
case OPT_X86:
coder_add_filter(LZMA_FILTER_X86,

3
src/xz/file_io.c
View file

@ -493,7 +493,8 @@ io_sync_dest(file_pair *pair)
return true;
}
#ifndef TUKLIB_DOSLIKE
#if !defined(TUKLIB_DOSLIKE) && !defined(_AIX)
// On AIX, this would fail with EBADF.
if (fsync(pair->dir_fd)) {
message_error(_("%s: Synchronizing the directory of "
"the file failed: %s"),

72
src/xz/hardware.c
View file

@ -11,6 +11,10 @@
#include "private.h"
#ifdef HAVE_GETRLIMIT
# include <sys/resource.h>
#endif
/// Maximum number of worker threads. This can be set with
/// the --threads=NUM command line option.
@ -321,6 +325,74 @@ hardware_init(void)
// /proc/meminfo as the starting point.
memlimit_mt_default = total_ram / 4;
#ifdef HAVE_GETRLIMIT
// Try to set the default multithreaded memory usage limit so that
// we won't exceed resource limits. Exceeding the limits would result
// in allocation failures, which currently make liblzma and xz fail
// (instead of continuing by reducing the number of threads).
const int resources[] = {
RLIMIT_DATA,
# ifdef RLIMIT_AS
RLIMIT_AS, // OpenBSD 7.8 doesn't have RLIMIT_AS.
# endif
# if defined(RLIMIT_VMEM) && RLIMIT_VMEM != RLIMIT_AS
RLIMIT_VMEM, // For Solaris. On FreeBSD this is an alias.
# endif
};
// The resource limits cannot be passed to liblzma directly;
// some margin is required:
// - The memory usage limit counts only liblzma's memory usage,
// but xz itself needs some memory (including gettext usage etc.).
// - Memory allocation has some overhead.
// - Address space limit counts code size too.
//
// The following value is a guess based on quick testing on Linux.
const rlim_t margin = 64 << 20;
for (size_t i = 0; i < ARRAY_SIZE(resources); ++i) {
// glibc: When GNU extensions are enabled, <sys/resource.h>
// declares getrlimit() so that the first argument is an enum
// instead of int as in POSIX. GCC and Clang use unsigned int
// for enums when possible, so a sign conversion occurs when
// resources[i] is convert to the enum type. Clang warns about
// this with -Wsign-conversion but GCC doesn't.
#ifdef __clang__
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wsign-conversion"
#endif
// RLIM_SAVED_* might be used on some 32-bit OSes
// (AIX at least) when the limit doesn't fit in a 32-bit
// unsigned integer. Thus, for us these are the same thing
// as no limit at all.
struct rlimit rl;
if (getrlimit(resources[i], &rl) == 0
&& rl.rlim_cur != RLIM_INFINITY
&& rl.rlim_cur != RLIM_SAVED_CUR
&& rl.rlim_cur != RLIM_SAVED_MAX) {
#ifdef __clang__
# pragma GCC diagnostic pop
#endif
// Subtract the margin from the current resource
// limit, but avoid negative results. Avoid also 0
// because hardware_memlimit_show() (--info-memory)
// treats it specially. In practice, 1 byte is
// effectively 0 anyway.
//
// SUSv2 and POSIX.1-2024 require rlimit_t to be
// unsigned. A cast is needed to silence a compiler
// warning still because, for historical reasons,
// rlim_t is intentionally signed on FreeBSD 14.
const uint64_t rl_with_margin = rl.rlim_cur > margin
? (uint64_t)(rl.rlim_cur - margin) : 1;
// Lower the memory usage limit if needed.
if (memlimit_mt_default > rl_with_margin)
memlimit_mt_default = rl_with_margin;
}
}
#endif
#if SIZE_MAX == UINT32_MAX
// A too high value may cause 32-bit xz to run out of address space.
// Use a conservative maximum value here. A few typical address space

2
src/xz/list.c
View file

@ -655,6 +655,8 @@ parse_check_value(file_pair *pair, const lzma_index_iter *iter)
/// be printed.
/// \param bhi Pointer to structure where to store the information
/// about the Block Header field.
/// \param xfi Pointer to structure where to store the information
/// about the entire .xz file.
///
/// \return False on success, true on error. If an error occurs,
/// the error message is printed too so the caller doesn't

4
src/xz/message.h
View file

@ -144,6 +144,10 @@ extern void message_filename(const char *src_name);
/// given *strm becomes invalid.
///
/// \param strm Pointer to lzma_stream used for the coding.
/// \param is_passthru
/// If true, we are copying input to output without
/// encoding or decoding, and thus cannot use
/// lzma_get_progress().
/// \param in_size Size of the input file, or zero if unknown.
///
extern void message_progress_start(lzma_stream *strm,

26
src/xz/sandbox.c
View file

@ -4,6 +4,15 @@
//
/// \file sandbox.c
/// \brief Sandbox support
///
/// \note When sandbox_init() is called, gettext hasn't been
/// initialized yet, and thus wrapping error messages
/// in _("...") is pointless in that function. In other
/// functions gettext can be used, but the only error message
/// we have is "Failed to enable the sandbox" which should
/// (almost) never occur. If it does occur anyway, leaving
/// the message untranslated can make it easier to find
/// bug reports about the issue.
//
// Author: Lasse Collin
//
@ -71,11 +80,8 @@ prepare_for_strict_sandbox(void)
extern void
sandbox_init(void)
{
if (pledge("stdio rpath wpath cpath fattr", "")) {
// gettext hasn't been initialized yet so
// there's no point to call it here.
if (pledge("stdio rpath wpath cpath fattr", ""))
message_fatal("Failed to enable the sandbox");
}
return;
}
@ -87,7 +93,7 @@ sandbox_enable_read_only(void)
// We will be opening files for reading but
// won't create or remove any files.
if (pledge("stdio rpath", ""))
message_fatal(_("Failed to enable the sandbox"));
message_fatal("Failed to enable the sandbox");
return;
}
@ -103,7 +109,7 @@ sandbox_enable_strict_if_allowed(int src_fd lzma_attribute((__unused__)),
// All files that need to be opened have already been opened.
if (pledge("stdio", ""))
message_fatal(_("Failed to enable the sandbox"));
message_fatal("Failed to enable the sandbox");
return;
}
@ -139,7 +145,7 @@ enable_landlock(uint64_t required_rights)
const int ruleset_fd = my_landlock_create_ruleset(
&attr, sizeof(attr), 0);
if (ruleset_fd < 0)
message_fatal(_("Failed to enable the sandbox"));
message_fatal("Failed to enable the sandbox");
// All files we need should have already been opened. Thus,
// we don't need to add any rules using landlock_add_rule(2)
@ -154,7 +160,7 @@ enable_landlock(uint64_t required_rights)
// prctl(PR_SET_NO_NEW_PRIVS, ...) was already called in
// sandbox_init() so we don't do it here again.
if (my_landlock_restrict_self(ruleset_fd, 0) != 0)
message_fatal(_("Failed to enable the sandbox"));
message_fatal("Failed to enable the sandbox");
(void)close(ruleset_fd);
return;
@ -274,7 +280,7 @@ sandbox_enable_strict_if_allowed(
// If not reading from stdin, remove all capabilities from it.
if (src_fd != STDIN_FILENO && cap_rights_limit(
STDIN_FILENO, cap_rights_clear(&rights)))
STDIN_FILENO, cap_rights_init(&rights)))
goto error;
if (cap_rights_limit(STDOUT_FILENO, cap_rights_init(&rights,
@ -305,7 +311,7 @@ error:
if (errno == ENOSYS)
return;
message_fatal(_("Failed to enable the sandbox"));
message_fatal("Failed to enable the sandbox");
}
#endif

17
src/xz/signals.c
View file

@ -31,9 +31,6 @@ static sigset_t hooked_signals;
/// signals_unblock() are called before signals_init() has been called.
static bool signals_are_initialized = false;
/// signals_block() and signals_unblock() can be called recursively.
static size_t signals_block_count = 0;
static void
signal_handler(int sig)
@ -137,6 +134,10 @@ signals_init(void)
#ifndef __VMS
/// signals_block() and signals_unblock() can be called recursively.
static size_t signals_block_count = 0;
extern void
signals_block(void)
{
@ -186,8 +187,14 @@ signals_exit(void)
sa.sa_handler = SIG_DFL;
sigfillset(&sa.sa_mask);
sa.sa_flags = 0;
sigaction(sig, &sa, NULL);
raise(sig);
// This shouldn't fail, but check it anyway.
if (sigaction(sig, &sa, NULL) == 0)
raise(sig);
// We shouldn't get here, but just in case we do,
// make main() exit with E_ERROR.
set_exit_status(E_ERROR);
#endif
}

4
src/xz/util.c
View file

@ -12,6 +12,10 @@
#include "private.h"
#include <stdarg.h>
#if defined(_WIN32) && !defined(__CYGWIN__)
# include <io.h>
#endif
/// Buffers for uint64_to_str() and uint64_to_nicestr()
static char bufs[4][128];

9
src/xz/xz.1
View file

@ -615,7 +615,7 @@ Compression is not supported.
.IP ""
The
.B .lz
format version 0 and the unextended version 1 are supported.
format versions 0 and 1 are supported.
Version 0 files were produced by
.B lzip
1.3 and older.
@ -625,15 +625,8 @@ People might have old personal files in this format too.
Decompression support for the format version 0 was removed in
.B lzip
1.18.
.IP ""
.B lzip
1.4 and later create files in the format version 1.
The sync flush marker extension to the format version 1 was added in
.B lzip
1.6.
This extension is rarely used and isn't supported by
.B xz
(diagnosed as corrupt input).
.TP
.B raw
Compress or uncompress a raw stream (no headers).

2
src/xzdec/xzdec.c
View file

@ -321,7 +321,7 @@ sandbox_enter(int src_fd)
// If not reading from stdin, remove all capabilities from it.
if (src_fd != STDIN_FILENO && cap_rights_limit(
STDIN_FILENO, cap_rights_clear(&rights)))
STDIN_FILENO, cap_rights_init(&rights)))
goto error;
if (cap_rights_limit(STDOUT_FILENO, cap_rights_init(&rights,