[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-October/018445.html
Conflicts:
.zuul.yaml
lower-constraints.txt
Changes:
bindep.txt
NOTE(stephenfin): Conflicts are due to the changed job template (ussuri
vs victoria) and slightly different lower constraints between branches.
We also need to remove 'libffi6' from 'bindep.txt' since Focal doesn't
have this. This was done in the Victoria cycle as part of change
Iab3634039845adb649c7fd69d1812b405a61433c but we must squash it in here.
Change-Id: I45e9a81d451c64cdd51f9b606d94161742bacdb7
(cherry picked from commit c677192d51)
(cherry picked from commit cde8db3d61)
(cherry picked from commit 40f56c653c)
(cherry picked from commit 84af7608ef)
The patch https://review.opendev.org/#/c/673389/ introduced a regression
by changing the osc-lib interface.
Two conflicting attempts to fix the regression were launched:
1) Reverting the patch.
2) The patch https://review.opendev.org/683119 changes the exception
from the generic CommandError back to a specific Forbidden exception.
The patch https://review.opendev.org/683118 catches this exception
and passes on, i.e. re-implements the same behavior as before.
The first idea was implemented, the initial patch reverted. The second
idea was partially implemented. The change in python-openstackclient
(683118) was merged. The change in osc-lib was approved but failed to
merge because the initial change had been reverted.
Now we have again a situation where the exception produced in osc-lib
does not match the exception expected by the caller.
It is unclear if the osc-lib interface will ever get a rebased version
of https://review.opendev.org/683119 merged, so the safest way to
address the issue is to also catch the exception that used to be
thrown before the inital change and is again thrown after the inital
change has been reverted.
Change-Id: I2ea2def607ec5be112e42d53a1e660fef0cdd69c
(cherry picked from commit 0a8753dc3e)
pip 20.3 finally includes a proper dependency resolver. Its use is
causing the following error messages on the lower-constraints job:
ERROR: Cannot install ... because these package versions have
conflicting dependencies.
The conflict is caused by:
bandit 1.1.0 depends on PyYAML>=3.1.0
cliff 3.4.0 depends on PyYAML>=3.12
openstacksdk 0.52.0 depends on PyYAML>=3.13
Bump our lower constraint for PyYAML to resolve this issue. With that
resolved, we see a new issue:
ERROR: Could not find a version that satisfies the requirement
cryptography>=2.7 (from openstacksdk)
ERROR: No matching distribution found for cryptography>=2.7
This is less self-explanatory but looking at the lower-constraints for
openstacksdk 0.52.0 shows a dependency on cryptography 2.7 [1], meaning
we need to bump this also.
Next up, flake8-import-order seems to cause the dependency resolver to
go nuts, eventually ending with the following error message in a Python
3.6 environment:
Using cached enum34-1.1.2.zip (49 kB)
ERROR: Command errored out with exit status 1:
command: ...
cwd: ...
Complete output (9 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File ".../lib/python3.6/site-packages/setuptools/__init__.py", line 7, in <module>
import setuptools.distutils_patch # noqa: F401
File ".../lib/python3.6/site-packages/setuptools/distutils_patch.py", line 9, in <module>
import re
File "/usr/lib64/python3.6/re.py", line 142, in <module>
class RegexFlag(enum.IntFlag):
AttributeError: module 'enum' has no attribute 'IntFlag'
----------------------------------------
A quick Google suggests this is because the enum34 package is not
complete [2]. We shouldn't even be using it since our base virtualenv
should at least use Python 3.6, but I guess some dependency doesn't
properly restrict the dependency to <= Python 3.4. This is moved from
'test-requirements.txt' to 'tox.ini' since we don't need to use our
constraints machinery for linters.
Finally, the versions of bandit and hacking that pip is bringing in both
requires in a newer version of babel, which in turn requires a new
version of pytz.
Collecting hacking>=2.0.0
...
ERROR: Cannot install oslo.i18n because these package versions have
conflicting dependencies.
The conflict is caused by:
babel 2.9.0 depends on pytz>=2015.7
babel 2.8.1 depends on pytz>=2015.7
babel 2.8.0 depends on pytz>=2015.7
babel 2.7.0 depends on pytz>=2015.7
Seeing as we shouldn't be tracking bandit in
lower-constraints, I'm not sure why we're want to bump these
dependencies for just that. As above, we move these dependencies out of
'test-requirements' and into 'tox.ini' since we can do that for linters.
Conflicts:
lower-constraints.txt
test-requirements.txt
NOTE(stephenfin): Conflicts are due to the absence of ddt and presence
of mock in lower-constraints.txt and test-requirements.txt,
respectively.
Modifications:
lower-constraints.txt
NOTE(stephenfin): There's no need to bump cryptography here since we're
using an older version of openstacksdk.
[1] https://opendev.org/openstack/openstacksdk/src/tag/0.52.0/requirements.txt#L19
[2] https://github.com/iterative/dvc/issues/1995#issuecomment-491889669
Change-Id: I8ec738fbcabc8d8553db79a876e5592576cd18fa
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
(cherry picked from commit 20769cd7b2)
(cherry picked from commit 83cd9b5b9c)
Neither Tempest itself nor any of the service projects use OSC. As such,
there's no reason to run Tempest jobs here. It's simply a waste of
resources.
Change-Id: I74b0b196fe59e5e1462e3dadc659cf6680a53f80
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
(cherry picked from commit 671f73694a)
(cherry picked from commit 75847d1e96)
Keystone let's users remove role assignments that reference non-existent
users and groups. This is nice when keystone backs to an identity store
like LDAP and users or groups are removed.
Previously, openstackclient would validate the user and group existed in
keystone before sending the request to delete the role assignment. This
commit updates the code to bypass that validation so that users can use
IDs to forcibly cleanup role assignments.
Change-Id: I102b41677736bbe37a82abaa3c5b3e1faf2475d5
Story: 2006635
Task: 36848
(cherry picked from commit e246732670)
I was writing some additional functionality and noticed these tests were
missing. This commit adds tests for adding and removing system role
assignments for users and groups.
Change-Id: I30fdc6ec55e1eb1cfa55f4cbf92c3f001d89865f
(cherry picked from commit a8aad9fec8)
They work just fine in Volume API v3 but they were limited in OSC
to v2.
Change-Id: I510383f8e0cbf05ec24caa1cad330f12f82a913d
Story: 2007896
Task: 40279
(cherry picked from commit b1fc587a6d)
assertItemsEqual was removed from Python's unittest.TestCase in
Python 3.3 [1][2]. We have been able to use them since then, because
testtools required unittest2, which still included it. With testtools
removing Python 2.7 support [3][4], we will lose support for
assertItemsEqual, so we should switch to use assertCountEqual.
[1] - https://bugs.python.org/issue17866
[2] - https://hg.python.org/cpython/rev/d9921cb6e3cd
[3] - testing-cabal/testtools#286
[4] - testing-cabal/testtools#277
Change-Id: I1ad0da8deda3a8cbec384b5a9c88860a526eb48c
(cherry picked from commit a15b1addb4)
When we use "--property" parameter, client get lists these the
value is string type, but the type of the value 'is_domain'
should be boolean, so we should judge it and parse it.
The patch parse string to boolean for value 'is_domain'.
Co-Authored-By: Lance Bragstad <lbragstad@gmail.com>
Conflict:
Direct backports of this patch fail because the original tests
proposed to the Victoria (master) branch included keystone
``options``. Support for ``options`` was added in:
I9c3bdd741f28bf558267fb217818d947597ce13e
This backport removes the ``options`` key from the expected values in
the tests since feature support for ``options`` isn't going to be
backported. Otherwise, the functionality of this change is fully
tested like it is on later releases.
Change-Id: I37c9eb854524bde3a1530bfe2e3a03810fb1a676
Task: 30039
Story: 2005246
(cherry picked from commit 533af9f1b2)
Update the URL to the upper-constraints file to point to the redirect
rule on releases.openstack.org so that anyone working on this branch
will switch to the correct upper-constraints list automatically when
the requirements repository branches.
Until the requirements repository has as stable/ussuri branch, tests will
continue to use the upper-constraints list on master.
Change-Id: Ie762327f9127b2aa75c4e5a3dae5ad384e07b074
Squashed two changes together since both are needed to fix jobs:
Resolve PEP8
No idea how this happened, but reviews started failing the
pep8 gate job. The failures are legitimate, see the commit.
I guess the pep8 tests became smarter and found these issues.
(cherry picked from commit 52ff421e3d)
Add libc6-dev to bindep
The python-builder base image was updated to no longer install
recommends. This is inline with the other Infra images and keeps
image sizes smaller. gcc recommended libc6-dev - but it turns out
we need that for limits.h for one of our depends. Add it to fix
our image builds.
(cherry picked from commit 97d027caec)
Change-Id: Id9a0dad644134dafd68eed37fe8f41c583d7a619
There is a much deeper and systemic issue going on here, but let's
start with fixing the immediate issue which is that adding a project
to an image fails trying to look up project information even if the
user passes the project id by id.
_is_uuid_like from sdk isn't perfect, but it'll be good enough
for this.
Change-Id: I541416d737b961c56aa2f584c172528632fd5537
In https://review.opendev.org/#/c/650374/ a work has been started to
switch image service support from glanceclient with all it's
dependencies to the SDK version. With this change version 1 (anyway
deprecated since ages) is also being switched to SDK.
Change-Id: Ic391500af02a73d81d64a9e9113cca85c9e24390
I'm guessing we should do this for everyone, but we have volume on
the brain right now. Rackspace is in the weird situation where
they do support v2 but only have v1 in the catalog (wut) So we
need to override the block-storage enpdoint by config. To do that,
we need to actually honor the config setting over here in OSC.
NOTE: We need to systemically overhaul how we're injesting config
over here - because there's too much variation. But we can leave
that for another day.
Story: 2007459
Task: 39137
Change-Id: Ifddf1ddd5abaa768ab18049c09d18bc269f3a4f5
This is a work to switch OSC from using glanceclient to OpenStackSDK.
With this change only v2 is using OpenStackSDK. V1 is still using
glanceclient and will be switched in a separate change.
Remove the direct depend on keystoneauth- let that flow through
openstacksdk.
Depends-on: https://review.opendev.org/#/c/698972
Change-Id: I36f292fb70c98f6e558f58be55d533d979c47ca7
This function should return an ordered set of ranges based on an
unordered list of numbers (int or str).
Change-Id: I918c8befc51236cc33d96a5c88fb6eafdd143e9c
Story: 2007341
Task: 38878
setuptools 46.0.0's drop of the Features feature broke
python-openstackclient's lower-constraints job on master
via the MarkupSafe package. Bump the lower constraint of
MarkupSafe to fix lower-constraints on master.
Change-Id: Ib0a6f94a6611b221efbf76f6f25b55c43782546f
We're failing on promote but not upload. That's weird. Make sure
the secret is appropriately encoded, and copy what zuul is doing.
Also make promote a zero-node job.
Change-Id: Ifcb5b4fe2486087a5ca1ff9609f7bf09ef026974
python-openstackclient currently has a non-zero number of dependencies,
so for admins who would like to run it on laptops or similar it can
get tricky. In opendev, for instance, admins have it installed into
a venv on a jump host, but it's really wonky to keep up with.
Use the opendev/python-builder opendev/python-base pair to make a
minimal image that contains an install of python-openstackclient
and publish it to the osclient org on dockerhub. There is an overall
policy against having binary artifacts such as this appear to be
official deliverables of the OpenStack project, which this is not.
It's also only publishing images based on master, so no warranties
should be implied. But if this makes life easier for a user somewhere,
cool.
Change-Id: I9a8bfc27c127e92b6856cb6a3e45b32c818db16c
This new query parameter will allow to send a query sending the
"fields" parameter. This "fields" parameter contains the needed
API fields, translated into OVO fields in Neutron server, that
require to be retrieved from the DB.
As commented in the related bug, the OSC "list" command only
prints five parameters, none of them the security group rules. In
systems with a reasonable amount of security groups, skipping the
unnecessary rule load can save a lot of time.
Depends-On: https://review.opendev.org/#/c/710820/
Change-Id: I16f48e292997d029d68f66365db949b9f4b5a0c8
Closes-Bug: #1865223
