This is a fun one driven by two separate changes. We recently started
checking whether the volume service was available before setting quotas
in order to allow us to use quota set for other services [1]. This
merged a number of weeks ago and was included in 7.1.0. More recently,
we modified DevStack to stop publishing a service catalog entry with a
service type of 'volumev3', preferring instead to use the correct
'block-storage' service type. Taken separately, neither of these changes
would have caused issues. Together, they mean our lookups for the volume
service now fail and we can't set volume quotas.
Fix things by checking for the block-storage service type also. A future
change will raise a warning (later an error) if the volume service is
not found and you're attempting to set a quota since this is clearly a
mistake.
Change-Id: Ibbeef52225e18757cd28d0fbfb14c1ca06975b60
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-bug: #2084580
The name of the errors change and we need to move things around a
little, but it's otherwise a straight swap.
Change-Id: I0a19765ebeaa14c0534faa1542165b76ed2bf4e2
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Unlike cinderclient, SDK attempts to connect to a service as soon as you
create a client. A keystoneauth1.exceptions.catalog.EndpointNotFound
exception can be raised if this service does not exist in the service
catalog. Avoid this for the quota and limits commands by first checking
if the service is enabled.
In the process, we rework the 'is_volume_endpoint_enabled' helper we are
using to check for the existence of the service to *not* require a
volume client, since this was causing a chicken and egg issue for us
(and was also pretty much unnecessary).
Change-Id: I56e68f00ea221d689eb7f668e9e5ffa7d1a20184
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-bug: #2076229
I have seen a few cases where import errors (distutils - I am looking at you)
result in an extension not being available, but there is no indication why this
is the case. We do configure logging, but this happens too late (as part
of the 'cliff.app.App.run' call to execute a command, which calls
osc-lib's 'configure_logging' but which happens long after we've tried
to import our plugins) to be of any use. Instead, make use of a callback
to make it more obvious.
Change-Id: Id68b06161e445b79fe43f463e06cda3c4771ef02
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This change is entirely automated save for the update of some mocks from
'io.open' to '__builtins__.open').
We are keeping this change separate from addition of the actual hook so
that we can ignore the commit later.
Change-Id: I0a9d8736632084473b57b57b693322447d7be519
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Black used with the '-l 79 -S' flags.
A future change will ignore this commit in git-blame history by adding a
'git-blame-ignore-revs' file.
Change-Id: Ifcb3c798666d74d596b8ecb3d6d507f782de7ba5
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
SDK starts caching token in keyring (when available and configured). A
small change is required in OSC not to reject this state.
Overall this helps avoiding reauthentication upon next openstack call.
If token is not valid anymore automatically reauthentication is done.
Depends-On: https://review.opendev.org/c/openstack/openstacksdk/+/735352
Depends-On: https://review.opendev.org/c/openstack/osc-lib/+/765650
Change-Id: I47261a32bd3b106a589974d3de5bf2a6ebd57263
Importing pkg_resources scans every installed distribution to find
all of the entry points. Stevedore is adding a new caching layer using
importlib.metadata, which will not. Switching to the stevedore should
eventually speed up load times, especially for command line apps. This
change makes the switch now to ensure API compatibility.
We were already using stevedore for tests, so this moves the dependency
from test-requirements.txt to requirements.txt and raises the minimum
version to something more recent.
Change-Id: I3e3632783bc745979b6db73e610df8a77ffaceb0
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This patch implements an endpoint lookup when showing limits. This
addresses the issue when showing limits without both Nova and Cinder
and will display limits if one is missing.
Change-Id: I2214b281e0206f8fe117aae52de2bf4c4e2c6525
Closes-bug: #1707960
On loading external plugin, OSC should be robust on importing the
plugin module so that commands from other modules can continue to
execute.
Closes-Bug: #1722008
Change-Id: Ibe716681c7f78fabee31b7ef281af2588d68ab30
In reviews we usually check import grouping but it is boring.
By using flake8-import-order plugin, we can avoid this.
It enforces loose checking so it sounds good to use it.
This flake8 plugin is already used in tempest.
Note that flake8-import-order version is pinned to avoid unexpected
breakage of pep8 job.
Setup for unit tests of hacking rules is tweaked to disable
flake8-import-order checks. This extension assumes an actual file exists
and causes hacking rule unit tests.
Change-Id: I12b596820727aeeb379bee16c2bc993dee9eb637
The commands that do not require authentication sometimes still need
to call ClientManager.is_network_endpoint_enabled() to see if Neutron
is available. Optimize the paths a bit to skip auth when it is not
necessary; the upshot is Neutron will be assumed in these cases now.
This gets a LOT cleaner when it appears is a future osc-lib.
Change-Id: Ifaddc57dfa192bde04d0482e2cdcce111313a22a
OSC networking commands need to authenticate to get
service catalog, then decide to show nova-network or
neutron command help message. Fake token and fake
auth_type in prepare_to_run_command() casue os-cloud-config
use AdminToken auth plugin, but pass all the auth information
(include: username, password and so on) to it, that casue the
class initialization error. Pop the fake token and url, then
try to load auth plugin again to fix the issue.
Change-Id: I8b140f0b0a60681fc2a35a013bb0c84ff8cb9589
Closes-Bug: #1650026
Auth option prompting happens waaaay to early in the default
os-client-config flow, we need to defer it until adter the commands
have been parsed. This is why ClientManager.setup_auth() exists,
as it is not called until the first attempt to connect to a server
occurs. Commands that do not require authentication never hit this.
Also, required options were not being enforced. By doing this we handle
when no authentication info is present, we fail on missing auth-url rather
than attempt to prompt for a password (default auth is password).
Closes-Bug: 1619274
Change-Id: Ia4eae350e6904c9eb2c8507d9b3429fe52418726
Leaving the pw_func uninitialize in osc-lib turned out to be a
bad idea as the test to prompt in setup_auth() doesn't check
for a callback of None.
Also, release note
Change-Id: I8f875fa8a942d02a040238359ee22c603a4e5956
We start by fixing this in the already-present OSC_Config class so OSC
can move forward. This change needs to get ported down into
os-client-config in the near future, maybe even soon enough to make the
client library freeze this week.
* Add the pw-func argument to the OSC_Config (or OpenStackConfig) __init__()
* When looping through the auth options from the KSA plugin look for any
that have a prompt defined and do not have a value already, so ask for one.
Closes-bug: #1617384
Change-Id: Ic86d56b8a6844516292fb74513712b486fec4442
Convert to using ClientManager and OpenStackShell from osc-lib.
* Change all internal uses of ClientManager private attributes that are
now public in osc-lib's ClientManager. Leave back-compat copies in
place in OSC's clientManager so we don't break plugins.
* Put some work-arounds in place for changes in osc-lib that we need until
a new release makes it through the g-r and u-c change process.
* Add a test for Unicode decoding of argv in shell.main() to parallel
the one in osc-lib.
Change-Id: I85289740d4ca081f2aca8c9b40ec422ad25d302c
* Add compatibility for plugin v2 interface removed from osc-lib
* ClientManager.is_network_endpoint_enabled() is wrapper for
new is_service_available()
Change-Id: I6f26ce9e4d0702f50c7949bacfbeeb0f98cddb5d
These were removed prematurely from osc-lib (by me) but the real fix in
https://review.openstack.org/329189 is having racy functional test issues
that may be related to osc-lib, so let's clear this up while we fix that...
Change-Id: I8f67466967751fdf6fd24ae1b16ccee2aec52323
The code is setting defaults for some scope parameters, cheking if the
name ends with some specific substring (namely ending in "password")
causing failures in some plugins that end with the same string, but do
not allow those parameters (like "user_domain_id" in "v3oidcpassword").
Closes-Bug: #1582774
Change-Id: Id7036db3b783b135353d035dc4c1df7c808d6474
This changes the scope validation to occur after a token has already
been created.
Previous flow:
1. Validate authentication options.
2. Validate authorization options if the command requires a scope.
3. Create a token (using authentication + authorization options)
4. Run command.
This means that scope was being checked, even if a default scope was
applied in step 3 by Keystone.
New flow:
1. Validate authentication options.
2. Create token (using authentication + authorization options)
3 Validate authorization options if the command requires a scope and
the token is not scoped.
4. Run command.
Change-Id: Idae368a11249f425b14b891fc68b4176e2b3e981
Closes-Bug: 1592062
Use osc-lib directly for exceptions.
Leave openstackclient.common.exceptions for deprecation period.
Change-Id: Iea3e862302372e1b31ccd27f69db59b4953ca828
Currently OpenStackClient uses keystoneclient for authentication.
This change will update OpenStackClient to use keystoneauth for
authentication.
All dependant test have been updated.
Updating how auth_ref is set in the tests to use KSA fixtures had
some racy side-effects. The user_role_list tests failed when they
picked up an auth_ref that was a fixture. This exposed a weakness
in ListUserRole that needed to be fixed at the same time re
handling of unscoped tokens and options.
Change-Id: I4ddb2dbbb3bf2ab37494468eaf65cef9213a6e00
Closes-Bug: 1533369
Currently, "/usr/bin/openstack --insecure token issue"
fails when OS_AUTH_URL and OS_IDENTITY_API_VERSION
indicate keystone v2 if OS_PROJECT_DOMAIN_NAME or
OS_USER_DOMAIN_NAME are set. This patchset ignore domain
related configs if using with keystone v2 and print warning
for each ignored config.
Change-Id: I8afbda787df7855c3f8e868b0f07cbf3b9cd97fd
Closes-bug: #1447704
Currently, "/usr/bin/openstack --insecure token issue"
fails when OS_AUTH_URL and OS_IDENTITY_API_VERSION
indicate keystone v2 if OS_PROJECT_DOMAIN_NAME or
OS_USER_DOMAIN_NAME are set. This patchset ignore domain
related configs if using with keystone v2.
Change-Id: If7eea2ed1a4877c60d055ed0114a5e5f31e282a0
Closes-bug: #1447704
Previously, if an AttributeError was raised in a plugin's make_client
method, the plugin simply wouldn't be an attribute of the ClientManager,
producing tracebacks like
Traceback (most recent call last):
File ".../openstackclient/shell.py", line 118, in run
ret_val = super(OpenStackShell, self).run(argv)
...
File ".../openstackclient/object/v1/container.py", line 150, in take_action
data = self.app.client_manager.object_store.container_list(
File ".../openstackclient/common/clientmanager.py", line 66, in __getattr__
raise AttributeError(name)
AttributeError: object_store
This made writing minimal third-party auth plugins difficult, as it
obliterated the original AttributeError.
Now, AttributeErrors that are raised during plugin initialization will
be re-raised as PluginAttributeErrors, and the original traceback will
be preserved. This gives much more useful information to plugin
developers, as in
Traceback (most recent call last):
File ".../openstackclient/shell.py", line 118, in run
ret_val = super(OpenStackShell, self).run(argv)
...
File ".../openstackclient/object/v1/container.py", line 150, in take_action
data = self.app.client_manager.object_store.container_list(
File ".../openstackclient/common/clientmanager.py", line 57, in __get__
err_val, err_tb)
File ".../openstackclient/common/clientmanager.py", line 51, in __get__
self._handle = self.factory(instance)
File ".../openstackclient/object/client.py", line 35, in make_client
interface=instance._interface,
File ".../openstackclient/common/clientmanager.py", line 258,
in get_endpoint_for_service_type
endpoint = self.auth_ref.service_catalog.url_for(
PluginAttributeError: 'NoneType' object has no attribute 'url_for'
Change-Id: I0eee7eba6eccc6d471a699a381185c4e76da10bd
`dict.get()` returns `None` by default, if a key wasn't found.
Removing `None` as second argument to avoid redundancy.
Change-Id: Ia82f7469cd019509bbeccbfe54b15eeedc7bb6ea
There are files containing string format arguments inside logging messages.
Using logging function parameters should be preferred.
Change-Id: I15b405bf4d4715263fe1e1262982467b3d4bc1f4
Closes-Bug: #1321274
Make scope check optional for the "token issue" command as unscoped token is
a valid Keystone V2/V3 API.
Change-Id: Ie1cded4dbfdafd3a78c0ebdf89e3f66762509930
Closes-Bug: #1543214
Move the network endpoint enablement checking from the 'server create'
command to the common client manager. This allows future network
commands to use either nova or neutron networking based on the cloud
environment.
This patch set also includes related unit test enhancements to the
common client manager to trigger authentication on the tests.
Change-Id: Ia37e81d4fb05a1e2fceb3e5d367bda769ab8e64b
Related-Bug: #1519511
Related-to: blueprint neutron-client
Create a `configuration show` command that displays the current
configuration of the CLI. Different configurations can be
displayed using options such as --os-cloud. Passwords and
tokens are redacted by default unless the --unmask
option is specified.
Closes-Bug: #1476729
Change-Id: I0792365d0c5fa526cd09c0ed88c6bb1e2cb813a7
External plugins need to know which plugin interface is being
used by the host OSC. Releases <1.6 (?) will not have a
version defined.
Plugins can add the following to their make_client() to discover
the interface version:
_plugin_interface_version = getattr(
instance,
"PLUGIN_INTERFACE_VERSION",
None,
)
Change-Id: Ifc0f40fec5bc27f6425139984936b7f6e032a580
