.. _periodicwork:

=============
Periodic Work
=============

Releasing
=========

Our release frequency is discussed in :ref:`reference_release`.

OSA CLI tooling
---------------

OpenStack-Ansible used to bump everything in a single script, which
made it hard to maintain, and was very branch specific. It made it
hard for users to consume either an update of the upstream shas, or
to bump roles with their own pace.

Since then, the OpenStack-Ansible has agreed to provide more metadata
necessary for releasing into the openstack-ansible code tree. This
allowed the tooling for releasing to be more flexible,
and lighter, over time.

All the functions are separated, and included as an extra console
script ``openstack-ansible-releases``.

By default, dependencies for the script are not installed to minimize amount
of them for regular users.
To ensure all required dependencies are installed for the script, you can run:

.. parsed-literal::

   pip3 install -e git+https://opendev.org/openstack/openstack-ansible#egg=openstack-ansible[releases]

Each subcommand contains help by default.

Updating upstream SHAs
----------------------

The dependencies for OpenStack-Ansible are updated
through the use of ``openstack-ansible-releases bump_upstream_shas``. This script
updates the project's pinned SHAs, located in the
``inventory/group_vars/<service_group>/source_git.yml`` file,
based on their ``_git_track_branch`` value.


Updating OpenStack-Ansible roles
--------------------------------

Updating the roles to their latest version per branch is done through
``openstack-ansible-releases bump_roles``.

This can do multiple things:

* Freeze ansible-role-requirements to their latest SHAs for the branch
  they are tracking.
* Copy release notes relevant to the freeze.
* Unfreeze of master.

Master doesn't get frozen, unless explicitly asked for it for release
milestones, using the command ``openstack-ansible-releases freeze_roles_for_milestone``

Updating required collections
-----------------------------

Updating collections to their latest version is done through
``openstack-ansible-releases bump_collections``.

Please note, that only collections with type `git` are supported.
Ones, that are installed from Ansible Galaxy should be bumped manually.


Check current pins status
-------------------------

You can check the current PyPI pins that are used in openstack-ansible
repository by running ``openstack-ansible-releases check_pins``. This will display
a table, showing the current pin in OSA, and what is available upstream on
PyPI.

This doesn't patch the ``global-requirements-pins``, as this should be
a manual operation. See the :ref:`cycle-checklist` to know when to bump
the global-requirements-pins.

Adding patch in releases repo
-----------------------------

When the patches to update SHAs and roles have landed, you can propose the
parent SHA as a release in the releases repo.

This can be done using the ``new-release`` command, and then editing
the SHA used for openstack-ansible. See also `new_releases page`_ for an
explanation of this command.

Please note that branches before Stein will require cleanup of the YAML file
generated by new_releases, as it will contain ALL the roles and
openstack-ansible repo SHAs. We have decided to NOT tag the roles anymore,
so you will need to remove all the lines which are not relevant to
the `openstack-ansible` repository.

.. _new_releases page: https://releases.openstack.org/reference/using.html#using-new-release-command


Trasition releases to Extended Maintenance
------------------------------------------

With migrating a release to `Extended Maintenance`_ (EM) status no future
releases of the release are possible. With that, a new tag <release>-em is
created and it must be based on the latest numeric release.

To transition a release to Extended Maintenance we need to:

  * Wait until all upstream repositories will transition to EM

  * Update OpenStack-Ansible roles normally

  * Update ``<service>_git_install_branch`` to be ``<release>-em`` instead of
    the SHA.

  * Propose a new numeric release to ``openstack/releases`` repository

  * Propose to create a ``<release>-em`` tag with the same SHA as previous
    numeric release has.

  * Once EM tag is created, switch ``version`` field in
    ansible-role-requirements.yml to track ``stable/<release>``.

  * Update ``index.rst`` for master branch to reflect support status of the
    release

.. _Extended Maintenance: https://docs.openstack.org/project-team-guide/stable-branches.html#extended-maintenance


Trasition releases to End Of Life
---------------------------------

With releases reaching `End Of Life`_ (EOL), a new tag ``<release>-eol`` is
created, after which branch will be deleted. Projects are free to EOL their
branches anytime, after coordinated migration to EM is completed. Due to this
we don't track ``stable/<release>`` branch for upstream services for EM, but
only for our roles. At the same time there is a coordinated deadline, when all
projects must EOL their old branches.


To transition a release to End Of Life we need to:

  * Create a <release>-eol tag for OpenStack-Ansible Roles

  * Switch ansible-role-requirements.yml ``version`` field to <release>-eol tag
    from stable/<release> branch.

  * Wait for all projects to EOL

  * Update ``<service>_git_install_branch`` variables to use <release>-eol tag
    instead of SHAs.

  * Create a <release>-eol tag for OpenStack-Ansible repository

  * Update release support status in index.rst on the master branch.


.. _End Of Life: https://docs.openstack.org/project-team-guide/stable-branches.html#end-of-life


.. _cycle-checklist:

Development cycle checklist
===========================

On top of the normal cycle goals, a contributor can help the OpenStack-Ansible
development team by performing one of the following recurring tasks:

* By milestone 1:

  * Community goal acknowledgement.

  * Define supported platforms release will run on. Remove testing and support
    for deprecated ones.

  * Update ``global-requirements-pins``, upstream SHAs and required collections

* By milestone 2:

  * Handle deprecations from upstream project's previous cycle.

  * Handle OpenStack-Ansible roles deprecations from the previous cycle.

  * Refresh static elements in roles. For example, update a specific version of
    the software packages.

  * Update release-versioned components such as Octavia test ampohora image
    and Ironic IPA image/kernel.

  * Bump ``ceph_stable_release`` to latest Ceph LTS release in the integrated
    OpenStack-Ansible repo, and inside the ``ceph_client`` role defaults.

  * Check and bump galera versions if required.

  * Check and bump rabbitmq versions if required.

  * Check outstanding reviews and move to merge or abandon them if no longer
    valid.

  * Update ``global-requirements-pins``, upstream SHAs and required collections

* By milestone 3:

  * Implement features

  * Update ``global-requirements-pins``, upstream SHAs and required collections

* After milestone 3:

  * Feature freeze, bug fixes, and testing improvements.

  * Ansible version and collections freeze

* After a new stable branch is created for services:

  * Update ``<service>_git_track_branch`` variables to match the new
    branch name.

  * Set all ``tempest_plugin_<service>_git_track_branch`` to None to
    prevent SHA update for them.

  * Update upstream SHAs

* After coordinated OpenStack release, before OpenStack-Ansible release:

  * Update release name in ``openstack_hosts`` repository. This will also bump
    RDO and Ubuntu Cloud Archive repositories.

  * Branch all the independent repos that aren't part of the release
    in gerrit. See also the ``projects.yaml`` in the governance repo.
    Manually branched repos need extra
    editions, like updating the .gitreview, or the reno index.
    Please reference previous branch creations by using the
    appropriate topic in gerrit (e.g.: ``create-stein``).
    The previous new branch creation may be different as the
    tooling/process may have evolved, so be aware that the changes
    needed may need to be adapted.

  * Switch ``trackbranch`` in ansible-role-requirements.yml to the new branch
    and update OpenStack-Ansible roles after that.

  * Add supported platforms for the release to ``os-compatibility-matrix.html``

* Immediately after official OpenStack-Ansible release:

  * Send a thank you note to all the contributors through the mailing lists.
    They deserve it.

  * Revert changes made to ansible-role-requirements.yml and
    ``*_git_track_branch`` variables to track stable branch instead of master.
    Update upstream SHAs.

  * Reflect changes in documentation

      * Create a patch to openstack-manuals and uncomment OpenStack-Ansible in
        `www/project-data/<release>.yaml`.

      * Update index page on master to mention release date of recently
        released version as well as set its status to Maintained. With that
        also add a new release that we are about to start working on.

      * Update index page on stable branch to mention only the release in topic
        rather all historical releases as well. Historical data should be
        present only on master branch.

      * Update upgrade scripts and documentation to keep track on supported
        upgrade paths:

        * For SLURP releases, define ``previous_slurp_name`` in
          doc/source/conf.py. For non-SLURP - set it to ``False``.

        * Update ``previous_series_name`` and ``current_series_name`` in
          doc/source/conf.py and deploy-guide/source/conf.py

        * Update ``UPGRADE_SOURCE_RELEASE`` in scripts/gate-check-commit.sh

        * Update ``SUPPORTED_SOURCE_SERIES`` and ``TARGET_SERIES`` in
          scripts/run-upgrade.sh. Also don't forget to cleanup irrelevant
          upgrade scripts.

        * Add/remove required for SLURP upgrade jobs. ACTION for such jobs
          should be defined as ``upgrade_<release>``.
