Switch legacy Zuul jobs to native

As part of the switch to Zuul v3, all the existing jobs have been automatically converted from their Jenkins counterparts. Unfortunately not all of them have been migrated to fully native Zuul v3 jobs, making their maintainance more complicated. Legacy jobs, created using automated scripts, brings with them a lot of duplicated code. They also rely on obsoleted (zuul-cloner) or difficult to maintain components (devstack-gate) and in general an older job creation logic. All these factors make their maintainence extremely difficult.

Given the benefits of the usage of native Zuul v3 jobs, it is worth asking why not all the jobs have been converted. There are several reasons. For example, the required functionality for native Tempest tests were not available at the time of the switch. Native Grenade jobs are not yet available, even though some work is in progress.

Nevertheless, the general status improved significantly in the last year and we can finally move the remaining legacy jobs to the native Zuul v3 jobs for all official OpenStack projects.

Champion

Luigi Toscano (tosky)

Gerrit Topic

To facilitate tracking, commits related to this goal should use the gerrit topic:

native-zuulv3-migration

Completion Criteria

  1. All legacy jobs for official OpenStack projects, if still relevant, should be replaced by native Zuul v3 jobs in the master branch.

  2. The replacement jobs should be designed in a way which does not prevent their backport to the stable branches.

Stretch goals

  1. If a replaced job is also used in some stable branches, backport it to all supported branches.

References

The main reference for converting legacy Zuul jobs is the Zuul v3 Migration Guide, originally part of Infrastructure User Manual and since then restored and imported into the testing section of the OpenStack Project Team Guide.

A complete reference to the Zuul jobs is available as part of the Zuul User’s Guide.

During the Dublin PTG 2018 the QA team provided some updates and examples on the Zuul v3 migrations. The slides are not merged yet in the publications repository but the work-in-progress patch can still be useful.

Examples of migration:

Current State / Anticipated Impact

Jobs which executes devstack alone or combined with common operations like running tempest tests have been ported already or can be ported without many problems.

Jobs which requires less standard operations (for example, a cycle of reconfiguration/test executions, or custom tests) require additional work.

While Grenade-based jobs had not yet been available at the time of the approval of the goal, they were completed and merged right before branching Ussuri, and then backported up to Train.

In fact finalizing the effort on Zuul v3 was considered part of the goal, while still moving forward with all the other non-Grenade conversions. As expected, porting most Grenade jobs should now be trivial.

While cleaning the legacy jobs from openstack-zuul-jobs and project-config is not part of the completion criteria, achieving the stretch goal would reduce the amount of items to track in those repositories.