Control Plane API endpoints deployment via WSGI

In some projects, API services can be run:

  1. As a Python command that runs a web server (e.g. wsgiref or werkzeug).
  2. As a WSGI application hosted by performant web servers (e.g. Apache with mod_wsgi or Nginx with uWSGI).

When a project provides a WSGI application the API service gains flexibility in terms of deployment, performance, configuration and scaling.

Gerrit Topic

To facilitate tracking, commits related to this goal should use the gerrit topic:

goal-deploy-api-in-wsgi

Completion Criteria

For all projects:

  1. Provide WSGI application script file(s) (e.g. to be used by web server). There shouldn’t be any web server restriction and the application could be deploying to any web server that support WSGI applications.
  2. Switch devstack jobs to deploy control-plane API services under uwsgi with Apache acting as a front end proxy.

uwsgi vs. mod_wsgi

When this effort was first approved, the push was to use mod_wsgi. mod_wsgi has many issues when being used in devstack for development or testing.

  • services log to a very different location / format
  • start/stop/restart/status of services is now very different depending on if they are api services or workers
  • apache restarts trigger restarts of all API services, and take long enough that intermittent races can occur.
  • API servers still need dedicated ports for certain parts of their config.

The effort was pushed forward because at the time no one signed up to do the ground work for the uwsgi transition in devstack. That work was done here - http://lists.openstack.org/pipermail/openstack-dev/2017-April/115423.html with the intent that the mod_wsgi support is deleted from devstack in Queens.

Current State / Anticipated Impact

On 12 Jan 2017 a review of git repositories owned by official projects showed the projects that don’t support their control-plane API services deployed via WSGI:

  • cloudkitty
  • congress
  • designate
  • glance
  • kuryr-libnetwork
  • magnum
  • murano
  • neutron
  • octavia
  • searchlight
  • solum
  • tacker
  • trove
  • vitrage
  • watcher
  • zun

These projects already deploy devstack with API service via WSGI:

  • aodh
  • ceilometer
  • cinder
  • freezer
  • gnocchi
  • ironic
  • keystone
  • mistral
  • nova
  • panko
  • swift
  • zaqar
  • (...)

Project Teams

Chef OpenStack

The Chef cookbooks do not provide any API directly, they consume downstream packages, and thus are not directly affected by this goal. When packages support deploying API services via WSGI, the corresponding cookbooks use it.

Planning Artifacts: None

Completion Artifacts: None

cinder

Planning Artifacts:

DevStack change still needed to properly run cinder-api under Apache:

https://review.openstack.org/#/c/441266/

Completion Artifacts:

cloudkitty

Planning Artifacts:

Completion Artifacts:

https://review.openstack.org/#/c/366043/

Community App Catalog

Planning Artifacts:

Completion Artifacts:

congress

Planning Artifacts:

Completion Artifacts:

designate

Planning Artifacts:

Completion Artifacts:

Documentation

Planning Artifacts:

Note: Dependent on upstream projects achieving deploy-api-in-wsgi goal.

Completion Artifacts:

dragonflow

Planning Artifacts:

Completion Artifacts:

ec2-api

Planning Artifacts:

Completion Artifacts:

freezer

Planning Artifacts:

  • Freezer has no planning documents at this time since support was introduced prior to Newton.

Completion Artifacts:

Freezer is already using wsgi to deploy the api service since Newton release. Freezer supports two ways of running dsvm gate job, apache2 (with mod_wsgi) or apache2 (with mod_proxy and uwsgi). The default way for running devstack is apache2 with mod_proxy and uwsgi.

https://review.openstack.org/#/c/471080/

fuel

Planning Artifacts:

Completion Artifacts:

glance

Planning Artifacts:

Completion Artifacts:

heat

Planning Artifacts:

  • Heat has no planning documents at this time since the support was introduced and enabled by default at Ocata.

Completion Artifacts:

horizon

Planning Artifacts:

Completion Artifacts:

I18n

Planning Artifacts:

  • The I18n team does not have any API services and therefore has nothing to do

Completion Artifacts:

  • None

Infrastructure

The Infrastructure team does not maintain any OpenStack trademark program services, much less any with REST APIs, so has no current need for WSGI conversion.

ironic

Planning Artifacts:

Completion Artifacts:

  • ironic: PARTIAL (using mod_wsgi instead of uWSGI)
  • ironic-inspector: TODO (Queens)

keystone

Planning Artifacts:

  • Keystone has no planning documents at this time since support was introduced prior to Kilo.

Completion Artifacts:

kolla

Planning Artifacts:

Completion Artifacts:

kuryr

Planning Artifacts:

Completion Artifacts:

magnum

Planning Artifacts:

Completion Artifacts:

manila

Planning Artifacts:

Completion Artifacts:

mistral

Planning Artifacts:

Completion Artifacts:

neutron

Planning Artifacts:

Completion Artifacts:

nova

Planning Artifacts:

Nova is tracking the work in the devstack-uwsgi etherpad. The placement service already runs under mod_wsgi in devstack but that will be changed to uwsgi. There is also a bug in nova-api that needs to be fixed before we can deploy it under uswgi in devstack for testing.

Completion Artifacts:

octavia

Planning Artifacts:

The octavia API is already implemented as a wsgi application, we just need to setup the web server integration. This is work in progress here: https://review.openstack.org/440934

Completion Artifacts:

OpenStack Charms

Planning Artifacts:

Completion Artifacts:

OpenStack UX

Planning Artifacts:

Completion Artifacts:

OpenStackAnsible

Planning Artifacts:

NB Individual roles are dependent on the upstream project achieving the deploy-api-in-wsgi goal.

Completion Artifacts:

OpenStackClient

Planning Artifacts:

None of the OpenStackClient deliverables have services so no work is required for this goal.

Completion Artifacts:

oslo

Planning Artifacts:

Completion Artifacts:

Packaging-deb

Planning Artifacts:

Completion Artifacts:

Packaging-rpm

Planning Artifacts:

Completion Artifacts:

Puppet OpenStack

Planning Artifacts:

Projects where we plan to add support:

  • puppet-zaqar

Completion Artifacts:

Projects that already support WSGI deployments for API:

  • puppet-aodh
  • puppet-barbican
  • puppet-ceilometer
  • puppet-cinder
  • puppet-gnocchi
  • puppet-heat
  • puppet-ironic
  • puppet-keystone
  • puppet-mistral
  • puppet-nova
  • puppet-panko
  • puppet-vitrage

Quality Assurance

Planning Artifacts:

  • The only project that includes a python web application is the API part of OpenStack Health, which is not an OpenStack control plane service. OpenStack Health API is deployed as a WSGI application as part of OpenStack infra. Further details in https://etherpad.openstack.org/p/pike-qa-goals-wsgi.

Completion Artifacts:

  • None

rally

Planning Artifacts:

Completion Artifacts:

RefStack

Planning Artifacts:

Completion Artifacts:

Release Management

Planning Artifacts:

  • The Release management team doesn’t have any API services and therefore has nothing to do

Completion Artifacts:

  • None

requirements

Planning Artifacts:

  • The requirements team do not have any API services and therefore has nothing to do.

Completion Artifacts:

  • None

sahara

Planning Artifacts:

Completion Artifacts:

searchlight

Planning Artifacts:

Completion Artifacts:

Security

Planning Artifacts:

Completion Artifacts:

senlin

Planning Artifacts:

Completion Artifacts:

shade

Planning Artifacts:

  • The shade team does not have any API services and therefore has nothing to do.

Completion Artifacts:

  • None

solum

Planning Artifacts:

Completion Artifacts:

Stable branch maintenance

Planning Artifacts:

  • The stable team doesn’t have any code repositories and therefore has nothing to do.

Completion Artifacts:

  • None

swift

Planning Artifacts:

Completion Artifacts:

tacker

Planning Artifacts:

Completion Artifacts:

Telemetry

Planning Artifacts:

Completion Artifacts:

tricircle

Planning Artifacts:

  • None

Completion Artifacts:

tripleo

Planning Artifacts:

During Pike, we plan to migrate some services under WSGI with Apache:

Completion Artifacts:

TripleO already deploy some services under WSGI with Apache:

  • Aodh API
  • Barbican
  • Ceilometer API
  • Cinder API
  • Gnocchi API
  • Keystone
  • Nova Placement
  • Panko API

vitrage

Planning Artifacts:

  • None.

Completion Artifacts:

watcher

Planning Artifacts:

Completion Artifacts:

Watcher API may now works with mod-wsgi. Patchset https://review.openstack.org/#/c/450740/ provided the following changes:

  • wsgi app script files, to run watcher-api under Apache HTTPd.
  • updated devstack plugin to run watcher-api default with mod-wsgi.
  • document to deploy watcher-api behind wsgi.

winstackers

Planning Artifacts:

Completion Artifacts:

zaqar

Planning Artifacts:

Completion Artifacts:

zun

Planning Artifacts:

Completion Artifacts: