3.14 - headline features

We are happy to announce the release of Pulpcore 3.14!

This release contains a number of bug fixes and enhancements. In this blog, we’ll take a look at the headline features.

A gentle reminder that the Community Survey is open, and we would love to hear from you.

Pulp has a new queueless tasking system! #8501

The largest and most significant change introduced with this release is the new tasking system. With this change, Pulp no longer needs a resource manager.

If you’d like an in-depth look at the motivations, benefits, and details of the new tasking system, read Matthias Dellweg’s blog post and also watch a YouTube livestreamed interview with Matthias conducted by Pulp’s Brian Bouterse:

As part of the work to implement this feature, this release also introduces a new more “graceful” shutdown for Pulpcore workers #8930

Note that for the moment, this feature is in tech-preview. Please let us know what you think.

Sticking with the old tasking system?

If you intend to keep using the old tasking system, you should configure the USE_NEW_WORKER_TYPE setting to False before upgrading. For more information about how to enable or disable the new tasking system, see the documentation.

New Orphan Cleanup Endpoint #8658

This release introduces the /pulp/api/v3/orphans/cleanup/ endpoint. When you call this endpoint with POST and no parameters, it is the equivalent of calling. DELETE /pulp/api/v3/orphans/. Note that as part of this release, DELETE /pulp/api/v3/orphans/ has been deprecated.

You can specify a list of content_hrefs to delete using the optional parameter content_hrefs with list of content hrefs. In this case, only the specified content units will be deleted by the orphan cleanup.

Increased Retries on Failed Downloads #8881

The default behavior for retrying downloads from remote sources after an error occurs has been updated. Downloads will now be retried on a wider range of errors, for example, HTTP 500 and websocket errors. The default number of retries is 3.

If you’re a plugin writer, you also have the option to configure the number of retries for your plugin with the new DEFAULT_MAX_RETRIES field that has been added to the Remote base class.

Filter the task list with correlation ID #8891

When using the task list endpoint, you can now filter the list by correlation ID to see all tasks being emitted by a single user action.

Changes to Download Concurrency #8897

Previously, download_concurrency would be set to a default value upon creation. With this release, the default value is set to NULL and returns NULL when queried. However, a default value will still be used. This value comes from the Pulp global settings or can be set by the plugin writer.

Updates to the Plugin API

Here is a summary of changes to the Plugin API in this release:

  • pulpcore.plugin.viewsets.DistributionFilter has been added. Use this instead of
    pulpcore.plugin.viewsets.NewDistributionFilter. #8480

  • user_hidden field has been added to Repository, with which you can hide repositories from users. #8487

  • timestamp_of_interest field has been added to Content and Artifacts. This field can be updated by calling a new method touch() on Artifacts and Content. Plugin writers should call this method whenever they deal with Content or Artifacts. For example, this includes places where Content is uploaded or added to Repository Versions. This will prevent Content and Artifacts from being cleaned up when orphan cleanup becomes a non-blocking task in pulpcore 3.15. #8823

  • AsyncUpdateMixin has been exposed through pulpcore.plugin.viewsets. #8844

  • DEFAULT_MAX_RETRIES field to the Remote base class - plugin writers can override the default number of retries attempted when file downloads failed for each type of remote. The default value is 3. #8881

  • DEFAULT_DOWNLOAD_CONCURRENCY field has been added to the Remote base class - plugin writers can override the number of concurrent downloads for each type of remote. The default value is 10. #8897

Plugin API Removals

Using non-JSON serializable types of args and kwargs to tasks is no longer supported. uuid.UUID objects however will silently be converted to str. #8501

The versions_containing_content method has been removed from the pulpcore.plugin.models.RepositoryVersion object. Instead use RepositoryVersion.objects.with_content(). #8729

The pulpcore.plugin.stages.ContentUnassociation has been removed from the plugin API. #8827

Plugin API deprecations

The pulpcore.plugin.viewsets.NewDistributionFilter is deprecated and will be removed from a future release. Instead use pulpcore.plugin.viewsets.DistributionFilter. #8480

Deprecate the use of the reserved_resources_record__resource in favor of reserved_resources_record__contains. Tentative removal release is pulpcore==3.15. #8501

Plugin writers who create custom downloaders by subclassing HttpDownloader no longer need to wrap the _run() method with a backoff decorator. Consequntly the http_giveup handler for the sake of the backoff decorator is no longer needed and has been deprecated. It is likely to be removed in pulpcore 3.15. #8881