Background

Silver-Platter makes it easier to publish automated changes to repositories. However, in its default mode, the only option for reviewing changes before publishing them is to run in dry-run mode. This can be quite cumbersome if you have a lot of repositories.

A new “batch” mode now makes it possible to generate a large number of changes against different repositories using a script, review and optionally alter the diffs, and then all publish them (and potentially refresh them later if conflicts appear).

Example running pyupgrade

I’m using the pyupgrade example recipe that comes with silver-platter.

 ---
 name: pyupgrade
 command: 'pyupgrade --exit-zero-even-if-changed $(find -name "test_*.py")'
 mode: propose
 merge-request:
   commit-message: Upgrade Python code to a modern version

And a list of candidate repositories to process in candidates.yaml.

 ---
 - url: https://github.com/jelmer/dulwich
 - url: https://github.com/jelmer/xandikos

With these in place, the updated repositories can be created:

 $ svp batch generate --recipe=pyupgrade.yaml --candidates=candidate.syml pyupgrade

The intermediate results

This will create a directory called pyupgrade, with a clone of each of the repositories.

$ ls pyupgrade
batch.yaml  dulwich  xandikos

$ cd pyupgrade/dulwich
$ git log
commit 931f9ffb26e9143c56f20e0b85e6ddb0a8eee2eb (HEAD -> master)
Author: Jelmer Vernooij <jelmer@jelmer.uk>
Date:   Sat Feb 25 22:28:12 2023 +0000

Run pyupgrade
diff --git a/dulwich/tests/compat/test_client.py b/dulwich/tests/compat/test_client.py
index 02ab6c0a..9b0661ed 100644
--- a/dulwich/tests/compat/test_client.py
+++ b/dulwich/tests/compat/test_client.py
@@ -628,7 +628,7 @@ class HTTPGitServer(http.server.HTTPServer):
         self.server_name = "localhost"

     def get_url(self):
-        return "http://{}:{}/".format(self.server_name, self.server_port)
+        return f"http://{self.server_name}:{self.server_port}/"


 class DulwichHttpClientTest(CompatTestCase, DulwichClientTestBase):
...

There is also a file called batch.yaml that describes the pending changes:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

name: pyupgrade
work:
- url: https://github.com/dulwich/dulwich
  name: dulwich
  description: Upgrade to modern Python statements
  commit-message: Run pyupgrade
  mode: propose
- url: https://github.com/jelmer/xandikos
  name: xandikos
  description: Upgrade to modern Python statements
  commit-message: Run pyupgrade
  mode: propose
recipe: ../pyupgrade.yaml

At this point the changes can be reviewed, and batch.yaml edited as the user sees fit - they can remove entries that don’t appear to be correct, edit the metadata for the merge requests, etc. It’s also possible to make changes to the clones.

Once you’re happy, publish the results:

$ svp batch publish pyupgrade

This will publish all the changes, using the mode and parameters specified in batch.yaml.

batch.yaml is automatically stripped of any entries in work that have fully landed, i.e. where the pull request has been merged or where the changes were pushed to the origin.

To check up on the status of your changes, run svp batch status:

$ svp batch status pyupgrade

To refresh any merge proposals that may have become out of date, simply run publish again:

svp batch publish pyupgrade