Releasing a New Version

Privoxy Developer Manual
Prev |  | Next

# 6. Releasing a New Version

When we release versions of Privoxy, our work leaves our cozy secret lab and has to work in the cold RealWorld[tm]. Once it is released, there is no way to call it back, so it is very important that great care is taken to ensure that everything runs fine, and not to introduce problems in the very last minute.

So when releasing a new version, please adhere exactly to the procedure outlined in this chapter.

The following programs are required to follow this process: `ssh`, `gmake` (GNU's version of make), autoconf, git, a web browser.

## 6.1. Version numbers

First you need to determine which version number the release will have. Privoxy version numbers consist of three numbers, separated by dots, like in X.Y.Z (e.g. 3.0.0), where:

- X, the version major, is rarely ever changed. Majors 1 and 2 were Junkbuster, and 3 is the first stable Privoxy release.

- Y, the version minor, is increased for every release except for pure bug fix releases in which case only Z, the point or sub version, is increased.

- The point version is reset to zero when the minor changes.

## 6.2. Before the Release

The following must be done by one of the developers prior to each new release.

- Make sure that everybody who has worked on the code in the last couple of days has had a chance to yell "no!" in case they have pending changes/fixes in their pipelines. Announce the freeze so that nobody will interfere with last minute changes.

- Update the code status (CODE_STATUS="xxx") in `configure.in` to one of "alpha", "beta" or "stable".

- Rebuild configure and GNUMakefile to make sure the updated values are being used.

        $ autoheader && autoconf     # rebuild configure
        $ ./configure                # rebuild GNUmakefile

- make dok-release to update the sgml documentation source files.

- If action file processing has changed and is not backward-compatible, make sure the "for-privoxy-version=x.y.z" minimum version number in `default.action.master` has been updated:

        {{settings}}
        #############################################################################
        #MASTER# COMMENT: The minimum Privoxy version:
        for-privoxy-version=3.0.11

- Create the change log:

        $ git tag
        #   to see the tags
        $ git log [last release tag]..master > /tmp/log
        #   get the commit log since the last release
        $ utils/makeChangeLog /tmp/log > /tmp/change.log
        #   reformat the commit log

  Edit `/tmp/change.log` to remove trivial changes and group the changes under general headings like:

        - Bug fixes:
        - Action file improvements:
        - Filter file improvements:
        - General improvements:
        - Documentation improvements:
        - Build system improvements:
        - Code cleanups:
        - Privoxy-Log-Parser:
        - Privoxy-Regression-Test:

  Add the contents of `/tmp/change.log` to the start of `ChangeLog` and re-create `doc/source/changelog.sgml`:

        $ utils/changelog2doc.pl /tmp/change.log >| doc/source/changelog.sgml

- All developers should look at the `ChangeLog` and make sure noteworthy changes are referenced.

- Update the announcement at `doc/webserver/announce.txt`.

- All documentation should be rebuilt:

        $ make man
        $ make dok
        $ make dok-man
        $ make dok-tidy
        $ make config-file

  Finished docs should be then be committed to Git (for those without the ability to build these). Some docs may require rather obscure processing tools. `config`, the man page (and the html version of the man page) fall in this category. README, the man page, AUTHORS, and config should all also be committed to Git for other packagers. The formal docs should be uploaded to the webserver. See the section "Updating the webserver" in this manual for details.

- Commit all files that were changed in the above steps!

- The User Manual is also used for context sensitive help for the CGI editor. This is version sensitive, so that the user will get appropriate help for his/her release. So with each release a fresh version should be uploaded to the webserver (this is in addition to the main User Manual link from the main page since we need to keep manuals for various versions available). The CGI pages will link to something like `https://www.privoxy.org/$(VERSION)/user-manual/`. This needs to be updated for each new release and is done with the "webserver" target.

- Tag all files in Git with the version number with "git tag -s v_X_Y_Z". Don't use vX_Y_Z, ver_X_Y_Z, v_X.Y.Z (won't work) etc.

- Push the tag to the remote with "git push origin v_X_Y_Z".

- On the webserver, copy the user manual to a new top-level directory called `X.Y.Z`. This ensures that help links from the CGI pages, which have the version as a prefix, will go into the right version of the manual. If this is a development branch release, also symlink `X.Y.(Z-1)` to `X.Y.Z` and `X.Y.(Z+1)` to `.` (i.e. dot).

## 6.3. Building and Releasing the Packages

Now the individual packages can be built and released. Note that for GPL reasons the first package to be released is always the source tarball.

For all types of packages, including the source tarball, you must make sure that you build from clean sources by exporting the right version from Git into an empty directory (just press return when asked for a password):

      mkdir dist # delete or choose different name if it already exists
      cd dist
      git clone https://www.privoxy.org/git/privoxy.git
      cd privoxy
      git checkout v_X_Y_Z

Do NOT change a single bit, including, but not limited to version information after export from Git. This is to make sure that all release packages, and with them, all future bug reports, are based on exactly the same code.

Warning
Every significant release of Privoxy has included at least one package that either had incorrect versions of files, missing files, or incidental leftovers from a previous build process that gave unknown numbers of users headaches to try to figure out what was wrong. PLEASE, make sure you are using pristene sources, and are following the prescribed process!

Please find additional instructions for the source tarball and the individual platform dependent binary packages below. And details on the Sourceforge release process below that.

### 6.3.1. Note on Privoxy Packaging

Please keep these general guidelines in mind when putting together your package. These apply to all platforms!

- Privoxy requires write access to: all `*.action` files, all logfiles, and the `trust` file. You will need to determine the best way to do this for your platform.

- Please include up to date documentation. At a bare minimum:

  `LICENSE` (top-level directory)

  `README` (top-level directory)

  `AUTHORS` (top-level directory)

  `man page` (top-level directory, Unix-like platforms only)

  `The User Manual` (doc/webserver/user-manual/)

  `FAQ` (doc/webserver/faq/)

  Also suggested: `Developer Manual` (doc/webserver/developer-manual) and `ChangeLog` (top-level directory). `FAQ` and the manuals are HTML docs.

  The documentation has been designed such that the manuals are linked to each other from parallel directories, and should be packaged that way. `privoxy-index.html` can also be included and can serve as a focal point for docs and other links of interest (and possibly renamed to `index.html`). This should be one level up from the manuals. There is a link also on this page to an HTMLized version of the man page. To avoid 404 for this, it is in Git as `doc/webserver/man-page/privoxy-man-page.html`, and should be included along with the manuals. There is also a css stylesheets that can be included for better presentation: `p_doc.css`. This should be in the same directory with `privoxy-index.html`, (i.e. one level up from the manual directories).

- `user.action` and `user.filter` are designed for local preferences. Make sure these do not get overwritten! `config` should not be overwritten either. This has especially important configuration data in it. `trust` should be left in tact as well.

- Other configuration files (`default.action`, `regression-tests.action` and `default.filter`) should be installed as the new defaults, but all previously installed configuration files should be preserved as backups. This is just good manners :-) These files are likely to change between releases and contain important new features and bug fixes.

- Please check platform specific notes in this doc, if you haven't done "Privoxy" packaging before for other platform specific issues. Conversely, please add any notes that you know are important for your platform (or contact one of the doc maintainers to do this if you can't).

- Packagers should do a "clean" install of their package after building it. So any previous installs should be removed first to ensure the integrity of the newly built package. Then run the package for a while to make sure there are no obvious problems, before uploading.

### 6.3.2. Source Tarball

First, make sure that you have freshly exported the right version into an empty directory. (See "Building and releasing packages" above). Then run from that directory:

      autoheader && autoconf && ./configure

Then do:

      make tarball-dist

### 6.3.3. Windows

Note that the docbook generated files might need some hand editing, so the Windows build makefile does not rebuild the docs.

First, make sure that you have freshly exported the right version into an empty directory. (See "Building and releasing packages" above).

Check that you have the current versions of the NSIS installer, PCRE library, MBED TLS library, Brotli library, and that the MAKENSIS evar in `windows/GNUMakefile` points to the NSIS installer program. (See the Building from Source / Windows section of the User Manual for details.)

Then you can build the package. This is fully automated, and is controlled by `windows/GNUmakefile`. All you need to do is:

      cd windows
      make

Now you can manually rename `privoxy_setup.exe` to `privoxy_setup_X.Y.Z.exe`, and the `build` directory to `privoxy_X.Y.Z`. Create a .zip file of the newly renamed `privoxy_X.Y.Z` directory, GPG sign the installer and zip file,

      gpg --armor --detach --sign privoxy_setup_X.Y.Z.exe
      gpg --armor --detach --sign privoxy_X.Y.Z.zip

and upload the files to SourceForge.

When releasing the package on SourceForge, use the release notes and Change Log from the source tarball package.

### 6.3.4. Debian

Using git-buildpackage we start with a clone of the last Debian version:

      gbp clone https://salsa.debian.org/debian/privoxy.git
      cd privoxy

or if the repository is already there

      cd privoxy
      gbp pull

Now import the newly released upstream tarball via debian/watch file:

      gbp import-orig --uscan

Next update all Debian quilt patches to the new version:

      while quilt push; do quilt refresh; done

If some patch is no longer required (because it is already merged upstream), it can be removed using

      quilt delete XX_patchname.patch
      git rm debian/patches/XX_patchname.patch

If the patch needs modification, you can apply, edit and update it with

      quilt push -f
      quilt edit some_file
      quilt refresh

until

      while quilt push; do quilt refresh; done

succeeds. Then you can

      quilt pop -a

Now add a new entry to the debian/changelog representing the new version:

      dch -v 4.0.0-1

and describe what you did before and don't forget to git commit all changes.

Now you can build the package on the local machine using

      gbp buildpackage -us -uc

You should check for warnings using

      lintian -iI ../build-area/privoxy_4.0.0-1_amd64.changes

Maybe rebuild the package in different defined cowbuilder environments like

      sudo cowbuilder --build --basepath /var/cache/pbuilder/base.cow ../build-area/privoxy_4.0.0-1.dsc

And try to run autopackage testing suite on the result:

      autopkgtest /var/cache/pbuilder/result/privoxy_4.0.0-1_amd64.changes -s -- schroot sid

Or just push the changes to salsa.debian.org, where a CI pipeline is defined for the package, that builds and tests it.

If everything is okay, run cowbuilder with i386 and amd64 environments for current Debian stable release and build privoxy_4.0.0-1_i386.deb and privoxy_4.0.0-1_amd64.deb. Then sign both files:

      gpg --detach-sign --armor privoxy_4.0.0-1_i386.deb
      gpg --detach-sign --armor privoxy_4.0.0-1_amd64.deb

Create a README file containing the recent block from debian/changelog and upload the two packages, the two signatures and the README to a freshly created folder below https://sourceforge.net/projects/ijbswa/files/Debian/

#### 6.3.4.1. Debian GIT Snapshot

For building just a git snapshot build the following workflow may be useful. First create a build environment, for this you may have to run the following commands:

      sudo apt install build-essential devscripts
      sudo apt-get build-dep privoxy

After this enter the checked out privoxy git tree and check that all (new) build dependencies are met:

      dpkg-checkbuilddeps

If something is missing, just add it using

      sudo apt install foobar

Now you may update debian/changelog, especially the version number using

      dch

and finally build the package:

      debuild -us -uc -b

If everything went okay, you may find the resulting Debian package in the parent directory.

You may want to clean up the build tree using

      debian/rules clean

And maybe repair some artefacts using one or both of the following commands:

      git reset --hard
      git clean -fd

### 6.3.5. macOS / OS X

First, make sure that you have freshly exported the right version into an empty directory. (See "Building and releasing packages" above).

The OSXPackageBuilder module can generate OS X installer packages supporting all Macs running OS X 10.4 and above. Obtain it from Git as follows into a folder parallel to the exported privoxy source:

      git clone ssh://git@git.privoxy.org:23/git/OSXPackageBuilder.git

The module contains complete instructions on its usage in the file `OS X Package Builder HOWTO.txt`.

Once the package(s) have been generated, you can then upload them directly to the Files section of the Sourceforge project in the Macintosh (OS X) folder. Each new version release of Privoxy should have a new subfolder created in which to store its files. Please ensure that the folder contains a readme file that makes it clear which package is for which version of OS X.

### 6.3.6. FreeBSD

Update the www/privoxy port and submit a diff upstream. For details see the FreeBSD Porter's Handbook.

## 6.4. Uploading and Releasing Your Package

After the package is ready, it is time to upload it and go through the release steps. The upload is done at SourceForge after logging in.

Now just follow the prompts. Be sure to add any appropriate Release notes. You should see your freshly uploaded packages in "Step 2. Add Files To This Release". Check the appropriate box(es). Remember at each step to hit the "Refresh/Submit" buttons! You should now see your file(s) listed in Step 3. Fill out the forms with the appropriate information for your platform, being sure to hit "Update" for each file. If anyone is monitoring your platform, check the "email" box at the very bottom to notify them of the new package. This should do it!

If you have made errors, or need to make changes, you can go through essentially the same steps, but select `Edit Release`, instead of `Add Release`.

## 6.5. Updating the RSS feed

Once the packages are uploaded to SourceForge they should be mirrored on the Privoxy websites (https://www.privoxy.org/ and http://l3tczdiiwoo63iwxty4lhs6p7eaxop5micbn7vbliydgv63x5zrrrfyd.onion/). This is usually done by Fabian who uses a couple of shell functions for this that aren't documented or published yet.

Once the packages are uploaded to the mirror the RSS feed has to be regenerated with a command like:

           fk@t520 ~/git/privoxy $utils/create-package-feed.pl /tank/backups/sourceforge/frs/project/ijbswa/ doc/webserver/feeds/privoxy-releases.xm

The updated RSS feed then has to be uploaded to the SourceForge webserver and mirrored on the Privoxy websites again. This, too, is usually done by Fabian with undocumented and unpublished shell functions.

## 6.6. After the Release

When all (or: most of the) packages have been uploaded and made available, send an email to the announce mailing list, Subject: "Announcing Privoxy X.Y.Z $CODE_STATUS". Be sure to include the download location, the release notes and the Changelog. Also, post an updated News item on the project page Sourceforge, and update the Home page and docs linked from the Home page (see below). Other news sites and release oriented sites, such as Freshmeat, should also be notified.

Then update the source code for the next version to be released:

- Increment the version number and change the code status to "UNRELEASED" in `configure.in`

- Rebuild configure ("autoheader && autoconf") and GNUMakefile ("./configure")

- "make dok-release" to update the sgml documentation source files.

- Commit all your changes.

Prev | Home | Next
Testing Guidelines |  | Update the Webserver