Right now the website is a pain to update, and definitely not beginner friendly.

So as they say on Futurama, time to kick it up a notch!

Goals

For Collaborators

  • Continuous Integration (automatically update the published website when the source is updated)
  • Simplify deployment
  • make online editing possible

For Business

  • add banner logo
  • add sections for major services
  • declutter sidebar
  • add a forum
  • add a wiki

Adding a forum and wiki will be covered in another post

Background

Many pelican static blogs are hosted via Github. By some clever usage of git the source and generated site are kept in different branches of the same repository.

However, this is a bit of a subversion (yes, bad pun for the dev geeks): usually branches are for different versions of the same thing. This setup breaks that expectation. The generated output is usually force pushed, overwriting any other changes.

It’s another thing to remember, and makes it much harder to explain to people new to version control.

That’s how this website (was) generated.

Googling for pelican and CI (continuous integration) lead to these great guides:

The first article suggests splitting the pelican site into two git repositories. One for the source code, and one for the generated website.

Face-palm time ;-). Much simpler. Collaborators can just use github‘s web editing in the source repo, on the default master branch. Travis will see the changes, generate the website content, and upload it to the right place in another repo. The updated content will be served up by github. Collaborators can go to this ‘output’ repository and see the generated content- they won’t have to switch branches.

So the publishing process will be:

uml diagram

But there is a big security problem: how to securely authorize travis to access our github repositories?

This is handled automatically for the source repository when you activate travis for a given github repository. As part of the process travis, with your permission, asks github for a ‘token’. This token imbues travis with the authority to add itself to the list of services notified when there are changes to your source repository. It also gives travis permission to access and download the source code of your repository… not really needed in this case, as the repository is public. But that’s how travis can work with private github repositories. You can verify this- go to your github repo’s settings, and look under “Integrations and Services”. You’ll see travis listed. You can remove the token, and revoke travis‘s access.

But we have a second repository: where the generated website is published to!

We can use github to generate an access token for the destination repository. Travis get’s all of it’s configuration and data from the .travis.yml file. So simple: put the token in the .travis.yml file. But that’s publicly viewable- anyone can read that file and hence get access to push anything to your public website!

Thankfully the travis team thought of this. They provide a ruby tool, called travis that will let you add private data as an encrypted blob inside the .travis.yml file. So the data is still stored publicly, but only the travis system can decrypt it and turn it into something useful! The data is stored as environment vairiables that will be available from the build script. Be careful though- even the environment variable name is encrypted.

Process

Split Source and Destination Repos

So the first thing to be done: split the source (markdown files, images, etc) from the generated website.

This is very easy if you don’t need to keep the repository history. Just create a new repository on github and copy over the content.

Configuring Travis

Travis does things right, and keeps you honest.

Every time travis builds your website it creates a brand new, clean, virtual computer/container with just the basic OS and some typical development packages. You then have to add any extra software your build needs, then get the source code and build it.

It is a little slower, but it stops the “builds on my computer” arguments before they can get started.

Travis offers a few different OSs and versions to choose from, including Debian and OS X.

To deploy this website I’m using an Ubuntu based setup.

A couple of articles worth reading about the flavours of Ubuntu available:

The above will give you enough information to set the sudo and dist settings in the .travis.yml file.

Next you want to add the extra packages your application needs.

There are lists of packages available for the Ubuntu build containers on github, and instructions for how to install them.

Now would be a good time to review how to customize the build.

The before_install step can be used to install any other applications your build needs.

The install step is where you install any language libraries your application uses.

In this case I used the before_install to download plantuml, and install calls a script that I created to install all the Python packages required.

Getting the build to work is pretty straight forward: add the next section/stop to .travis.yml, watch travis execute it, fix issues, carry on.

Once it’s building correctly you need to have travis push the generated website files to the publishing repository on github.

To do that you need to generate an access token for github and use the travis command to add an encrypted version of the token to your .travis.yml file.

I found Zonca’s guide to cover this clearly. While you’re there, copy his deploy.sh script.

Everything should now be good to go: you check in a new file and magically the website gets regenerated and published… or it will after you debug everything.

Here’s what my .travis.yml ended up looking like: 

sudo: false
dist: precise
addons:
  apt:
    packages: graphviz optipng libjpeg-progs
branches:
  only:
  - master
language: python
python:
- 3.6
before_install:
- wget "http://sourceforge.net/projects/plantuml/files/plantuml.jar/download" -O plantuml.jar
- export PATH=$PATH:$PWD
- git submodule update --init --recursive
install: "./setup.sh"
script: make html
after_success: "./deploy.sh"
notifications:
  email:
    on_success: always
    on_failure: always
env:
  global:
    secure: ppc2yN2TTfDB5SwPMqHjQsyQFfaDLxcGCE9s7dy7d7RQMIGS/dsqge54pLJjokrg4a2MfBlF+MUcD9UoZZjsSIoRLgf798iPDurBDozxO2lySPIcvEp/Ix7urbwslbBR6KwLh1MWEARFif/bVy00HOFevQX5sZ04Rvdnc8lvCHjB1k1DRuiWD8G+MHmAY2Kr91OuZCRLvfxpAEOBM0Rr3fhdL4At0m6HlLMWUllSqX6rvn3hgtCKQGlwJDWL3+DI+8bSnXyFE+J+SZKFGmfOIhj6GQnw1Znqc8qju6PknYxucKyVGOIrqFQ9SSu/trqWJrSddnCH+Vs+q6npGQzWNThthkGCQ2k2VYpR2HiavJ2GqgzUVRq1mMHApJGeG3nnmKBohDbgBY+BEvepZoC+XSU8kSwl8D8wYZZGtBJUKslCnk7dmCxNnAToJoNzD9a+9lskiBHu8zjkQG4EHf8vXX98bHcHG7N+Kpr64/RyESnTF+snNeiLLGBlrQRds1to7626lbTqQntZ0JsPDPNBuoPAYv+dw4AuxhYu1Wf8azUVVjaZHzaUVwuebG0Z8lnhXht0MSPnTbpZAX7IMImLpj7KxjlKLQkeqiw4ZXtO/MYglBMHo2amCSJLJYn/HWC1iEtGSP3dpKS3RGAEUlqzZHS5vrBXyBLzIC7cJOHj55c=

(See this on GitHub: .travis.yml)

You can see the deployment and other scripts on github.

Final Clean Up

To save future confusion, delete the source branch from the repo that holds the published static files:

git branch -d source
git push origin :source