Thursday 29 August 2019

Project Diary 2019-08-29

This week work is still continuing on the historical map tiles for Waltham, Linwood, Opawa and Woolston. We have gained some additional resources this week to enable us to speed up the tile production work, so we are also simultaneously working on tile production for Middleton to Hornby section of the Main South Line, and the Bryndwr to Ashley section of the Main North Line. So the production of these tiles should be able to be sped up noticeably, as part of the reason for it being slow is the inherent slowness Gimp has when working with very large mosaic projects. Some of these cover a canvas surface having a working area of 3.5 billion pixels (3,500,000,000) and around 100 layers in total. The Woolston-Waltham project is now completed and tile extraction will be commencing and could be completed tomorrow but will probably have to be deferred for a couple of days due to other commitments, and we have had numerous Gimp crashes, so it has been slowed down a lot from what we expected last week.

The issue with a city like Christchurch or any main centre is you are looking at an entire section of a corridor, rather than a number of individual stations along it. Because there are sidings and stuff all the way along that corridor that you want to document, and that is certainly the case for the Main South Line from Woolston to Islington in particular, and on the Hornby Industrial Line as far as Prebbleton. It is not quite the same on the Main North Line which has historically much less industrial development. So that dictates Gimp projects that cover large areas. We could split these into multiple smaller projects but then there is too much problems with overlapping at the edges of these projects because those edges all have to line up when the tiles are put together in Qgis, and the historical aerial photos boundaries don't neatly end on a tile boundary. So a smaller number of larger projects is definitely the way to go, but we have had a few instances of having to split some of them up when they got too large. For example Addington to Ashley started off as one project, but then Addington and Riccarton got combined into another project with Christchurch and it ended up too large to put the rest of the MNL suburban area into it, so Bryndwr to Ashley is what it has ended up as. Same with Lyttelton to Waltham originally being one project that now has Lyttelton and Heathcote in one file and Woolston to Waltham in another. And so on. 

The extra resources mean the time Gimp wastes on saving large projects, which can actually take several hours to complete, can be used to work on another project at the same time. So in fact today we really were working on three at the same time at one stage because the first one was being saved. After completing the progress on the second one, it too was saved, and while that was happening, we worked on the third. Another time saving measure is to use Cubic interpolation for unified transforms instead of LoHalo. My hunch is that Cubic is a lot faster than LoHalo and makes very little difference to the quality of practically any layer we throw at it.