Tuesday, 23 April 2019

MSL Rolleston-Timaru [0A]: Intro 1

With this particular article I am making it clear I am working towards Volume 11 as a major effort to get something that is actually going to be completed as a volume as soon as possible. This will not be a continuous effort throughout the year but it will be a major focus for the maps this year. At the same time I am refocusing my ambitious schedule that was planned for the year and suggesting it might have to be put out over two or three years.

Rolleston all the way through to Timaru is covered by a 0.3 metre aerial layer. At the moment I am downloading all of what is available in that layer for MSL right down to Waikouaiti and then from there all the way down to Dunedin and then Invercargill will soon follow. Mosaic projects have been assembled over the past few days to cover Lyttelton through to Templeton (the limit of 0.075 metre coverage of Christchurch), and from there will cover Rolleston and any other townships that are in 0.125 metre coverage for the Selwyn District, and from there at 0.3 metres heading south.

We will be using all available NZR surveys for the stations heading south and to match this resolution the 0.3 metre images will have to be rescaled to 0.1 metre size and segmented mosaic canvas used. Last time I wrote about using linear and segmented mosaic projects and indicated a clear preference for linear. However this is only possible where the base imagery is no more than 0.125 m pixel size as too much detail is lost scaling down the NZR images to match larger pixel sizes. At that point I have to bring in a tile grid to sit in the background of the canvas to provide grid indexes for segments and we don't want to duplicate multiple copies of this grid across a linear canvas because there are too many additional layers. Segmenting is actually OK for this situation because the stations are quite a distance apart, meaning no overlap problems, and therefore there is no benefit to a linear canvas.

So I am just about to create the first segmented canvas mosaic project in Gimp for Weedons / Weedons AFB / Burnham and maybe a few other stations heading south as well.

As one can imagine, completing a volume for a line as long as the MSL is quite an ambitious undertaking and it will be staged out and I will take breaks to do work on other areas as I always do. Volumes 5 and 12 in particular are other areas I want to make significant progress on this year and as previously noted there is the schedule I talked about earlier this year and I will look at the standard of completion for other volumes. But to get any of these volumes together I have to download all the aerial imagery for the base maps and that is always a big effort and that at the minimum is what is needed to bring the maps together regardless of what level is produced.

MSL also incorporates previous work done with Methven and Springburn lines and this will be brought in as well. There are no NZR surveys for either line so no issue with having to match NZR survey resolution.

Sunday, 21 April 2019

MSL Lyttelton-Rolleston [0B]: Christchurch-Hornby Intro

Following on from recent efforts to speed up mosaic production, due to having discovered the best way to combine multiple stations or a continuous length of corridor into one Gimp project, I have started doing this with the MSL around Christchurch. Yesterday and part of today has been spent combining several smaller projects into a big one covering Christchurch to Templeton Hornby, which is much of the western suburban corridor for Christchurch itself. There will also be one done for Lyttelton to Christchurch and for Addington to Kainga and Kaiapoi to Rangiora, this one might be taken a bit further north as the stations are further apart so we may be able to get more distance into it. If that was the case it would only go to just south of Amberley at the most. 

The focus is really going on ways to speed up the production of the mosaic tiles and this system of combining several areas in a larger project (up to 100 layers) does speed things up and is only really possible since Gimp 2.10 when they implemented support for projects more than 4 GB. At the moment 100 layers is the practical limit of the computer resources I have, but is also a practical file size limit because of the time it takes Gimp to save a file of that size, which can be as much as 30 minutes (for a 30 GB file). Part of the streamlining is also to do away with all the masking off except in very occasional situations, cropping off the boundaries is easier but I also dispensed with composite aerial tiles that blended old and new together as for historical comparisons just old layer vs new layer gives more surrounding detail for interest.

The last time I worked on this corridor was in May of last year and I was then looking at Christchurch Station in particular but had drawn a project that went from there out to Opawa. That was at the very early stages of my learning how to use Gimp and getting stuff done so it was time consuming with a lot of trialling different things. One of the lessons being learned at that time that was not finally solved until later last year was needing to have background imagery that matched the historical stuff in resolution, and as it can be seen earlier this year, that has been for the surveys that NZR had done for their own use, which often have a scale of around 1:4300 on each image and are very detailed indeed, source imagery of pixel size 0.1 metres or less (0.075 is the smallest pixel size currently being produced by LDS and is generally available for the main urban areas) is needed, and if that isn't what is available then it has to be scaled to match. Hence I spent a bit of time earlier in the year coming up with a way to segment scaled up imagery from 0.3 or 0.4 metres to 0.1 metres so that the NZR surveys could be used to their best advantage. In particular, some of the work recently done in Northland has used that process.

But with the recent work being around urban areas and being able to use 0.1 metre or finer pixel size LDS base imagery, I haven't had to muck around with scaling up and segmenting the source imagery and have just been able to use the base stuff as is. So then I did Dunedin-Mosgiel which was a learning curve of doing the largest possible project size, which turns out to be 100 layers, and from there I went on to do Horotiu-Frankton-Claudelands on the NIMT, which was a learning curve of laying out everything in a really large canvas (in this case one with 1000 million pixels) which is a major step up in ease of use from the way I laid out the stations in Dunedin. Christchurch-Templeton is an interesting one (as will be Christchurch-Kainga) where the corridor is closely aligned to a geographic meridian, meaning a long narrow canvas - the Christchurch-Hornby one is currently 192000 by 21600 without all the tiles yet in place, because that doesn't happen very often in NZ rail geography. Originally I was planning to take it out to Templeton but this was going to push the layer count way over 100 so it has had to be cut back to Hornby, about 9-10 km.

Meanwhile I still did get a reasonable amount of drawing of Dunedin done yesterday and this work continues so I am not slowing down that part too much, because it is important to get that through as soon as possible.

It is imperative to get as much of the base imagery for each project in place as quickly as possible as well, because all of the downloads on one of my computers are taking up a lot of disk space and that space is needed to save the mosaics which are using hundreds of GB so getting these corridors finished for base imagery and then also pushing ahead with the historical maps as well on the corridors is much more of an imperative now.

I think after Dunedin-Mosgiel then possibly Dunedin-Port Chalmers really quickly and then Volume 5 for PNGL is going to be the order of things, but I also do hope to get back to the OCR reasonably soon.

Friday, 19 April 2019

North Island Main Trunk [0C]: Intro 3: Hamilton Urban Area 2

I have now completed a mosaic project to cover the area of NIMT and ECMT that is within the bounds of Horotiu, Hamilton (Frankton) and Claudelands. These are the main areas of interest that I will be mapping at some future time (unscheduled at present) with NZ Rail Maps. 

Whilst the mosaic is now complete with just over 100 layers on a canvas of 96000x93600 pixels (8,985,600,000 pixels, or 9 gigapixels), as noted in my previous post, much of the canvas is unoccupied because of the L shaped aspect of the rail corridor section covered.

At present for actual mapping I am still working on Dunedin and I do not know if any work will be done on the NIMT for some time as there are a number of areas being worked on. The idea at the moment has simply been to put the mosaics together quickly to see that it can be done a lot more smoothly and rapidly than the Dunedin urban mosaic project which has taken weeks to complete. This has been greately sped up by being able to use a linear corridor project rather than the segmented model used for Dunedin, one of the principal drawbacks of which is the need to duplicate overlapping layers between segments.

However another factor in Dunedin has been the significant number of layer eras sourced and incorporated into the project. This has not been done with the Hamilton project. Most of the historical imagery is from a single generation for most areas. The main exception is the ECMT section from Hamilton to Claudelands which incorporates three generations: 1953, 1961 and 1975. This is because of the major changes when the railway between Hamilton and Claudelands was undergrounded in the early 1960s and the change upon the urban landscape that was produced. There is also a 1966 image of Frankton Junction which shows the locomotive depot that was gone by the 1970s along with some other features in part of the yard.

In the Te Rapa mosaic project we have a corridor of 10 km roughly in length, which because of its shape uses the canvas inefficiently, yet we also have 100 layers and a file size of 18 GB. The clear implication from this is that there is no inherent resource usage efficiency from a segmented model as opposed to a linear model and therefore the linear model with its clear speed advantages is what will be preferred in future. As soon as the Te Rapa project is completed I intend to start on mosaic projects for Christchurch, which is at a higher priority for actual mapping, and integrate several sets of mosaics each for around 10 km of the urban rail corridors, which are currently implemented in small projects covering a km or two each.

The Te Rapa imagery dates from 1972 and I have since discovered there is some more available from the 1980s. However at the moment I am not intending to add any more generations to Te Rapa. Multiple generations are a feature of Christchurch and Dunedin mosaic projects at present because of a greater personal interest in this level of detail, which is not going to be the case across all mosaic projects, so that is essentially the difference in this case, because I simply do not have the time to delve into all of the historical details at every location at present.

The images below are from an album on the NZ Rail Maps page. Visit that page to see more information about them. 

 Hotoriu freezing works.
 Hotoriu station.

 Te Rapa Air Force Base.

 Main Te Rapa yard with hump to the far right.
 Te Rapa loco depot 1972.

 Frankton yard.
 Frankton Junction. Was never a triangle.
 Frankton passenger station.

 Frankton Loco Depot 1966.

 Hamilton Railway Station

 Commencement of lowering works in Hamilton 1961. Bridge under construction at right.

 Claudelands 1953.
Claudelands 1975.