Friday 28 February 2020

NZ Rail Maps Project Development Report [2020D]

So here is our latest update on the overall NZ Rail Maps project. As we made clear earlier this year, we are working on completing all 12 volumes in 2020. That is an aspirational goal and for various reasons it's quite possible that goal won't be met. However it is worthwhile making a serious attempt to achieve it, even if we only get 10 of the 12 volumes actually completed by Christmas.

At the moment the schedule has slipped by about 10-14 days, based on when Volume 5 was finally published, and our anticipated schedule for Volume 6 which is currently in development. Looking at our last project development report published two weeks ago, we anticipated strong progress at that time. The first set of mosaics completed was for Wellington Yard, which was finished around the time of that report, or a couple of days later. These then sat for 10 days whilst the mosaics for the rest of the corridor were pushed ahead. This turned out to take somewhat longer than anticipated, naturally, and then when we finally produced some tiles for Wellington, bugs were found and have had to be fixed.

Because of the scale of the mosaics being somewhat more ambitious than expected, we have taken a bit longer than planned to churn these out, but progress is starting to happen and depending on how many bugs are found and have to be resolved, we anticipate completion of them will push into the middle of next week. We then have to update the map content as quickly as possible. On the other hand, like Volume 5, the actual content that is presented will be much more extensive than what we have published at any level before. For example we now have the full coverage of Hutt Shops and the Seaview industrial area on the Gracefield Branch, plus historical views of every station in the Hutt Valley, including multiple eras of Upper Hutt. We also now have coverage of Summit and Cross Creek when they were open, plus extra material for Maymorn, Kaitoke, Rimutaka Loop, Speedys Crossing , Featherston, Carterton and Masterton - as well as the Woodville junction that has already been completed in Volume 5.

The next question of course is what will follow Volume 6 and this is being eagerly anticipated already. That will be probably Volume 1, North Auckland, followed by Volume 3, ECMT. And whilst Volume 1 is relatively straightforward to complete with many mosaics recently produced, Volume 3 has very little. Because the ECMT is a very extensive corridor with numerous branches, and a few small areas have already been completed, but there would be a lot more needed. So maybe these two volumes together are a month each but Volume 1 will take less than a month and Volume 3, more. Volume 1 does not have any mosaics for Auckland and this is essential so this will add some work.

This leaves us Volume 4 and 2 to complete the North Island and these have been left as long as possible because significant chunks of Horizons Regional Council's territory and Taranaki does not yet have full Retrolens coverage because it has only been added since the start of this year. Volume 2 would be the most affected but we do have extensive coverage completed now of parts of Wellington, and Hamilton, but none of Auckland, and probably for the NIMT we would expect to have some of the major deviated areas covered as well as stations like Taihape and Taumarunui. For Volume 4 we would look to update Whanganui, which is already improving, and add New Plymouth, Stratford, any significant SOL station, and Okahukura.

So the schedule for the next four volumes after this is pretty well thought out and the work is going to keep being pretty intensive up to the middle of the year and probably a bit beyond.

Wairarapa Line [0J]: Volume 6 Progress Update 10

Since our last update we've made a lot of progress on all the mosaics which are in six separate projects. We have extracted tiles so far for Wellington to Ngauranga. Whilst that doesn't sound like a lot, the mosaics for the other areas have practically all been completed except for Featherston-Masterton and the Greytown Branch (all in one project) and those tiles are being extracted over the next couple of days. So the time schedule is going quite well and we are focussing on completing historical tiles rather than drawing maps. The tiles are going into the GIS as soon as they are extracted to be checked for any issues which are then being resolved as needed.

The significant works done in the last few days are particularly in the following:
  • Petone to Waterloo and the Gracefield Branch where we have added 1938 historical content, particularly around Petone-Melling and Seaview. There were some surveys done in 1938 which are at scales varying from 1:6000 to 1:10000 which is quite high for that era - most of what we can get for the period around the country tends to be around 1:16000. So that enables good quality maps for the era.
  • Waterloo to Haywards where larger scale 1938 surveys again were used to get all the way up the western route to Belmont Quarry (the last station before Haywards) and on the current route to Taita. Smaller scale coverage completes what is needed for this set of maps so that both routes are completely covered.
  • Haywards to Upper Hutt where some 1963 coverage was found for the Upper Hutt station.
  • Upper Hutt to Featherston where 1941 coverage of Summit and Cross Creek was added to the existing maps which as we mentioned in a previous post we didn't have coverage of previously while the line was open. With other bits and pieces we now have better coverage of the original route over the Rimutakas quite well, including Maymorn during the deviation construction era when it was the work camp site. 
  • We haven't got around to enquiring about the 1951 survey of the Hutt Valley and Rimutakas that is currently unavailable in Retrolens, but have been able to use some historic maps that we were able to download from another source. Also there is some 1957 coverage of parts of the area such as Maymorn which was still very much like the construction era in appearance, and Speedys Crossing near Featherston. 

Maymorn and Speedys Crossing were the two places where connections between old and new routes were maintained for construction access during the deviation contract and remained as such for several years afterwards until the original line was completely dismantled. Having two connections was necessary because the centre rail of the Rimutaka Incline and the old small section tunnels imposed loading gauge restrictions for modern rolling stock. Most of the actual construction supplies and equipment needing to be sent by rail during deviation construction was sent from Wellington through the connection at Maymorn. After the deviation was opened, with the much greater ease of access to Featherston, the connection at Speedys Crossing was used more. Both connections are shown on 1955 and 1957 S&I diagrams, the one at Speedys Crossing came off the main line with its own switchlock and at the time the diagram

So these areas are practically complete and extraction is being prioritised along with the completion of the Featherston-Masterton / Greytown project and resolving any issues that have been discovered when tiles are extracted from the mosaic projects.

The base aerial coverage for Featherston to Masterton is only available in 0.3 metre resolution and has been scaled to 0.15 m in the mosaic projects to be a better match for the high resolution NZR station surveys of the three stations concerned. As usual, Qgis will be instructed to draw the extracted tiles at 0.15  ensuring that we are able to make use of the full resolution of the historical tiles.

At the time of writing we have spent a couple of days fixing issues with historical content for Wellington Yard. This is some indication of the type of problems that we often don't discover until we are looking at the tiles in the GIS. However that same period of time has also been spent on extracting tiles from the other projects all the way up to Waterloo, so actual presentation of completed tiles to the GIS is more advanced than just at the Wellington end of the line. It is, however, quite possible that the tiles may not be fully completed for another three or four days, depending on how many other issues are found, but we hope in reality that we will have all of them in use ready for the start of next week. However there is so much extraction work, checking, and finishing the one remaining project that has not yet been completed to do, that it is quite likely we may not have actual maps starting to come out for another 4-5 days and that it could well be the case that the publication deadline for Volume 6 ends up being somewhere around the 10th of March, or exactly a month after Volume 2, and leaving us in actuality unable to make up any time at all on our overall schedule.

Tuesday 25 February 2020

Wairarapa Line [0I]: Volume 6 Progress Update 9

In the last few days we've been working on historical maps for Upper Hutt to Featherston. This takes in the old railway over the Remutakas that was bypassed by the Rimutaka Deviation with the big tunnel in 1955.

Although this seems to suggest a great rate of progress, we in actuality still have a fair amount to do with the projects covering previous sections of the line, there still being work for example in Melling-Haywards for the project covering the present main line from Waterloo to Haywards. We have actually only got historical tiles for three stations so far, all of them north of Wallaceville, for the entire Wairarapa Line. So there is still a great deal of work extracting the tiles from the completed projects, and in some cases, some historical work still to be done.

For Upper Hutt - Featherston, we have managed to come up with more aerial photos that are of greater benefit. For Upper Hutt itself, we have 1943, 1963, 1970 and 1980 historical coverage now, in addition to the current. We also have 1963 for Kaitoke, which whilst post closure, will help in the placement of the buildings, which were preserved for some years complete (they have all gone in more recent times), 

The biggest discovery has been 1943 aerial photos of Summit and Cross Creek, which are the two stations at the top and bottom of the Rimutaka Incline, respectively. These will enable more detailed mapping of the yards, although site plans are really needed to confirm impressions on the ground as the old aerial photos are not really sharp enough to pick out the details accurately. Still, it will be a good addition to the maps.

So at this stage the schedule looks much as predicted; there will be one more mosaic project that will take in Featherston to Masterton and the Greytown Branch. We naturally hope to get some better historical aerials of that line but have not looked into it just yet. That plus finishing the existing projects will take until the end of the week, and then it will be an all in intensive effort to try to get Volume 6 completed the following week. We will have to see how that pans out.

Summit 1943
 Cross Creek 1943/current.

Saturday 22 February 2020

Wairarapa Line [0H]: Volume 6 Progress Update 8

We have now completed the 1969/1970 mosaics for the entire line from Wellington to Upper Hutt. We still have to take a look at the 1930s/1940s mosaics for the old western line from Melling to Haywards which will be our next task.

Also we will be starting work shortly on the section from Upper Hutt to Featherston. This section incorporates both the original route over the Rimutakas and the current route via the Rimutaka Tunnel.

The expectation continues to be that it will take well into next week to complete both of these as well as the three major stations north of Upper Hutt. However some mapping can start before then provided we have extracted historical tiles from the completed Gimp projects. Right now we are doing this parallel to the mosaic work and the first station we are actually working on is Upper Hutt itself.


Upper Hutt is where the two routes diverged, with the old route climbing above the present on gradients of 1 in 35, and then crossing over it on a temporary bridge that was built during deviation works. Bridge 35 over Park St was not built until some time after 1970, consequently there was a level crossing at King St at this time.

Thursday 20 February 2020

Wairarapa Line [0G]: Volume 6 Progress Update 7

Another day, another update report for Volume 6. Work has now moved onto mosaics for the sections from Waterloo to Haywards, which we hope will include some low quality coverage of the western route (the original railway from Petone to Haywards, which closed in 1954) as well as the current main line.

We are also working on the mosaics for Haywards to Upper Hutt. Both of these sections will incorporate the work already done around 2 years ago when we mapped a lot of the route with the oldest aerial photos available from the 1930s/40s. The main surveys that are going to be of most benefit to the maps are the 1969 and 1970 NZR surveys which, unlike other regions, cover all of the Wellington suburban area at around 1:3500. In most parts of NZ, corridor surveys are at 1:5500 and station surveys will be done at 1:4300. As has been carried out with the Johnsonville line to date, the historical coverage will mainly focus on stations and sidings, but as stations are so close together, we will probably get full historical aerial coverage of the entire corridor in any case.

Given progress to date we expect it will take the rest of this week to complete these two areas and then move onto amalgamating the existing Upper Hutt-Featherston Gimp projects into one project, which of course is going to include the Rimutaka Incline. This will simply copy the previous projects that have been created for this section. We then will move on to cover Featherston, Carterton and Masterton in one project and that will complete historical aerial photo coverage for Volume 6.

The schedule therefore is that we expect actual mapping and production will not commence until late next week at the earliest and take at least a week so the timetable for this volume will push into March, meaning the whole volume does need about a month to be completed as originally planned, but of course we are running more than a week behind on the overall schedule so it's doubtful we are catching up on that at this stage.

Wednesday 19 February 2020

Wairarapa Line [0F]: Volume 6 Progress Update 6

Since our last update we have completed the historical maps for Petone-Lower Hutt, Petone-Waterloo and Woburn-Seaview (Gracefield Branch) using in all cases what areas we had previously covered with the earliest available aerial photography from the 1930s, and adding in the big NZR survey of all these sections from 1969. In fact there are NZR surveys at larger than usual scales for every line in the Wellington district from around this time, as well as some later ones. In this case, due to lack of time, we are not checking to see if there are any other surveys either earlier or later than 1969 that would add more detail to the maps, especially the industrial sidings at Seaview which have changed numerous times over the years. We will get 1969 coverage of Hutt Shops to add to what we have already completed in the past for this specific site as well.

This essentially completes the historical map coverage for these sections of line. Whilst the old main line on the western side of the Hutt River from Melling to Haywards is something we'd like to be able to cover, there are only a few surveys available prior to the closure date of 1954, and the 1930s/1940s ones are of too poor a quality to be worth the extra effort of trying to create accurate mosaics given the time interval and significant physical change in the landscape of the area, where road improvements have obliterated a lot of traces of the railway. There is one survey from 1951, No. 570, that is listed on the Retrolens site but for some obscure reason the actual scan prints are unavailable, so we are not confident of being able to see anything from that survey in time for it to be included in this release of Volume 6, although we will not rule out the possibility it could be added in future, but only if the scale and quality are good enough. 

Work is now moving onto the next section of the Greater Wellington suburban section, namely Waterloo to Haywards, which is within Lower Hutt City boundaries, and will take in the junction of the old western line which has been already covered in the previous version of these Gimp projects. These also have 1930s coverage in them already which was basically done in mid 2018 to provide an interesting comparison because the Hutt Valley line at that point in time actually ended at Waterloo so further north of that point hadn't even been built in the late 1930s. To recap the dates, Petone-Waterloo was opened in 1927, then there was a big gap until 1946 when Waterloo-Naenae opened, Naenae-Taita in 1947, Taita-Manor Park (old name Haywards) opened 1954 and the old Haywards-Silverstream route was deviated in 1955. So there was actually a lot of construction on this route that is covered by the available aerial photos.

As we said yesterday this will be followed by Silverstream-Upper Hutt which we expect will be along the same lines as Lower Hutt except we haven't got any existing maps for most of this route as it all dates from the 1870s and has changed little since then. The main points of interest, again using large scale NZR surveys from 1969, are likely to be found at Trentham with the army camp and industrial sidings in the area, and at Upper Hutt which used to be a freight yard as well as passenger and had some sidings as well, and we will also have a look to see if we can find any better coverage of the original route at Silverstream (which has been taken over by the Silver Stream Railway). Again we will not have time to look at any other surveys. It is going to be pretty hard to keep to the proposed timetable as it is.

(UPDATE: We have had a look at what we already have for the western line, and it is possible we might be able to include it in the maps, because it might be good enough to use just for the basic route, but not for stations. So we will take a look at whether it is usable in any form but this will be a very perfunctory look and it may well still be the case that there are no historical maps produced on the western side of the Hutt River and that we only use this coverage to inform a rough rendering of the diagrams for that side)

Tuesday 18 February 2020

Wairarapa Line [0E]: Volume 6 Progress Update 5

Since our last update we've spent a lot of time creating and updating mosaics for the historical part of the maps. We completed a set for Wellington to Ngauranga, and whilst Wellington more properly fits into Volume 2, Ngauranga is quite interesting due to the amount of changes that have occurred due to the motorway construction and industries changing. We used a number of different generations of aerial imagery for Wellington because it is a major yard, but in actuality the Wairarapa Line is said to commence 1.8 km from the zero peg at Wellington Station.

We are now working up into the Hutt Valley where the focus will be on combining a number of smaller Gimp projects into larger ones. The smaller ones seem to reflect on the computing resources we had a couple of years ago, so for example we expanded one that covered Woburn to Waterloo, into a much larger project including Petone, Lower Hutt, the entire Gracefield Branch and the first part of the former Hutt Park line, which was used for industrial sidings. 

We expect probably only two more Gimp projects will be needed to Upper Hutt (one from Waterloo to Silverstream and then another from Silverstream to Upper Hutt seems likely) and the current several from Upper Hutt to Featherston probably can be combined into a single project. After that one more project for Carterton and Masterton will see out Volume 6, and that will have to be all that is going to be completed for this volume, because timeframes are important. That will leave a week to put all the info into the maps and spit them out and hopefully we can get back on track with the timetable but it may not be possible to make up the lost time in reality and it may be the first or second week in March before this volume is actually complete.

Friday 14 February 2020

NZ Rail Maps Project Development Report [2020C]

So a few days ago we completed Volume 5 of NZ Rail Maps. This is not the first time we have completed a volume in any format, but it is the first volume completed in the new online format. As previously noted, the maps in this photographic format are specifically designed to be used on handheld devices when actually travelling in the field, such as when riding on a train or travelling on adjacent roads.

Having had a few days to reflect, it is a big achievement. Volume 5 was issued in PDF format previously for the Napier-Gisborne section only, so it isn't so much a major achievement for Volume 5; it is much more of a major achievement for the whole project, being 1/12 of a methodical and planned megaproject of completing all of the maps in one year. 

When we planned this out, we didn't know how quickly we could do it, and therefore whether the maps would all be at basic level, with not very much of the historical aerial photos. There does seem to be, however, enough time to add some of the mosaic tiles for useful areas into the maps (at least for the stations that have official surveys) so we are adding as much of these as we can within time limits. In any case, it seems likely one or two volumes may take longer than planned, especially for example Volume 2 covering the NIMT, so a modest time overrun, say 2 months of next year, might be necessary. We will still be working to keep to time as much as possible with the rest of this year though.

Currently working is beginning on Volume 6, Wairarapa Line, with aerial mosaics of the Wellington end of the line currently being put together. This includes some work for Volume 2, NIMT. The first set of mosaics will cover Wellington yard and the Johnsonville Line for the NIMT, as well as Kaiwharawhara for both volumes, and Ngauranga for the WL. We had already done 1938 images of Wellington and 1944 of Johnsonville-Tawa a year or two back and to these we will add a 1941 view of Johnsonville station, 1971 views of the other stations on the Johnsonville Line, and stuff for Ngauranga from the 1960s and 1970s. Of Wellington itself, in addition to 1938, we will have an era for 1960, then 1988, and then 2000, the latter at quite a small scale but in colour as is 1988. There are a great many scans covering the Wellington railway yards at large scale in many cases (even for the highway surveys 1:3000 is not uncommon) which has given a lot of choice, and we already spent part of a day adding 1971 coverage of the yards and then dumped it the next day for the 1960 photos which were a motorway survey.

Once these are complete then the focus will be exclusively on the Wairarapa Line and pushing in additional coverage for all of the yards previously completed, although probably mostly just the official NZR surveys rather than the multiple generations and sources used for Wellington station. We anticipate starting the actual map drawing for the Wairarapa Line sometime next week and will be hoping the first set of maps produced doesn't need to be revised at all - unlike the Volume 5 situation where so many additionals were added from deciding at a late stage to add lots of extra historical stuff.

Tuesday 11 February 2020

Wairarapa Line [0D]: Volume 6 Progress Update 4

Last time we posted on this subject was a month ago today and it's surprising it has taken that long to get back to Volume 6. Now that Volume 5 has been posted as detailed in our last post, we can push ahead with Volume 6 from next week, but the rest of this week will be spent gathering aerial photos. After considering our options, we will only be updating the main stations, such as Wellington (strictly speaking not part of the WL), Ngauranga, Petone, Upper Hutt and anything in between that has hi res official station surveys and had extra sidings at the time, then probably Featherston, Carterton and Masterton as available and anything else that has NZR station surveys. ALthough not checked yet, we hope this includes good quality scans of the Gracefield Branch freight yards and Seaview sidings. The last time we looked at any Wellington Region stuff, as with other areas we have been working on more recently, the station surveys had not been uploaded, and once again it is pleasing to have discovered them.

However due to lack of time we won't be doing any other stations at the present unless official station surveys are found, so the list of stations in our previous post is probably not going to be made use of. Even though the corridor is much less involved than Volume 5, with fewer branches and a much shorter overall length, updating all these stations is going to be fairly involved. So a lot of work will be needed and we have to try to get our original schedule back on track as well, being rather late in our release date for Volume 5.

We may update Wellington if there is time as a side contribution to Volume 2 but that volume will be one of the last completed in the North Island (NIMT) due to being so long and the central part of it having the least complete collection online so far, due to Horizons region only having been added to Retrolens at the start of this year. Amongst all the coverage for Wellington rail yards, so far yielding complete sets for 1969 and 1974, is what appears to be a full set of aerials the entire length of the Johnsonville Branch from 1971 which includes the terminus which was quite different then from what it is today. That will be very interesting as so far we have not turned up a good quality aerial photo of that station before now. However we are keen to locate some from an earlier era if possible especially when Johnsonville was more of a freight terminal, which was certainly the case when it was part of the main line, although that is too far back. It was a stock handling station until Raroa became the primary stockyards on the line.

Palmerston North Gisborne Line [0Z]: Volume 5 Progress Update 26

So we delayed this until release time. We are pleased to announce that Volume 5 has released. As the legendary Steve Jobs once said, "Real Artists Ship". Being able to release a fully completed volume is a big milestone for this project in its current form. Although we have previously produced some printed volumes of maps, they were neither as comprehensive nor as detailed as this new release, which is the first major release that incorporates both historical and current aerial photography based maps as well as diagram maps together in a format that is specifically designed for use on handheld devices. It is also, naturally, a big deal after spending many years working on this project, to actually embark on the first step of a journey that should see every single map, of which there will likely be an estimated 10,000 in total, released in 2020. 

(To see the maps, scroll right to the bottom of this post for the link)

"Real Artists Ship" is a concept that addresses that a great many people start projects like this but never complete them. Our experience in the historical community in general is that many people have begun projects of historical research that have not resulted in a finished publication. This is likely reflective of the fact that such projects are extremely time consuming and for example, a publication may require many years of intensive research to complete for an output of perhaps a hundred pages or two. This knowledge has in turn informed the development of this project and the determination of the amount of external research that can or should be undertaken. This has been extensively streamlined with the assistance of Archives New Zealand to enable us to access a large number of historical NZ Railways files in a short time frame. Apart from that external resource access, we are proud of the fact that 99% of the remaining work of the project has been carried out from home. Releasing the finished project for public access is a great personal highlight and our timeframe for completion in 2020 reflects on the fact that we expect our personal resources will be shifted elsewhere after that time as we have been hard at work on NZ Rail Maps for the past 12 years exactly (one year for each volume on average) and it has to be completed sometime to enable us to move on to other challenges in life. 

"Real Artists Ship" is also an appropriate concept for the fact that these maps are, in fact, artistic creations and reflect our own creative interests and experiences, The published map format and the contents of each individual map are carefully handcrafted with all the detail designed using the map symbology which is listed in the key for each map.  Although the Qgis layout composer outputs the information stored in the GIS database more or less in a standardised way, each map still has to be checked to ensure it is legible and contains all of the useful information that people are likely to want to make use of with the maps. This is one of the reasons why during production we changed the map formats several times in order to present the maps in a way that makes it easier for us to speed up the production of them considerably due to the many challenges in integrating information from other sources such as the Retrolens historical aerial photos and the Linz Data Service downloadable layers. We expect to be able to speed up the production substantially in the other 11 volumes of the maps. The maps themselves are not as information dense as some other publications such as the Quail Atlas, which incorporate extensive hand customisation of the content of every single page. We could not produce such a large volume of maps if every single page had to be hand edited to change the positions of items or add specific notes or tables of information for just that page. The key reason there is such a large volume of maps overall is a design decision taken that in order to give access to as much information about the history of the New Zealand railway network as is reasonably practical, the maps will not have a scale smaller than 1:10000 in the online format and that many of the maps of individual railway yards and where necessary elsewhere in a particular corridor will be published at scales as large as 1:1000. Being able to publish the maps free of charge via the Google Photos cloud storage system is a big factor as well in being able to put a large number of maps into each volume.

To emphasise how valuable the new maps format is, we have previously commented that it is not financially possible to create a mapping website that provides the same kind of functionality that people can use in Google Maps or even Open Street Maps these days, especially on phones and tablets. And whilst there is something in existence called Open Historical Maps, and we have considered publishing to it as a platform, it still does not offer all of the functionality that is being made available in the format we are publishing in.

So if we cannot publish a full web based map service comparable with other web based map services, the next best thing we can do is to produce static maps in a form that makes it easy to navigate quickly between maps as one is actually travelling along a particular route, whether on a train or by road nearby. This became a desirable capability arising out of a journey by road / sea from Christchurch to Gisborne around 18 months ago and it is appropriate therefore that the map volume for the main line to Gisborne and its various branches is the first to be completed. 

Whilst there are obvious limitations in the format we have implemented, which uses Google Photos to provide the map storage and the user interface, we expect end users to make use of other online maps in conjunction with our map volumes which have in them sufficient geographical information to enable a particular location depicted on a map to be easily located. Whilst it is obvious zooming in and out is not possible as in the Google Maps website, this in fact requires massive resources to produce and implement such as Google's own enormous collection of aerial photography taken by their own satellites and running in their massive world wide data center cloud so on that basis it is easy to understand why it is difficult for us to duplicate that kind of functionality. Whilst Open Street Maps is able to provide zoomable maps, they are only diagrammatic without any aerial photography which uses the most physical resources and costs the most to implement.

It is enough of an achievement for us to be able to generate these maps on purely a non commercial basis with very little money actually having been spent, mostly being a few hundred dollars on some of the early aerial photo scans (prior to the Retrolens site being established) and the web hosting for the nzrailmaps.nz domain name, which redirects to our free Wordpress site. Obviously there are day to day costs such as travel and accommodation in Dunedin to research the Otago Central Line and travel to Archives New Zealand's Christchurch office to research various railway lines in Canterbury. This project definitely could not have happened in its current form without either the free and open source Qgis software, or the freely available resources from or via Linz (the downloable data layers and contemporary aerial photography from the Linz Data Service website and the historical Retrolens aerial photography, which includes many surveys that were specifically undertaken for New Zealand Railways that are of a very high resolution). Another incidental expense for us has been around $2000 spent on our computer hardware over the last few years to have the processing capability and storage needed to assemble all the map resources. Free internet from our local community has also been very material. Of equal significance but perhaps less map specific has been the availability of the Linux operating system as Windows would have driven us crazy by now and it would have cost us unnecessary hundreds of additional dollars in license costs. Regardless of whatever personal prejudice or preference you may have over operating systems, it is a fact that Linux is used on many of the world's most powerful supercomputers and also on many lightweight low cost computers and its reputation as being highly capable for a wide range of different tasks is well earned. And need we mention it is also free and open source software? Because it is free we have not had to spend a single cent on any software for the project.

Volume 5 contains just over 1000 individual maps in six individual sets (photo albums). Each of these maps was individually created and checked visually from the GIS software and there have been numerous revisions and updates to produce the finished product. This can be appreciated by looking at the range of production dates and times that appear on the bottom left edge of each map, which in general show a range of about one month, from early January up to today. We are aware of a small number of issues with the maps that are now available and will be correcting these, but as far as we know the maps in each album are both complete and accurate. We have taken the opportunity to assemble map sets that contain both aerial and diagram maps in the correct sequence in the same set which required custom Python scripting to be developed alongside the production of the maps themselves. The user interface of Google Photos has its challenges at times, but it is fully capable of uploading the maps and assembling them into the correct order with the use of our scripting capability even when the maps are uploaded out of sequence and in a number of partial uploads. Thus it has been relatively easy to deal with instances of some individual maps failing to upload and having to be re-uploaded out of sequence. We now, however, need to develop a script to enable an individual map to be replaced, which involves copying the file modification time from the original version on the computer to its replacement and then uploading the replacement and deleting the original. This will be undertaken and tested over the next few days as the very first map in the PNGL Main Line set has an incorrect title and needs to be replaced. We also need to come up with a way to make the map key have an earlier file modification time than anything else in our Google Photos site so that the key will always appear at the top of every album set, even though it is shared across every album from one single source.

As we have previously stated, we are now ourselves going to take a few days' break to recuperate from what has been an intense few days of very hard work to reach this milestone. We are already planning the work on the next volume but it will not ramp up to a full pace until next week and we have to take a very good look at an appropriate time schedule for it to ensure we stay on track with our expected timeline for publishing the rest of the maps this year.


It is appropriate to make some acknowledgements here and hence below is a partial list. This list is in addition to the individual contributors who are partially acknowledged in the previously released printed map volume, and it is a list for the entire project.
  • God, the Holy Spirit and His Son the Lord Jesus Christ, who guide our overall direction of life and provide ever present and inexhaustible support for everything we do.
  • NZ Rail Heritage Community Online Signalling & Interlocking Diagram Database
  • Archives New Zealand, National Library, and DigitalNZ website.
  • Linz Data Service
  • Local Government Geospatial Alliance (Retrolens Site - hosted by Abley NZ)
  • Chat Club Facebook Community
  • Campaign for Better Transport Online Community
  • NZ Rail Geography Online Community
  • Railway Historical Group, NZ Railfans, New Zealand Locomotives and Locomotives of New Zealand Facebook Communities
  • Watts Publications / Steve Watts
  • NZ Railway and Tramway Atlas / Quail Map Co.
  • NZ Railway and Locomotive Society
  • Public Transport Users Association NZ (particularly Canterbury branch)
  • Canterbury Railway Society
  • Hillview Christian School
  • C3 Church Christchurch
  • Qgis Software Development Community
  • Kiwirail Ltd
  • Rail Heritage Trust

Volume 5 Home Page - NZ Rail Maps Wordpress Site

Saturday 8 February 2020

Palmerston North Gisborne Line [0Y]: Volume 5 Progress Update 25

Welcome to another PNGL Volume 5 progress update. Hopefully the next one will tell you the volume has been published.

The main progress that has been done since last time is we have added aerial photo generations to the Napier maps. So we have historical aerial mosaic tiles for 1951, 1973, 1983 and 1996.

We have also been busy checking the alignments of the main line sections around Napier because these affected the Ahuriri Branch - the main line of this branch was realigned a number of times over the years - as well as the PNGL which in particular was relocated in 1990 between Ahuriri Junction and Westshore to make way for the present motorway.

When we started doing Napier a couple of years ago, at that time, we only covered the station and only for 1973. The recent additions are for 1973 all the way up to the Westshore bridge and part of the Port line as well, 1983 all up the same section of main line as well as Napier Station, and 1951 and 1996 the same. We have not added extra coverage of the Port (Ahuriri) line, and as noted in our last update, we will not be making major changes to the existing Diagrams around Napier. This is largely to save time. Since we changed the Aerial map formats to display less detail, it gives us more leeway to bring in extra generations without worrying too much about how they all line up together. It may well be the case in future maps that this allows us to create this historical content more rapidly than in the past and this would be a great benefit to the development of maps in general.

So anyway we are working hard to complete at the moment and will push on, still hope to complete as much as possible of the maps in what is left of today and if not well then top priority to finish tomorrow or at the latest Monday but we think it will happen by the end of the weekend.

Friday 7 February 2020

Palmerston North Gisborne Line [0X]: Volume 5 Progress Update 24

Welcome to the Volume 5 Progress Update 24. We are moving on steadily with our plan to publish the full volume online not later than tomorrow, Saturday 8 February, 2019 and things are working to schedule.

The last couple of days have been spent expanding the historical aerial coverage of Napier to what can be reasonably achieved in the time available. This screenshot shows a part of we now have to look at which we are working on updating our maps from.

The view of Napier in the GIS.

We discovered that we can get a full NZR "station" survey set of Napier in 1973, including all the way up into Ahuriri and even the south end of the Westshore bridge, except for the Port area which has a full NZR "station" survey set from 1970. However you'll see there is no coverage of the wharves in the top right corner. The reason for this is it was becoming too difficult to overlay the historical aerial photos onto the current layout of Napier Port because in 50 years it has changed so much. As the main historical features are the tracks on the wharves, which can actually be traced anyway for the most part off current aerial photography because they are still there, we felt it was not worth the extra work to produce the historical mosaics for the port area.

This also means where the 1973 imagery joins to the 1970 imagery, only one actual 1970 image has had to be used so the changes at that junction will hopefully not be too much of an issue. There is one other place where an aerial image from a different era is used and that is the coverage of the Robert Holt siding (the premises are in Angus Place and the siding is distinctive because it includes a bridge). Here we were able to locate an image from the NZR corridor survey from 1983 that covers the whole siding. This siding used part of the original main line route between the Ahuriri junction and the Westshore bridge after the main line was diverted in 1990 to make way for a future motorway. This section of the track is actually still in place. The rest of the siding is also still in place up to the first bridge except for where the motorway has been built through, which must have seen it lifted (it actually did cross the road before it was turned into a motorway).

We do not have time to show any other era of Napier (for example, 1983 from the corridor survey for the main line only is a possibility, as are earlier eras from non-NZR coverage), so it will be represented as just 1973 in the maps although as mentioned, it is mostly 1973 with one small part of the yard at the Port in 1970 and one siding in 1983.

So having this coverage now in the GIS, we will see how much time we have to add a few extra tracks etc to the diagram, but it has to be done pretty quickly as not much time is available before deadline.

We have also confirmed the use of aerial photos for the other areas that they were already drawn up for. These are Terrace End (Palmerston North), Woodville, Dannevirke (including Tapuata),  Marakeke and Waipukurau.  Again we will check what is shown on the diagrams to see if we have enough time to add any extra features that are not there already.

The big plus is that having already generated a full set of maps for the corridor two weeks ago, we just have to add in a few extras here and there to the existing set, so actually completing is a much smaller task than it would be if we had waited until finishing all these changes and then pushing out the full set of maps from scratch. At the moment there are 279 diagrams and 408 aerials. The base set of maps are the diagrams with one diagram for each physical aerial covered that combines all historical eras into one. The aerials file names all use the sequence number for the same area that the diagram with the same sequence number uses (from 001 to 279 in this instance) but with suffixes which are generally the year that the aerial covers, so for example you have D023 covering part of Woodville and then you have A023-1946, A023-1962, A023-1972, A023-1983 and A023-2015 covering five historical eras of aerial photography. 

For most of the maps there would be in this example only A023 which would be the current aerial photography for the area, which in practice is a range of dates we have not tracked due to using a global WMTS layer for the whole of NZ that LINZ makes available. In some areas we don't use this global layer, instead we have manually selected a particular WMTS layer for that particular area. Having the use of the WMTS layers has removed from us the requirement to download all the aerial photography for the entire corridor which speeds things up a lot, but we do still have to download areas that we need to create historical mosaics for, in order to lay everything out in Gimp. So for example Napier, we downloaded 3.9 GB of tiles at 0.1m resolution the other day, which when opened up in the Zip file for extraction contained 373 actual tiles, and we needed 60 to cover the area we were working on. In practice, a few were able to be taken out again later or were not needed, so that we can save a little bit of disk space for the Gimp project file because all the mosaics we have created so far for the maps across all of NZ use a huge amount of disk storage, well over 1 TB.

Once there is some time an article about Napier itself and possibly articles for some other areas may well be published - for the last two weeks we have only produced these progress reports.

Wednesday 5 February 2020

Palmerston North Gisborne Line [0W]: Volume 5 Progress Update 23

Yesterday we wrote about how important it was to stay on track and meet a deadline. Today we are tossing up how much flexibility we can have to meet a deadline and still be able to squeeze some more content into the maps before they are published.

Recently the map format was changed so that diagram maps show the most detail and aerial maps show only a subset of detail. Specifically the following is only found on diagram maps: non-rail corridors, roads, sidings, buildings, structures, sites, voids, land data, water and terrain. All maps display distance-based locations, bridges, tunnels, features and main lines. Aerial maps omit all the diagram information except that roads are indicated only by their name.

Because of this we do have the ability to increase the number of aerial maps displayed without checking over all the detail that is in them, since much of that detail will not be displayed on them. Because of the flexibility that offers to add more aerial maps without having to draw in the extra detail that is in them, we can add in more historical aerials without having to update the information in the GIS.

The result is we are juggling to see if we can squeeze in a few more aerial maps into the published map set without updating the diagrams so that we can cover a bit more, for example Napier, and in some of the other areas where we have added some aerials that haven't been fully checked over because of lack of time. 

Regardless however there is still an absolute deadline for publication which is going to be Saturday 8th February, the maps will be going online at that point regardless.

It means you may still see some historical maps of Dannevirke for example but the information in the diagrams in particular may not fully reflect what is in the aerial maps. We will strive to be as complete as possible but some information may be omitted, this is always a possibility and unavoidable at any time. We are prioritising fixing a few issues with historical aerial mosaics and at the same time looking to add a bit more into some of them, which leaves some possibility to add some more of Napier with a little work but without spending the time going over every square inch of them to check all the details in them. That means in practice, we are not marking in the names of the individual sidings in Napier as we have elsewhere (Gisborne for example) but have still made a reasonable effort to include as much historical detail about the area as we can achieve in our limited time frame.

Tuesday 4 February 2020

Palmerston North Gisborne Line [0V]: Volume 5 Progress Update 22

So here is the latest progress report on completing Volume 5, Palmerston North Gisborne Line, of NZ Rail Maps.

When we started to work on this late last year we set a deadline for completion which we thought would be the end of January. As we can see this is the first week of February and the maps are still not ready.

The really important thing to us is to stick to the schedule because we want all 12 volumes released to some level this year and whilst we are only a few days past deadline, we are really keen to get this volume released and move on to the next one.

Today we have been looking at maps of Napier trying to get some detail finished and added to the maps and it may well be asked why there isn't more of Napier mapped than just the main station, given how much time has been spent on Gisborne and a few other smaller stations.

The reality is that Gisborne has a special interest for us, but we realise also there would need to be more mapping done in the future in some of these other areas.

Whilst it would be very desirable to put in more of Napier, or for that matter Hastings, and while there are a lot of NZR surveys available, we have to push on and release this volume.

We have decided as of today that the maps are just going to be finished as they are right now, without any more delays. So no further changes will be made, not even with the extra historical tiles for Dannevirke or the other stations that were completed. There is to be some tidying up in a few areas and that's all. The rest of today will be spent on checking all of the maps that have been produced so far for any tidying up needed, and the existing maps will be refreshed if that is the case, but no new detail is going to be added from here on in, all work will be done with what we already have available in the GIS for production.

Sunday 2 February 2020

Palmerston North Gisborne Line [0U]: Volume 5 Progress Update 21

Today's big achievement has been to solve a software reliability issue which had caused the completion work to grind to nearly a halt and resulted in the work of creating the Ngatapa and Matawai-Wairata map sets to be stalled for a few days. We achieved this by using a virtual machine to install an older software version running on Debian 9. This means the project is now much more back on track, though it is still behind and will be about two weeks late by the time this volume is finally finished. This means possibly the subsequent volumes will have to be cut back in scope to resume the required pace of work, so Volume 6 might not get any additional Comprehensive coverage over that completed in the Wellington area, and Volume 4 over that completed in the Whanganui area. However if the pace of completing those two which are the next volumes in the schedule goes well, additional coverage could be provided for them.

Regardless of the delays and extensions to the Volume 5 production schedule, we still need a week of downtime as soon as Volume 5 is released. We have today completed the 2017 updates to maps between Gisborne and Makaraka using the 0.1 metre urban imagery to replace either 2012 0.125 imagery or 2017 0.3m imagery. This is the maximum extent to which we are updating 2012 0.125m imagery.

The other work completed today has been the new aerial images around Beach Loop (covering 1942, 1962 and 1986) and adding the boulder pit siding at Maraetaha. This completes the planned extent of updates between Westshore and Gisborne. This means we can now move rapidly down to Napier to complete some siding labelling around the Port but no additional aerial images will be added. From here on in, the focus is using the current aerial imagery (current or historic) that we already have and doing final checking for completeness against other data sources and no new historical imagery will be put into the maps.


Boulder Pit Siding near Maraetaha. The dates when it was in operation being unknown at this time.



Service Siding controlled by switchlock 22478 as shown in S&I DIagram No. 1276 of 1968. The number of service sidings in the area which were invariably of a short term nature (there is no trace of this one today, nor the one off switchlock 21845 between Paritu and Beach Loop) suggests a considerable number of them were built as track diversions to deal with washouts and slumps. This is another illustration of how difficult this section of the PNGL has been to keep open.



These maps around the extinct Tunnel 24 show two route changes: T24 itself, and another one made at the site of a current washout just a little south of the tunnel. The map shows as "Closed" the original route the line took in 1942, which at some later time was diverted. It seems highly likely the diversion line that was pushed on a curved route up the hill became the permanent route, allowing for what was probably a slump or washout below to be stabilised to protect the new embankment. As the 2017 imagery shows, there is a washout in this location that has undermined the current track.



Tunnel 24 original route and diversion. The aerial photo at the top was taken in 1942 during construction of the line and shows this tunnel as it was built at the time. It was only in use for around 13 years.



Beach Loop itself. The black and white aerial is from 1986. The track layout consists of a loop with a backshunt at the south end. The station was often in use for stabling work trains due to the intensive needs of maintenance in the area, and was also provided with water vats in the steam days. Beach Loop was originally called No.2 Crossing Loop and was never open for any type of public traffic. The earliest S&I diagram for the area is No.590 for Waikokopu-Gisborne issued 1943, but which we do not presently have access to. It is unclear if local signals of any type were provided at Beach Loop but in 1968 the points at either end off the main line were shown as frame lever points without switchlocks, which was consistent with the way that main line points were labelled on other stations in the area, which generally had an L light on the automatic signals at each end. Switchlocks at that time were limited to isolated sidings between stations (service sidings, boulder pit etc). This seems to have been the norm for stations in the Whareratas area until TWC was introduced and they were all changed to the type of points controls typically seen in such area (TW lever lock).

Saturday 1 February 2020

Palmerston North Gisborne Line [0T]: Volume 5 Progress Update 20

Since the last update we completed maps around Beach Loop. These include: 1942, the original route at T 24; 1962, a service siding around a washout at 225 miles; and 1986, the track layout at BL. Since then, we discovered the more of the S&I diagram No. 1276 which covers Paritu to Muriwai from 1968. The extra detail of interest is a service siding at 216.56 miles (Tikiwhata, or around 350 km, or about 0.5 km north of Wharerata Walkway Station), however aerial photos around that timeframe show no trace of it. There were probably many of these sidings over the years to fix problems such as slumps and washouts.

The features that will be able to be shown, apart from the 225 mile washout, include the Maraetaha boulder pit siding which was less than a mile north of the station. We will not do aerial photo maps of this one but just trace it roughly on the ground.

The major work completed today was to generate mapsets for Matawai-Wairata survey and the Ngatapa Branch. Progress on these was slowed considerably by software issues that have taken all day to overcome and within these sets you will still see an occasional gap in the aerial maps where for some reason the full set of tiles has not completely loaded off Linz's WMTS server.

So we have made progress albeit slow and whilst acknowledging being a bit behind on delivery, we are continuing to press on and still very much looking forward to getting down to Napier ASAP.