NZ Rail Maps Project Development Report [2020F]: Website Hosting Options [1]

Good morning. Here is our sixth project development report for 2020.

We spent the last day looking at possible web hosting options for NZ Rail Maps This meaning that there would be a paid site instead of wholly free ones, although some free hosted content is likely to remain. There are a number of possible options ranging from sites specialised for various types of content, through to full web hosting.

At the moment we are going down the specialised track, which is of course the simplest to set up by far, but does have some limitations, naturally. For example if we chose to stick with WordPress.com, we would have the problem that to get a photo gallery plugin to be used in our site would require us to purchase the most expensive WordPress.com hosting plan. Or another example, we are currently on 14 day trial with SmugMug (the company that purchased Flickr). Their hosting is reasonably low in cost, and easy to set up, but would require us to continue hosting the development blog and PDF files on another site such as WordPress.

Going down the full website route, it could be moderately easy to set up a WordPress site but there is a lot of work involved. We have had sites previously hosted with a particular company in NZ before now and their fee of $15 per month is probably quite reasonable for a site with WordPress available, although a gallery plugin could have an extra cost associated with it. We also have had sites previously hosted on TrainWeb, and still maintain an empty site there. It is possible we could continue to have the blog hosted on WordPress (replacing this Blogger blog), some content appearing on TrainWeb (PDFs) and the image based maps appearing on SmugMug. The latter would be the only paid site hosting. We would have subdomain redirection to the three sites in operation, but the SmugMug site would be the only site that would remain in-domain when accessed, the other sites would redirect to the local free site domain when the domain based URL is selected.

It had been hoped for some time that TrainWeb would convert their site to be a WordPress host, which would have allowed us to continue hosting content on the TrainWeb site and done away with the need for a separate WordPress.com site as we have now, but TrainWeb have not made any progress with the WordPress conversion idea.

So for the next 14 days we will be trialling SmugMug and might well continue with it after the trial as a paid option, unless a better option comes up before then.

NZ Rail Maps Project Development Report [2020F]

In our last update we mentioned that we were considering relaxing the development schedule and allowing two years to complete the maps as expected. The key reason for this is that the level we are currently developing maps to will need longer than the current year to achieve, with the Intermediate level that we are working to.

This has naturally brought about the suggestion that we should work to release all 12 volumes to the Basic level as quickly as possible, and then push on with volume by volume completion to the next level.

However at this point we need to redefine the Basic, Intermediate and Comprehensive levels. In summary:

• Basic is a set of diagram maps of the current and known historical rail corridors based on current aerial photography and other current sources.
• Intermediate adds aerial maps, including georeferenced historical aerial photography, to the Basic level
• Comprehensive adds a printable PDF volume to the Basic and Intermediate levels.

It's appropriate to change these because our priorities have changed. We now regard the online formats (diagrams and aerials) as being the highest priority and reaching the greatest number of users. The posts we made earlier this year defining these levels will therefore be republished.

Whilst the PDF volume format of the maps is still important to us, there are two facets of it that have to be considered for resourcing. The first is that the maps that are produceable in PDF form have to be a unique form factor just for those volumes. This means we have to generate a completely different set of diagrams from the ones that are on the web site. The second issue is simply that there is low demand for this format. Hence it is becoming the lowest priority overall and we now consider it to be the premium format that will take a lot longer to appear.

We are still very serious about completing the 12 volumes and bringing the overall project to a close. We anticipate staging that out over the next three years. The stages will be as follows:

1. Completion of all 12 volumes to a Basic level will be undertaken serially after Volume 6 is released. This should only take about 2 months.
2. Serial completion of individual volumes (after Volume 5 and Volume 5) to Intermediate level. This will continue much as it does now, but is now expected to take until the end of 2021.
3. Complete the Comprehensive level, i.e PDF volume, in all 12 volumes over 2022.

 So for now we'll be pushing on with the Volume 6 Intermediate level, then we'll be switching to Basic level of the other 10 volumes. We will then go back to Intermediate level for those 10 volumes, and then Comprehensive level for all 12. This will ensure there is at least some maps available for all 12 volumes in the shortest possible time and then they are being progressively upgraded to the full level of coverage.

Wairarapa Line [0000]: Volume 6 Research Article Series Numbering

The next article will be one of our research articles into the Wairarapa Line. We will do this one as a break from straight progress updates. We have looked back on previous articles and there was a series numbered from 1 to 9. After that there was the second of the Volume 6 Progress Updates almost two years ago. We have renumbered the earlier article series to make a structure so we can put in further articles such as this one into that structure. The numerical series being adopted will be as follows:

1. Main Line as it currently exists
2. Hutt Park Railway
3. Western Hutt Section / Melling Branch
4. Hutt Valley Branch
5. Gracefield Branch
6. Silver Stream Section
7. Rimutaka Incline Section
8. Featherston Camp Siding
9. Greytown Branch

Many of these names are unofficial. We do not know necessarily which official names may have been used at various times, because in several cases they were part of an officially named section and only got to be separate because of a deviation.  When maps are produced they are put into these series as well and there is a separate map set for each of these series.

We are currently working on the "Rimutaka Incline Section" and have just completed as far as we can, maps for the part of this route from Upper Hutt to the second tunnel. This will be detailed more in the article. There may be more than one research article for this section depending on how much content we decide to publish.

NZ Rail Maps Project Development Report [2020E]

Here is our fifth project development report for 2020. These reports give a general overview of the whole maps project for the year 2020.

As has been outlined in previous posts in this series as well as general blog posts, we started this year on the assumption it might be possible to cover the entire set of 12 volumes in 12 months. The first volume (Volume 5) ended up 10 days behind this schedule but we thought it might be possible to catch up a bit and have the second volume completed in only about three weeks. But in fact it will take about 8 weeks to complete the second volume (Volume 6) so not only have we not caught up to the schedule, we are even further behind than we should be.

The reason for Volume 6 being behind schedule is that it covers a big urban area - Wellington - which has a large amount of history associated with it, as well as having had a lot of rail served facilities like the big Wellington yard and various branches, closed sections, depots and sidings. To give reasonable coverage to the suburban area we have had to intensify the coverage of the first 32 km from Wellington to Upper Hutt and as of this date, 6 weeks in, we have only just finished mapping that relatively short distance. We still have, however, another 140 km of route to cover to reach Woodville. It gets easier to do this because north of Upper Hutt, everything thins out a lot. However, we still expect it will take until early or mid April to complete the entire volume, and as that date approaches, we will become increasingly more keen to see this volume completed as quickly as possible.

With the completion of two volumes in close to four months and the need to have a healthy life balance in the allocation of time, we are now looking at a more realistic completion timeframe of up to two years for the 12 volumes. We feel that the NIMT (Volume 2) and some of the South Island volumes will take a bit longer to complete. We will continue to push ahead at a reasonable pace because we still want to have some sort of goal to aim at, otherwise this will never get finished. So the actual deadline is a bit more flexible, but right now we are still looking at about April 10th to complete Volume 6 at the moment.

The slowest part of the process is the historical mosaics which give us a look at the way things used to be in the past, which provide actual maps in the GIS that we can trace stuff off. Various economies and improvements have been effected in map design and production that will follow through into all future maps to make more efficient use of time going forward. The most important change is to accept impossible-to-avoid misalignments between the aerial backgrounds and the vector layers (the layers that show most features such as main lines, sidings, roads etc) by removing the expectation that vectors will always appear in exactly the right place on the aerials.

Volume 6 currently has 96 distinct map views for the 32 km of main line completed so far (in other words, an average of 333 metres of track in each view). There are 96 diagram maps and 354 aerial maps (although some of those are in fact historical maps of Wellington from 1900, rather than aerial photos). So 450 individual maps have been created so far. In Volume 5, there were just over 1000 individual maps created. We could expect probably at least that number in Volume 6 despite the significantly shorter corridor length.

NZ Rail Maps Project Development Report [2020D]

So here is our latest update on the overall NZ Rail Maps project. As we made clear earlier this year, we are working on completing all 12 volumes in 2020. That is an aspirational goal and for various reasons it's quite possible that goal won't be met. However it is worthwhile making a serious attempt to achieve it, even if we only get 10 of the 12 volumes actually completed by Christmas.

At the moment the schedule has slipped by about 10-14 days, based on when Volume 5 was finally published, and our anticipated schedule for Volume 6 which is currently in development. Looking at our last project development report published two weeks ago, we anticipated strong progress at that time. The first set of mosaics completed was for Wellington Yard, which was finished around the time of that report, or a couple of days later. These then sat for 10 days whilst the mosaics for the rest of the corridor were pushed ahead. This turned out to take somewhat longer than anticipated, naturally, and then when we finally produced some tiles for Wellington, bugs were found and have had to be fixed.

Because of the scale of the mosaics being somewhat more ambitious than expected, we have taken a bit longer than planned to churn these out, but progress is starting to happen and depending on how many bugs are found and have to be resolved, we anticipate completion of them will push into the middle of next week. We then have to update the map content as quickly as possible. On the other hand, like Volume 5, the actual content that is presented will be much more extensive than what we have published at any level before. For example we now have the full coverage of Hutt Shops and the Seaview industrial area on the Gracefield Branch, plus historical views of every station in the Hutt Valley, including multiple eras of Upper Hutt. We also now have coverage of Summit and Cross Creek when they were open, plus extra material for Maymorn, Kaitoke, Rimutaka Loop, Speedys Crossing , Featherston, Carterton and Masterton - as well as the Woodville junction that has already been completed in Volume 5.

The next question of course is what will follow Volume 6 and this is being eagerly anticipated already. That will be probably Volume 1, North Auckland, followed by Volume 3, ECMT. And whilst Volume 1 is relatively straightforward to complete with many mosaics recently produced, Volume 3 has very little. Because the ECMT is a very extensive corridor with numerous branches, and a few small areas have already been completed, but there would be a lot more needed. So maybe these two volumes together are a month each but Volume 1 will take less than a month and Volume 3, more. Volume 1 does not have any mosaics for Auckland and this is essential so this will add some work.

This leaves us Volume 4 and 2 to complete the North Island and these have been left as long as possible because significant chunks of Horizons Regional Council's territory and Taranaki does not yet have full Retrolens coverage because it has only been added since the start of this year. Volume 2 would be the most affected but we do have extensive coverage completed now of parts of Wellington, and Hamilton, but none of Auckland, and probably for the NIMT we would expect to have some of the major deviated areas covered as well as stations like Taihape and Taumarunui. For Volume 4 we would look to update Whanganui, which is already improving, and add New Plymouth, Stratford, any significant SOL station, and Okahukura.

So the schedule for the next four volumes after this is pretty well thought out and the work is going to keep being pretty intensive up to the middle of the year and probably a bit beyond.

NZ Rail Maps Project Development Report [2020C]

So a few days ago we completed Volume 5 of NZ Rail Maps. This is not the first time we have completed a volume in any format, but it is the first volume completed in the new online format. As previously noted, the maps in this photographic format are specifically designed to be used on handheld devices when actually travelling in the field, such as when riding on a train or travelling on adjacent roads.

Having had a few days to reflect, it is a big achievement. Volume 5 was issued in PDF format previously for the Napier-Gisborne section only, so it isn't so much a major achievement for Volume 5; it is much more of a major achievement for the whole project, being 1/12 of a methodical and planned megaproject of completing all of the maps in one year. 

When we planned this out, we didn't know how quickly we could do it, and therefore whether the maps would all be at basic level, with not very much of the historical aerial photos. There does seem to be, however, enough time to add some of the mosaic tiles for useful areas into the maps (at least for the stations that have official surveys) so we are adding as much of these as we can within time limits. In any case, it seems likely one or two volumes may take longer than planned, especially for example Volume 2 covering the NIMT, so a modest time overrun, say 2 months of next year, might be necessary. We will still be working to keep to time as much as possible with the rest of this year though.

Currently working is beginning on Volume 6, Wairarapa Line, with aerial mosaics of the Wellington end of the line currently being put together. This includes some work for Volume 2, NIMT. The first set of mosaics will cover Wellington yard and the Johnsonville Line for the NIMT, as well as Kaiwharawhara for both volumes, and Ngauranga for the WL. We had already done 1938 images of Wellington and 1944 of Johnsonville-Tawa a year or two back and to these we will add a 1941 view of Johnsonville station, 1971 views of the other stations on the Johnsonville Line, and stuff for Ngauranga from the 1960s and 1970s. Of Wellington itself, in addition to 1938, we will have an era for 1960, then 1988, and then 2000, the latter at quite a small scale but in colour as is 1988. There are a great many scans covering the Wellington railway yards at large scale in many cases (even for the highway surveys 1:3000 is not uncommon) which has given a lot of choice, and we already spent part of a day adding 1971 coverage of the yards and then dumped it the next day for the 1960 photos which were a motorway survey.

Once these are complete then the focus will be exclusively on the Wairarapa Line and pushing in additional coverage for all of the yards previously completed, although probably mostly just the official NZR surveys rather than the multiple generations and sources used for Wellington station. We anticipate starting the actual map drawing for the Wairarapa Line sometime next week and will be hoping the first set of maps produced doesn't need to be revised at all - unlike the Volume 5 situation where so many additionals were added from deciding at a late stage to add lots of extra historical stuff.

Palmerston North Gisborne Line [0Z]: Volume 5 Progress Update 26

So we delayed this until release time. We are pleased to announce that Volume 5 has released. As the legendary Steve Jobs once said, "Real Artists Ship". Being able to release a fully completed volume is a big milestone for this project in its current form. Although we have previously produced some printed volumes of maps, they were neither as comprehensive nor as detailed as this new release, which is the first major release that incorporates both historical and current aerial photography based maps as well as diagram maps together in a format that is specifically designed for use on handheld devices. It is also, naturally, a big deal after spending many years working on this project, to actually embark on the first step of a journey that should see every single map, of which there will likely be an estimated 10,000 in total, released in 2020. 

(To see the maps, scroll right to the bottom of this post for the link)

"Real Artists Ship" is a concept that addresses that a great many people start projects like this but never complete them. Our experience in the historical community in general is that many people have begun projects of historical research that have not resulted in a finished publication. This is likely reflective of the fact that such projects are extremely time consuming and for example, a publication may require many years of intensive research to complete for an output of perhaps a hundred pages or two. This knowledge has in turn informed the development of this project and the determination of the amount of external research that can or should be undertaken. This has been extensively streamlined with the assistance of Archives New Zealand to enable us to access a large number of historical NZ Railways files in a short time frame. Apart from that external resource access, we are proud of the fact that 99% of the remaining work of the project has been carried out from home. Releasing the finished project for public access is a great personal highlight and our timeframe for completion in 2020 reflects on the fact that we expect our personal resources will be shifted elsewhere after that time as we have been hard at work on NZ Rail Maps for the past 12 years exactly (one year for each volume on average) and it has to be completed sometime to enable us to move on to other challenges in life. 

"Real Artists Ship" is also an appropriate concept for the fact that these maps are, in fact, artistic creations and reflect our own creative interests and experiences, The published map format and the contents of each individual map are carefully handcrafted with all the detail designed using the map symbology which is listed in the key for each map.  Although the Qgis layout composer outputs the information stored in the GIS database more or less in a standardised way, each map still has to be checked to ensure it is legible and contains all of the useful information that people are likely to want to make use of with the maps. This is one of the reasons why during production we changed the map formats several times in order to present the maps in a way that makes it easier for us to speed up the production of them considerably due to the many challenges in integrating information from other sources such as the Retrolens historical aerial photos and the Linz Data Service downloadable layers. We expect to be able to speed up the production substantially in the other 11 volumes of the maps. The maps themselves are not as information dense as some other publications such as the Quail Atlas, which incorporate extensive hand customisation of the content of every single page. We could not produce such a large volume of maps if every single page had to be hand edited to change the positions of items or add specific notes or tables of information for just that page. The key reason there is such a large volume of maps overall is a design decision taken that in order to give access to as much information about the history of the New Zealand railway network as is reasonably practical, the maps will not have a scale smaller than 1:10000 in the online format and that many of the maps of individual railway yards and where necessary elsewhere in a particular corridor will be published at scales as large as 1:1000. Being able to publish the maps free of charge via the Google Photos cloud storage system is a big factor as well in being able to put a large number of maps into each volume.

To emphasise how valuable the new maps format is, we have previously commented that it is not financially possible to create a mapping website that provides the same kind of functionality that people can use in Google Maps or even Open Street Maps these days, especially on phones and tablets. And whilst there is something in existence called Open Historical Maps, and we have considered publishing to it as a platform, it still does not offer all of the functionality that is being made available in the format we are publishing in.

So if we cannot publish a full web based map service comparable with other web based map services, the next best thing we can do is to produce static maps in a form that makes it easy to navigate quickly between maps as one is actually travelling along a particular route, whether on a train or by road nearby. This became a desirable capability arising out of a journey by road / sea from Christchurch to Gisborne around 18 months ago and it is appropriate therefore that the map volume for the main line to Gisborne and its various branches is the first to be completed. 

Whilst there are obvious limitations in the format we have implemented, which uses Google Photos to provide the map storage and the user interface, we expect end users to make use of other online maps in conjunction with our map volumes which have in them sufficient geographical information to enable a particular location depicted on a map to be easily located. Whilst it is obvious zooming in and out is not possible as in the Google Maps website, this in fact requires massive resources to produce and implement such as Google's own enormous collection of aerial photography taken by their own satellites and running in their massive world wide data center cloud so on that basis it is easy to understand why it is difficult for us to duplicate that kind of functionality. Whilst Open Street Maps is able to provide zoomable maps, they are only diagrammatic without any aerial photography which uses the most physical resources and costs the most to implement.

It is enough of an achievement for us to be able to generate these maps on purely a non commercial basis with very little money actually having been spent, mostly being a few hundred dollars on some of the early aerial photo scans (prior to the Retrolens site being established) and the web hosting for the nzrailmaps.nz domain name, which redirects to our free Wordpress site. Obviously there are day to day costs such as travel and accommodation in Dunedin to research the Otago Central Line and travel to Archives New Zealand's Christchurch office to research various railway lines in Canterbury. This project definitely could not have happened in its current form without either the free and open source Qgis software, or the freely available resources from or via Linz (the downloable data layers and contemporary aerial photography from the Linz Data Service website and the historical Retrolens aerial photography, which includes many surveys that were specifically undertaken for New Zealand Railways that are of a very high resolution). Another incidental expense for us has been around $2000 spent on our computer hardware over the last few years to have the processing capability and storage needed to assemble all the map resources. Free internet from our local community has also been very material. Of equal significance but perhaps less map specific has been the availability of the Linux operating system as Windows would have driven us crazy by now and it would have cost us unnecessary hundreds of additional dollars in license costs. Regardless of whatever personal prejudice or preference you may have over operating systems, it is a fact that Linux is used on many of the world's most powerful supercomputers and also on many lightweight low cost computers and its reputation as being highly capable for a wide range of different tasks is well earned. And need we mention it is also free and open source software? Because it is free we have not had to spend a single cent on any software for the project.

Volume 5 contains just over 1000 individual maps in six individual sets (photo albums). Each of these maps was individually created and checked visually from the GIS software and there have been numerous revisions and updates to produce the finished product. This can be appreciated by looking at the range of production dates and times that appear on the bottom left edge of each map, which in general show a range of about one month, from early January up to today. We are aware of a small number of issues with the maps that are now available and will be correcting these, but as far as we know the maps in each album are both complete and accurate. We have taken the opportunity to assemble map sets that contain both aerial and diagram maps in the correct sequence in the same set which required custom Python scripting to be developed alongside the production of the maps themselves. The user interface of Google Photos has its challenges at times, but it is fully capable of uploading the maps and assembling them into the correct order with the use of our scripting capability even when the maps are uploaded out of sequence and in a number of partial uploads. Thus it has been relatively easy to deal with instances of some individual maps failing to upload and having to be re-uploaded out of sequence. We now, however, need to develop a script to enable an individual map to be replaced, which involves copying the file modification time from the original version on the computer to its replacement and then uploading the replacement and deleting the original. This will be undertaken and tested over the next few days as the very first map in the PNGL Main Line set has an incorrect title and needs to be replaced. We also need to come up with a way to make the map key have an earlier file modification time than anything else in our Google Photos site so that the key will always appear at the top of every album set, even though it is shared across every album from one single source.

As we have previously stated, we are now ourselves going to take a few days' break to recuperate from what has been an intense few days of very hard work to reach this milestone. We are already planning the work on the next volume but it will not ramp up to a full pace until next week and we have to take a very good look at an appropriate time schedule for it to ensure we stay on track with our expected timeline for publishing the rest of the maps this year.

It is appropriate to make some acknowledgements here and hence below is a partial list. This list is in addition to the individual contributors who are partially acknowledged in the previously released printed map volume, and it is a list for the entire project.

• God, the Holy Spirit and His Son the Lord Jesus Christ, who guide our overall direction of life and provide ever present and inexhaustible support for everything we do.
• NZ Rail Heritage Community Online Signalling & Interlocking Diagram Database
• Archives New Zealand, National Library, and DigitalNZ website.
• Linz Data Service
• Local Government Geospatial Alliance (Retrolens Site - hosted by Abley NZ)
• Chat Club Facebook Community
• Campaign for Better Transport Online Community
• NZ Rail Geography Online Community
• Railway Historical Group, NZ Railfans, New Zealand Locomotives and Locomotives of New Zealand Facebook Communities
• Watts Publications / Steve Watts
• NZ Railway and Tramway Atlas / Quail Map Co.
• NZ Railway and Locomotive Society
• Public Transport Users Association NZ (particularly Canterbury branch)
• Canterbury Railway Society
• Hillview Christian School
• C3 Church Christchurch
• Qgis Software Development Community
• Kiwirail Ltd
• Rail Heritage Trust

Volume 5 Home Page - NZ Rail Maps Wordpress Site

NZ Rail Maps Project Development Report [2020B]

This report as always with these reports is a general overview of the project as a whole across all volumes. Our earlier report 2020A was written on the first day of the year and outlined our expected programme for map development across the whole project this year.

At present with the development of the maps across all volumes, the following sequence is proposed at this time, but could change later:

• Volume 5 (PNGL) - on track to be completed by the end of January
• Volume 6 (WL) - February
• Volume 4 (MNPL) - March
• Volume 3 (ECMT) - Apri
• Volume 1 (NAL) - May
• Volume 2 (NIMT) - June

This means we have priorisitised the 6 volumes of the North Island for the first half of the year, leaving the 6 volumes of the South Island for the second half of the year. Most of these volumes are already completed to some extent from previous work. The largest amount of work is likely to be needed in the two longest corridors apart from PNGL - these are ECMT and NIMT of course. NIMT is one of the more challenging ones because of the length of the corridor and the fact it runs between two of our largest cities.

We now turn to a look at some evolving changes in the way certain technical aspects of the maps themselves have been developed. What is becoming more commonplace in the maps in general in the last year or two is widespread use of aerial photography to provide references for historical and contemporary content. How that aerial photography is used to provide finished maps has varied to date. Now that we are moving towards completed volumes for all areas, this is very relevant.

Map data in a GIS is handled in layers. Each layer manages a particular type of object and generally for convenience we can further divide the types of data into our own personal classifications. Here are some examples:

• Locations (stations, distance pegs etc)
• Bridges and Tunnels
• Features (such as premises that contain sidings)
• Main Line Corridor(s)
• Branch Line Corridor(s) 
• Yards and Sidings
• Buildings
• Structures
• Sites
• Roads
• Land
• Water
• Terrain
• Aerial Photos

All of the map volume GIS projects follow that general style.

The maps are presented in two overall types. Diagram maps are drawn against a terrain background. The terrain is rendered greyscale relief imagery that shows the type of terrain for the area (i.e. flat or hills). It does not have a significant visual impact upon the finished maps as most of the grey shades are less than 50% density.

Aerial maps are drawn against an aerial photography background. With this type of background there are many more challenges mostly in making data layers stand out against a background that is much more dense and often quite dark. There is also the issue of the data layers obscuring visible content in the aerial photography itself. Another issue is the alignment of some of the general (non rail) data layers such as roads.

Because of these issues with aerial photography in particular, the work on Volume 5 has begun to classify layers as being diagram specific. This means when we produce a map of one particular area, such as one station, a diagram map will show much more data than an aerial map. This means the aerial map will be visually a lot less cluttered and more of the aerial background will be visible.

Another important aspect specifically relating to diagrams has been the production of diagrams that cover more than one historical era. We developed a system a year or two back using date range fields to enable us to determine where possible when a particular object became visible and when it became invisible, driven by aerial photography reference. By applying filter formulas, we could cause these objects to appear and disappear on the resulting maps. The main drawback however is the amount of work needed to create these different historical eras. In major yards a lot of changes can take place over a long period.

It has been decided that most diagrams going forward will be composites that show multiple eras in one diagram, with labelling to clarify certain aspects. Eras will be solely represented by producing an aerial map for each era, but as the map information on an aerial map is not the full range of data visible on a diagram map, much of the information that is era specific will not be marked on the aerial map itself. Instead, a reader will use the diagram map to cross reference specific features which are visible on an aerial map.

A small change has also occurred where aerial photo tiles are partly filled by empty canvas in the graphics editing software that produces (mainly) historical tiles, but also is used to remove some black edges that we get in the tiles we download from Linz. For some time we have filled these gaps with current aerial photos. However, to avoid confusing different eras of aerial photography, starting with Volume 5 we will fill these gaps with a white background.

We are also moving towards producing combined map sets that incorporate all aerial and diagram maps together in sequence. This is in addition to producing separate aerial and diagram sets, which has been the norm up until now.

NZ Rail Maps Project Development Report [2020A]: Work Programme For 2020

Welcome to the new year of 2020. This article is really just a summary of previous recent posts setting out our development priorities for the NZ Rail Maps project for 2020. As noted in those posts, it is mainly a rehash of the 2019 vision, which was largely not completed as planned, but is now being given a much greater priority due to resource prioritisation.

The key achievement expected and the highest priority expected in 2020 is to complete all 12 volumes of the maps to a Basic level. For most volumes this will be a relatively straightforward task and these volumes should all be completed by about the middle of the year. For Volume 12, part of the information in that volume relies on Comprehensive level development of part of a rail corridor, so it will be slower to be completed. 

After that, completing all 12 volumes at the Intermediate level is the second priority, and is also highly likely to be finished in 2020, probably towards the end of the year.

The third priority is Comprehensive level development of some areas. This includes the area of Greater Christchurch, where a significant number of Comprehensive maps have already been made. These will be completed as quickly as possible, but the impetus of generating a reference source of historical imagery of the Greater Christchurch area is not as great as it was last year because the work of Christchurch Transport Blog has been downgraded. However, maps for most stations are currently so close to being completed that it should be possible to create these maps quite quickly but in general they are still third priority. Fourth priority is the Comprehensive level maps for Volume 12 (the Otago Central corridor in particular) and fifth priority is Comprehensive level for all other areas around the country. The historical research in Christchurch for the Comprehensive level maps is nearly complete and will be prioritised for completion as quickly as possible.

In 2020, we also plan to change the project's web site and online resource utilisation. At the moment our domain name redirects to our old Trainweb site, but in the short term the redirection will be changed to our Wordpress site. Depending on funding we may move to a fully hosted Wordpress site this year, but that is not certain at the moment. Google Photos remains our key online resource for map publication with most other third party site use being deprecated, as there is enough space on our free Wordpress site to host the PDFs from the Intermediate level.

Map Development Levels [3]: Comprehensive Level

Yesterday we posted about the Basic and Intermediate levels of the maps of the NZ Rail Maps project. Today we are going to have an in depth look at the Comprehensive level of the maps.

The main additional level of information added at Comprehensive level is georeferenced historical maps. This requires that the aerial photography sourced from a site such as Retrolens, or Archives New Zealand, is overlaid over existing georeferenced aerial photography, and then tiles are produced that use the same tile grid system as the originals, so that the reference information for the original tile grid system (of the base aerial photography) can be copied to the new tiles.

If you've ever taken a photo on a camera or phone that is tagged with GPS information, you'll understand a little bit about georeferencing. In this instance, a set of source imagery of an area is divided into tiles of a certain size (the ones we use are usually 4800x7200 pixels) and then for each tile, the georeferencing information is added into several additional files (sidecar files) that have the same title as the original image, but with a different file extension. According to the standard format for GIS georeferencing, there is a file that contains the top left coordinates of the tile, the resolution in each axis, and a couple of other coordinates that tell the GIS how much, if any, rotation of the tile is needed. 

The other important information that is present in one of the other files is the Coordinate Reference System that applies to the tile. Maps are drawn according to what is called a "projection", which is essentially a mathematical model for flattening the earth's spherical form into a two dimensional map. In practice, the earth isn't close enough to a real sphere for these projections to be really simple and easy, so there are many different ones, often produced for specific countries. When coordinates are used in GPS, they are specified in degrees and minutes, which are equal to the coordinates from a CRS called EPSG:4326, which also happens to be the main one that is used by Google Earth. In the maps, we choose to use a CRS called EPSG:3857, which uses coordinates specified in metres. This makes it possible for the map scale to show measurements in metres and kilometres. The main reason for using EPSG:3857 is that it is compatible with the KML files that Google Earth produces, which was an early source of information for the maps but is not used now. To sum up, the combination of the Coordinate Reference System and coordinate information provided for each tile tell a GIS how and where to draw a tile onto the map canvas.

Now, rather than invent our own system of coordinates to tell the GIS how to render the historical map tiles, the easiest way to get them into the GIS is to use an existing tile grid. So first of all we download the existing LINZ base aerial imagery for an area, and then lay out the tiles in a graphical editing program (our preferred software is Gimp, which apart from being free and open source software, is eminently suited to handling very large graphics files on a well resourced computer). We then take the historical images, and overlay them on top of the grid of tiles, using coinciding terrain features to line everything up. This is not as easy as it sounds, mainly because Linz orthorectifies its base imagery whereas Retrolens and other historical imagery sources generally don't. 

Orthorectification is a mathematical image processing task that aims to eliminate visual distortion that is inherent in photography, caused by variations in height of terrain and terrain that is out to the sides of a photographic image (i.e. that is being viewed side on). These visual distortions mean that a series of aerial photos taken close together along a visual path, that overlap each other, will each have a different view of target points in the areas that are overlapped. This isn't a problem if the target area is completely flat, but where the target has raised or sunken areas, the three dimensional aspect of these areas is represented differently depending on how far that area is from the lens centre (because of the viewing angle). The concept for orthorectification is to generate an altitude reference model of the target area (for example, by using LiDAR radar data) and then feed that into a program along with the source tiles, the software then corrects the visual distortion (if the altitude model is accurate enough). In practice this process is only as good as the accuracy of the altitude model, and we have seen a number of instances even in the Linz base aerials of bridges appearing to be curved, that are actually straight, due to the incorrect height information being recorded for that area (most likely the height was represented only for the valley not being crossed and not actually for the bridge itself).

When it comes to raw aerial photography like that we can get from Retrolens, the fact it generally isn't georeferenced, or simply that the centreline coordinates for the aerial survey (in other words, the navigational track that the plane taking the photos flew along) is quite likely different, means that alignment of the historical imagery over the base is often difficult especially for raised or sunken terrain (that is, higher or lower than the railway lines we are interested in mapping). It can get really messy where there are a number of rail lines that are at different heights, such as a few places where a rail line goes on an embankment or bridge alongside or over a shunting yard. So at the best of times, the positioning of features according to a historical image can only ever be an approximation.

When we started doing the Comprehensive level mapping, we commenced in the Cromwell Gorge of the Otago Central Railway with the intention of mapping more or less accurately the route of the old branch railway which is now almost all submerged beneath a hydro lake. There have been so many physical changes in that area, not the least because of the lake itself but also the hydro works, that have made it somewhat difficult to obtain accurate rendering of the route, and our first attempt is currently being revised somewhat. This is partly why the Otago Central maps have not been fully released at the moment. The relevant map volume, Volume 12, is the only one that relies on the Comprehensive level mapping for a significant part of its Basic level information, because of the need to use this technique to generate the route in the Cromwell Gorge which is now mostly invisible; on all the other routes, the closed lines are visible enough to draw them directly from other sources. 

Comprehensive level mapping is inherently much slower to produce than the Basic or Comprehensive levels because of the large amount of graphical data needed to generate map tiles at a useful resolution. Typical historical tiles that are at a useful resolution are at a scale of 1:10000 or less. The most common scales found in NZR aerial photography are 1:4300 for station surveys and 1:5500 for corridor surveys. Highway surveys at typical scales of 1:8000 have also proven very useful in a number of cases. These scales at the the sizes generated in Retrolens scans generally require the use of base imagery at a pixel size around 0.1 to 0.2 metres. This naturally results in a need to process very large image sizes to cover a relatively small area on the ground. For example, a tile size of 4800x7200 pixels at 0.1 metres will cover an area of only 480 by 720 metres, with 34 megapixels in the tile. The result is that a powerful computer and a lot of memory and disk space are needed to process the georeferencing of the images and store them as well. These factors combine to ensure that we have decided in the project that Comprehensive level is a lower priority and is mainly confined to the use of the official NZR surveys except in a small number of major sites where other historical generations are useful.

Well that's been a long and very technical post but we felt it was useful to dig into this information about the levels of maps and we hope this creates a greater understanding of what the basis for the type of maps that we produce is, and what sort of output we can expect to see at the various levels in the future.

Map Development Levels [2]: Intermediate Level

In the last post we described the process of creating the maps to a Basic level. This post describes the second level of maps, the Intermediate level. The main difference between Basic and Intermediate level is that a printable PDF file is produced for each map volume.

For a printed format, like a book, the maps will have the tiles put into pages of a document in a way similar to the electronic format to enable intuitive navigation between pages of the document. The Project will only produce and distribute PDF documents that can be downloaded and printed out by an end user; we did consider online publishing but decided against proceeding with it. The PDFs should be able to be printed out and bound by any print shop, although we have not yet evaluated the real world feasibility of this. To make a printed format map easy to print or copy for casual use on a home computer or regular photocopier, the maps are purely monochrome ("diagrams"), using black, white and shades of grey, rather than colour elements. Consequently a large number of different symbols and line patterns are used, as described in the maps key. 

 

 The process needed to produce a PDF volume file is that data tables need to be compiled from various sources for each route to be included within the volume. The major amount of content is map tiles that are specifically designed to fit onto physical pages and to use the space on those pages to the highest level of efficiency whilst also being easy to navigate. These last two considerations in particular are what have led to maps being designed incorporating navigational aids such as arrows to indicate the direction origin and terminus stations of the particular corridor, as well as adjusting the map orientation so that the rail corridor always goes horizontally across the corridor (aligned more or less with the navigational arrows mentioned above) regardless of the actual direction by the  compass it is going in. Hence the maps always have a north facing arrow shown in the bottom left corner that rotates with the map tile and quite commonly, north will not be at the top of the map. 

 

We have had criticism made of this format by those who suggest that the top of a map should always be the northward direction. Whilst this paradigm may be appropriate to large format printed maps, it is not much of an aid in a situation where only a small segment of a map can be displayed at one time (as is the case in the static tiles format we use online) and where it is therefore necessary to quickly change to a new tile without losing track of orientation. Or to put it another way, you need to be able to be quickly align the way the map is displayed with the actual alignment of the corridor relative to the direction you are physically travelling in, and not to lose track of that when you scroll to the next or last tile. Having to produce all maps "north up" would not only make that potentially very difficult but also waste a lot of space on pages.

Whilst the maps used in a printed volume are the same as the diagrams published at a Basic level, they have to be regenerated at the size needed to work within the page format of a PDF. In a nutshell, whilst the format that is most useful for a phone or tablet is landscape orientation, the usual format for a PDF is portrait orientation. These differences mean that producing the PDF version of a map volume is not a simple cut and paste exercise.

The next post in this series will be made tomorrow and describe the Comprehensive level, which is the most time consuming step and accordingly, we are not doing maps at this level for all stations.

Map Development Levels [1]: Basic Level

At the beginning of this year we defined a vision for 2019 and we also talked about three levels of map development. Although the 2019 vision was not followed for the most part, it is being picked up and pursued much more firmly for 2020. Inherent in that is that the maps will be developed in three levels. This post expands on the initial description given then of what is the Basic level and the intentions to be achieved from it.

The Basic level of the maps is intended to be fully developed for all of New Zealand and it ensures that we have at the very least a full set of maps at this level. It is designed not just to depict the routes of the railway lines in this country, but in fact to supplant and complement existing topographical maps by incorporating most of the key information in these maps that will aid in the use of the maps to locate railway infrastructure easily and quickly. So, Basic level maps include roads, rivers, streams, lakes, contour lines, elevations, place names, property boundaries, bridges, tunnels, and of course, railways. The background of the maps is a set of terrain reliefs. Most of this data comes from freely downloadable layers licensed under Creative Commons by Land Information New Zealand. The real difference off a regular topo map is that we are guaranteeing that the information about the railway system is comprehensive and accurate. It is compiled to a much more detailed level than any topographical map that is available from a mainstream publisher because the maps are specialised to railways.

The Basic maps are published only online and include two separate formats: the basic monochromatic format with terrain relief backgrounds (known as "diagrams") and the identical format with the Linz aerial photography background (known as "maps"). The content at a Basic level is produced by a multi stage process:

• The first stage is to obtain the Linz aerial photography for a volume and then align the railway routes within that volume to the actual rail lines displayed on that aerial photography. For Volumes 1 to 11, these volumes are each based upon one existing main line rail corridor, so the alignment is being done for that main line. For closed branch lines and other rail routes in a volume, we are drawing upon all existing knowledge we have obtained elsewhere about the routes, which may be aided by visible features remaining on the ground that are visible in aerial photography.
• The second stage is to traverse the same routes in the opposite direction and mark in all useful features such as bridges, tunnels, stations and yards. These are all traced off the aerial photography. Our original plan for Basic level has been expanded to make use of Retrolens historical aerial photography for a station site as a reference source to mark in some features that may not be readily visible on current aerial photography. So in this stage the historical aerial photos will help to pinpoint the exact location of individual stations, many of which are closed today.

Once the information is all in place within the maps, they are ready to be produced (published).

Right now we are completing Volume 5 to a Basic level and have been working continuously on it for the past few days (of which more in another post shortly). So it looks like a week or two is what it would take to complete each volume to a Basic level. This will vary a lot as some volumes have already been completed at a Basic level, and others have been only partly completed. A lot of volumes were put together before we were able to get the use of the current Linz aerials and of course well before Retrolens came along, other volumes have Intermediate and Comprehensive level content that have been put into them as well. But for now, Basic level is what is happening first.

The maps in general (across all levels) are designed to be highly usable across a range of different formats. The key assumption we have designed them to at the very lowest level is that they will be adaptable both to modern electronic devices and to hardcopy printed formats. For a handheld device such as a tablet or phone, maps will be published online in a photo album with navigational indications that enable intuitive scrolling between tiles. We don't have the means to publish the maps in a digital globe format that can be hosted by a tile server, so the tiles are put into a photo album and statically scrolled and displayed one at a time. They appear at a variety of scales depending on how much detail needs to be shown, so you will see a small scale where there isn't much information, zooming in to a large scale where there is a lot of information such as a yard or siding layout that needs to be displayed.

NZ Rail Maps Project Development Report [2019L]

After reviewing progress for the year so far, the goals that were set back in January and the progress achieved in them, the following decisions have been reached:

• The maps for Greater Christchurch to be completed as intended, with multiple map generations, for four sections of line - Christchurch to Lyttelton, Christchurch to Lincoln, Christchurch to Burnham and Christchurch to Ashley.
• Research undertaken at Archives New Zealand for map development purposes will be prioritised to that needed to produce the Greater Christchurch maps.
• The estimation to complete the Greater Christchurch maps will be the end of 2019 but it is hard to be sure exactly how much time is needed as there is still a considerable amount of work needed to finish them off. The maps themselves are largely complete but the tile sets have to be extracted from every project file, and then checked for accuracy to see if any adjustments are needed. The maps then have to be traced off the tiles, and correlated with research information. This all takes a lot of time, and could well stretch into 2020.
• After that, work will resume on the schedule published back in January, the priority being Basic maps for all other areas, with Intermediate or Comprehensive in a few areas. The amount of Comprehensive work will be strictly limited as we are running out of disk space in which to store the mosaic tile project files, so probably Basic completion will be prioritised except in Volume 12 which is committed to be completed as Comprehensive.  (To give some perspective, project development files currently occupy 1.5 TB of disk space of which 1.1 TB is mosaic tile projects)

Although the schedule has been pushed back a year by embarking on the Greater Christchurch maps, it was planned as the first year of approximately a 3 year schedule, so there is still a reasonable amount of time in which to push ahead and complete all 12 volumes in 2020. As we now have access to Linz aerial layers off WMTS, this will save greatly on the work needed to download the background aerial imagery for tracing Basic maps, which is quite time consuming. 

However the loss of this year from what was originally planned does mean the overall schedule is going to be reduced. To put it in a nutshell, the plan is that by the end of 2021, the project work will be substantially completed, with maps produced for every part of the national railway network of NZ. This will wind up 14 years of continuous work on the maps project. After that time only maintenance updates will be performed. We haven't yet confirmed this exact schedule for 2020-2021 yet but it will become clearer during 2020 exactly which schedule for project wind down will be appropriate. If 2020 is taken with all 12 volumes to Basic level this leaves 2021 available for Intermediate or Comprehensive level for some of those volumes but not all of them.

Project Diary 2019-09-01: Comment on our relationship with the rest of the NZ rail heritage community

As many of you know, we have not always enjoyed good productive relationships with the rest of the NZ rail community which is partly why this project has its own separate Facebook presence and our involvement of various hobbyist groups has waxed and waned over the years. Ongoing politics relating to this project and to Patrick personally has continued to the present day.

Because of this the decision was made the decision over the last week to withdraw from every Facebook rail-specific group and the implementation of this was completed as of today. A project-related membership is still held of one or two minor and specialist groups that are not read or visited very often which are exclusively related to map production and sharing.

It may be we will review the group membership and rejoin in the future but at the present time we just need a break from these forums which has been a difficult and ongoing issue over the past decade or more.

There are a few people in the Project's own Facebook group who have provided us with or may be able to continue providing us with valuable resources in the future and we would certainly like to thank each and every one of you who has assisted with this and is able to keep assisting with this into the future.

This decision does not in any way affect the continuation of the project in the slightest. It is simply a reflection that a break is needed from these groups which are currently contributing very little to the overall project and we need time to evaluate the contribution they are making to the project and whether it has been worthwhile. We have left those groups on good terms with the admins and could easily choose to rejoin any of the major ones at any future time.

The project will continue the same tomorrow as it did yesterday or last week or the month before and all of its resources including our own Facebook group, Blogger blog, the Wordpress website, and Google Photos albums of all the information will carry on exactly as they do now.

The decision reached is to allow us to concentrate on this project more exclusively and to facilitate stronger relationships with other related projects that are Canterbury based. This includes membership of local and national railway preservation societies and contributions made to them, and the development of rail campaigning that is being facilitated through Christchurch Transport Blog. Membership is retained of a small number of groups related specifically to the campaign interest. It has been found that the personal relationships being fostered through these developments in groups with relatively smaller membership have become far more valuable than much of the associations and acquaintances that have occurred through the broad based rail community Facebook forums and the interaction with their large membership.

We believe the future of this project and of our work in Canterbury and to some extent nationally in promoting rail and rail heritage looks bright for the coming weeks and months and we welcome and acknowledge all the ongoing support of each and every person who has supported this project since it began in its current form at the start of 2008. Currently the project is working to an assumption that over the next two or three years there will be a concerted and conclusive effort to bring as much of the project as possible to a completed state so that the activity level on it can be wound down. This has been alluded to in posts at earlier times referring to development schedules. As this year's original development schedule has not been met, due to other projects becoming more prominent, it will be reviewed at the end of this year and revised accordingly to make sure it is still possible to achieve the original goal. We anticipate it will not be possible to continue the current scale of map production of the last three years in particular although there is no timeframe for when the current level of resourcing is expected to decrease, and as referred to in the last one or two posts, resourcing has been increased this week to enable the mosaics in particular to be produced more rapidly because that is the part of the project that takes the most time to complete.  The additional resources cost a mere $62 to provide and consist of a larger SSD for a computer and have had a significant impact and will enable us to much more efficiently utilise project time which is very important as there are now the competing demands that will continue.

To conclude and summarise the project leadership does not believe that this decision will have any major impact on the project, the project schedule will not be changed from where it was yesterday or in the last week or month, the project welcomes and continues to welcome the contribution of all the members of our Facebook forum and other associated means of contribution, and the project continues to forsee a bright future for the continuation of the project in the coming weeks and months. Thankyou to all personally for your support.

Project Development Report [2019K]

This brief report is to update progress for 2019 so far as the year is now more than half over.

At the start of this year work was happening on the MNPL line which continued until the end of January and then work was switched onto the Main North Line, followed by the Kinleith Branch, North Auckland Line, then parts of the Main South Line around Dunedin. Then we had a look at Hamilton on the NIMT, followed by a significant chunk of the MSL within Canterbury. The next section to be looked at was the Cromwell Gorge.

At the start of June, work was begun on the Christchurch specific project in conjunction with some of the other areas that we support, and the Midland Line work was started towards the end of that month. Since then, work has alternated between Christchurch and the Midland Line.

It seems a good move right now to push ahead to complete Volume 9 for the Midland Line over the month of August, alternating with the Christchurch specific project as necessary. 

In terms of the overall project and its goal to have all 12 volumes completed at a basic level by the end of this year, that hasn't really fired, however neither has the alternative preference of completing any volume at all. We are now substituting an alternative schedule, which is to prioritise the South Island volumes of which we have six, to be completed first at whatever level seems appropriate, which will be more than basic. This includes revising Volume 7, the Nelson Section, with inline yard layouts and other detail using the aerial photography we previously obtained, which was put into an appendix at the end of the most recent edition of the printed volume.

However it is possible to work on the North Island maps by using WMTS to pull in live aerial layers of any area without the extensive work that is needed to download map layers for offline use, so using this to complete any area of the North Island to the Basic level that we proposed could make it happen a lot faster than has been the case to date. 

The priorities have shifted from first completing all of the maps to the Basic level in 2019, to completing as much as possible of the South Island at intermediate or comprehensive level before completing the Basic maps in the rest of the country. Some intermediate or comprehensive work of the North Island will continue throughout next year, especially for the Palmerston North Gisborne Line. Basic maps for North Island areas will also continue this year but we can't say how complete that will be by the end of 2019 at this stage.

Project Diary 2019-06-22A

A quick follow up to the previous post. The key question for many reading this blog is "Does the Christchurch Transport Blog project subtract from the NZ Rail Maps project time commitment"?

The simple and perhaps slightly regrettable answer to this question is "Yes". This is key to understanding the recent slowdown in progress on NZ Rail Maps, since creating a Christchurch specific Qgis project file for parts of Volumes 10 and 11 of the maps has been the key focus for CTB and that has taken all of the priority in the last month or so.

The historical content that is relevant to the NZ Rail Maps project will continue to be included in Volumes 10 and 11 of NZRM and it's not expected to become a separate volume at the moment. However the possibility of the Christchurch map data going into some other type of publication is quite likely as well. It is already being published in the CTB and being used in various contexts related to CTB.

At the same time I also hope to get some historical map data of the Christchurch central business district which so far has eluded me (it would seem the surveys in Canterbury Maps are not available on Retrolens, so I will have to see if I can source CM's data in some form). It may be I end up having to use some lower resolution stuff but my preference by a long way will be at least 0.3 metre pixel resolution or maybe 0.5 or even 0.75 which I have not found anything similar to that goes back to say the 1960s or 1980s and covers the CBD area. 

But at least I can draw upon the NZ Rail Maps mosaics for the historical rail corridor coverage and that is why the effort is going into that at the moment. So I just need a little more work to get what I need out of somewhere.

As we saw last week I posted on how to get NZ Rail Maps in general back on track, refocusing on Volumes 1-12 Basic level maps, and that has to be the most important thing for NZ Rail Maps right now outside Christchurch.

Visit Christchurch Transport Blog to see what is happening there - just give me a week to update it with the current focus and some important stuff that I want to put onto there to give a context to that project. So just keep an eye on it between now and the end of June as I get some posts into there to explain more about what the CTB project is about and where it is going.

Project Development Report [2019J]

Currently I am in the position of reviewing the earlier PDRs of this year and as it is now the middle of the year, considering the best way of advancing the project for the rest of 2019. At the moment there is somewhat of a perception of low productivity more recently, and of having too many unfinished projects on the go at the moment.

At the end of January I looked at what it would take to have all 12 volumes completed at a "Basic" level by the end of 2019. But so far in the first half of the year, there hasn't been much work focused on a basic level completion, with the majority of time spent on "Comprehensive" level, and whilst there has been a lot done at that level, the goal of getting "Basic" levels advanced across all 12 volumes hasn't actually been achieved much at all.

At the same time I have other interests, including new ones that have been developed in the last few months, mainly to do with rail history and activism at a more local level, and whilst it's hard to see where these will go over coming months, it means I have to re evaluate my priorities and look at what can be achieved with the resources I have.Lately more time has been spent developing maps to support these local priorities, and whilst these maps have been part of the bigger scheme, they are being developed at Comprehensive level and will always take priority over any other region of maps production.

The PDR 2019D from earlier this year suggested a timeframe for development of the project with the Comprehensive coverage that might be achieved in addition to full Basic coverage. The main issue is that Basic coverage is not being progressed for a significant number of volumes. This involves getting a full set of recent best available Linz aerial coverage for every centimetre of every volume and then tracing whatever is on it and filling in other details from other current resources. To date whilst a significant amount of aerial coverage has been downloaded from Linz, it has not been imported into map projects and the disk space needed to store these large volumes of downloads (of which only a fraction will be used and it will be stored in another location) is significant.

It is therefore necessary to switch the focus back towards Basic completion in order to free up some disk space for Comprehensive level mosaic work to continue. At the same time I have to take a closer look at the level of Comprehensive coverage that can be realistically completed. Whereas in the past the widespread availability of aerial coverage of a reasonable quality at multiple generations for some areas led me to try to document all the changes in multiple generations, this takes a lot of work to achieve. Because Retrolens is making available NZR station and corridor surveys, I intend for the majority of yards to document the track layout based mainly on these surveys, even where other surveys are available that can document a track layout change. Building/structure layout will try to incorporate changes in major buildings or structures but if at all possible at one all-encompassing generation instead of multiples. So in terms of historical layouts, most of it will be referenced to when NZR official surveys were produced, which for most yards is around the 1970s or 1980s, the goal being to have as much as possible prior to deregulation in 1981 which had a major impact on the railway landscape across the country.

Right now priority is still with Christchurch maps and the mosaics are making major steps forward and there will be tiles produced this week for a lot of stuff. However it is necessary to progress Basic level development across all 12 volumes and so this will be the secondary priority but will be pushed ahead more than it has in the last few months to attempt to get the schedule back on track to where it was intended six months ago.

Project Diary 2019-06-02

In the past week there have been two main areas of work in this project. The first area is the Christchurch City maps referred to in a previous diary entry. It has been decided to combine existing maps of the city created for non-rail purposes with the existing rail maps data drawn for the city, which means there will be a combined Volume 10 and Volume 11 Qgis project that covers only Christchurch. To make things easier this project will share the existing Volume 10 and 11 layers with the rest of those volumes existing projects and will not have separate layers except for the aerial photos that are not of rail corridors or yards. At the same time the existing Volume 10 and Volume 11 projects will have all aerial photography removed that is inside the boundaries of the city.

The other area is the NZ Rail Maps website on Wordpress.com. This currently is accessed by the address http://nzrailmaps.wordpress.com . It will generally be known that the project also uses the domain name nzrailmaps.nz which currently redirects to a Trainweb site for historical reasons. Given the length of time that the Trainweb site has not been used for the project, it is likely the redirection will be changed soon to point directly to the Wordpress site. However there is no intention for the present to change the site URL to nzrailmaps.nz as this requires a paid Wordpress hosting plan. The other change has been driven by the decision that the Wordpress site has enough free space to host the map volume PDF files, which means they can be removed off Scribd thus reducing the number of external sites used to host project content. However the free version of Wordpress is not conducive to hosting the map tiles which are best to remain as Google Photos albums for the time being. Generally we are well behind in publishing completed project content and this needs to be put together and put on the site as more of a priority in future.

It is hoped that the computer used to do the map mosaic tiles will get a bigger SSD soon to enable it to deal with very large project files more easily, or to be able to have two 100-layer projects open at the same time.

Project Diary 2019-05-17

We try to post at least once a week depending on what we are doing in this project. Most of the posts are about specific volumes or regions we are working on. Other times there will be a progress report for the whole project like there was last week. This week it will just be a diary entry because whilst we have been doing a lot of map work this week, not a lot of it has related specifically to NZRM.

This week the majority of our time has been taken up with getting a complete set of maps for Christchurch City. This meant downloading a lot of stuff from Land Information New Zealand, because they have a 3.5 GB limit for the estimated size of each download. As it happens, it is more like 5 GB actual download in a lot of cases. But it meant that at a pixel spacing of 0.075 metres, there are a LOT of layers for the whole city. It works out to something like 60 GB. If you have a few dollars you can get them to courier the data layers to you. With the fact we don't have any spare cash to spend, we just settled for splitting up the city layers into a series of downloads to get all of it. There are some overlaps which means we don't quite have the expected efficiency and because we inadvertently got some gaps between downloads we had to make some extra downloads to be able to fill in those gaps.

So we now have Christchurch City covered for 2018 (central only), 2015 (whole city), 2011 just after the quake (central only) and c.2010 (actually between 2004-2010 and for some reason the dates aren't more precise) which again is the whole city. We also have some historical retrolens layers to overlay at some point. We did consider downloading all the 2011 imagery and in fact did set up a series of downloads in Linz but haven't actually executed those downloads because that imagery is at 0.1 metre pixel spacing and while it is very good it is also just after the quakes so it does have limits. But it is a shame that there isn't high resolution imagery before then that can be downloaded. Even 0.3 metres would be better than the 0.75 metres 2004-2010 stuff.

As far as the NZRM project actually goes the work this week has been on the Otago Central line. First up is to redo the Cromwell Gorge section with the Retrolens scans of survey 1452 which are distinctly sharper than the original ones we paid for a few years ago, and then particular stations. At the moment Omakau-Cromwell is being considered as one Gimp project covering a number of stations. It is separate from the Cromwell Gorge mosaics because of differing resolutions. The Cromwell Gorge part was originally done at 0.75 metre base imagery because that was all that was available for the whole section, but in 2017 the full area became available at 0.4 metres. But initially all that we did was scale the 0.4 metres to 0.75 metres and overlay it with the 0.75 metre stuff. Now we have scaled the 0.75 metre base to 0.4 metres so as to be able to use this scale base imagery for the Gorge section.

Project Development Report [2019I]

This week we have continued working on downloading Linz base imagery and Retrolens historical imagery for the entire MSL corridor. We currently have the base imagery down to about Tumai (just south of Moeraki) and Retrolens down to Pareora, and progress has been a little slow this week, because we took some time on the following subprojects:

• testing the replacement for grid segmenting for most parts of the maps project, which is to scale 0.3 or 0.4 metre imagery to 0.15/0.2 metre size and use doubled dimension tiles in Qgis.
• writing a script to create and copy the sidecar files for the doubled dimension jpeg tiles automatically to save on a lot of manual steps especially with multiple generations of historical images that all need these files duplicated and having to change the pixel size in the world files to match the larger mosaic layer sizes.
• converting the Dunedin-Burnside part of the Dunedin-Mosgiel mosaic tiles Gimp project from a sectioned canvas format to a linear canvas format. 

The Dunedin-Burnside Gimp project is something we have had to work on this week because of wishing to extract the layers for Hillside Workshops, an earlier version of which are displayed at the bottom of this post. Being able to convert to a linear canvas means being able to remove a lot of overlapping layers between the canvas sections. As all of the base layers are always displayed, it speeds up selecting layers for display as well. Since extracting these views of Hillside, we have fixed a few issues with the overlapping of different layers so that one place in particular where quite a rough cut is visible between two layer going across the middle of the workshops site, this has now been made much smoother.

As well as the southern part of Dunedin going out as far as we can with the 0.125 metre resolution urban imagery until we have to switch to the 0.3 metre or 0.4 metre stuff that covers probably the rest of the way to Invercargill except for probably a small number of townships that will have something of higher resolution, we also have to go north to the limits of the Dunedin coverage in that direction so we are working to get that sorted today.

Last week we also downloaded 0.075 metre coverage of Timaru City and will have a look to see what is available going further south. Retrolens coverage downloaded is currently down to Pareora, and rural base imagery down to Tumai. Finishing all the downloads for the MSL corridor and retrieving the 100 GB or so of disk space used remains a high priority so that is one reason why the work seems slow at the moment.

 Hillside Workshops as seen in 1972. As mentioned above a rough overlap can be seen between two layers vertically in the left hand third of the image, this has been correct in current aerial mosaics.

The same Hillside site seen in 2013, which is the latest aerial imagery of Dunedin at high resolution. Now a bit out of date with further changes at Hillside Shops since it closed a few years ago.