6. Finding how curves fit together.

There are two situations where you want to sort and compare ring width curves to make them match together in a family:

1. When you have lots of sample-files from one site and want to build a reference curve of them all. A single sample from your site does normally not match other reference curves. But a sum of samples - a reference curve for your site - can often be matched to a reference curve from a place quite far away.

2. When you want to create a serie of ring width mean values out of several radii measured from one and the same log. Using several radii makes your data more representative for the log as a whole, and so makes data easier to match against other ring width data series.

For these two cases, you can use the same procedure in CDendro to find out how curves fit together!

We will now build a mean value ring width curve out of three measured radii from the same log.

nodetrend


Click Collections/Create new collection to create a new empty collection!
The new collection will automatically become the target collection.

In this case we are going to add samples from the same trunk, so we do not want to have any detrending of ring widths. The radiobutton for no detrending is selected by default, so you do not have to do anything now.

When you add samples from different trunks, it is best to select either "NegExp for detrend" or "Heavy detrending".

Note: For a description of the detrending mechanisms, see the section "Whats's new in CDendro?".

Click Collections/Add to this collection and navigate to the NMVK directory under the demo directory where CDendro was installed.
Select all three NMVK14 coordinate files and click open.
(Click on NMVK14A.pos, then hold down the shift key and klick on NMVK14C.pos! Then release the shift key and Click on open!)

The three files are now added to the target collection, but they all have the same "starting point" in time (0) - this is wrong and we have to fix it.

We will now use a semi-automatic method to successively add matching files to a new empty collection. This time we have only three files, but we can work the same way with many more files to match together!

We start by doing a cross correlation to see how well these files fit together.

Click on the button "Find cross correlations"!
Check all the five boxes in the options window which pops up and click ok.

The cross correlation table becomes quite large and vast when there are many members in the collection, so the best matching samples are listed at the end of the table.

This looks fine, so let's get a new empty collection to copy our samples to but then with proper offsets.

Click on Collections/Create new collection! The new collection will automatically be the target and it will float up on top of the three-file NMVK14 collection.
See that the checkbox "Sum by stem" is unchecked for this new empty target collection.

Click on the top of the NMVK14 collection to make it the current window again!

Click on the button "Add best members to target collection"!

See that "Stop before adding a candidate (incremental mode)" is checked.
Then Click Ok!

Now, CDendro uses the cross correlations to select the best pair of matching samples. One is automatically added to the target collection and the other is presented to you as a candidate for addition to the target collection.
Click OK!

What is already in the target collection is now normalized and averaged and handled as a reference curve. Though this time it is based on only one single sample (not much average...).

The candidate sample window automatically pops up with its diagram together with the reference curve.
You have to decide if this is a good match or not!
Click the button "Add to target collection" and accept "-1" in the offset popup window!
Close the sample window!

Now we are back at the original collection where two samples are already copied to the target collection.
Notice that those two old candidate samples are now unchecked which implies that CDendro will not care about them when searching for the best matching sample.

Click the button "Add best members to target collection" again"!
You will be presented with the last candidate.

This time the reference curve is based on averages of the two samples with their proper relative offset already in the target collection.

Inspect the curves and add the sample to the target (offset will be 5) as we did above!

Close the sample window!

Then close also the collection with our three misplaced samples - you do not need to save it!

Note: If the diagram above does not look as your diagram, this is probably because you forgot to uncheck the "Sum by stem" checkbox for your target collection. In that case the mean values of NMVK14A and NMVK14B will be fed as one-stem-data to the summing algorithm which will then only collect what looks as one sample so the plotted mean-value curve of that will not get any enclosing standard deviation curves.

This is what is left: Our target collection, now filled with our three samples at their right offsets!

Click Collections/Save collection As and save the collection as a file to document what we have done!

This time we matched only three files together. But this process of matching curves works well with many files as long as the curves match each other quite well.

Let's finish the job of making a ring width mean value serie for our NMVK14 log:
Click on "Create mean value sample"!
This will update the normalized sum sample window, but you then have to click on the top of that window to make it float up!

Click on the tab Display curve and inspect the resulting curves!
The ring width mean values are at the bottom of the diagram.

Save the ring width mean values either as a .wid file (one ring width per row) or as a decadal file! Though you have to set a dating year to the file before writing it as a decadal file - any year works well, e.g. 3000.
Tips:

There is a very useful command Collections/Create new collection from decadal file. This will build a collection with members out of a decadal file.

Another useful command is Collections/Add to this collection giving you a chance both to add not only extra decadal files but also whole groups of coordinate files or ring width files to a collection.

Such a collection - based on e.g. coordinate files - can then be written out as a decadal file - with members at their correct years or all members dated to a nonsense year as e.g. 3000 or with their correct offsets relative to an unknown "base year" selected to be e.g. 3000.


What to do if the adding process stops?
Please, skip this at first reading!

The method described above works even when there are lots of samples to be sorted in their right order - nota bene when curves actually fit together and are covering each other with a good overlap!

If samples are many and they do not match good enough, the process of successive additions will stop after a while telling that the next best candidate has not those good correlation values asked for.

How to find best matching samples?

After an "Add best members..." operation, there may still be some samples left which could be added after a visual inspection.
To find out how all samples left matches your new target collection:
1. For the target collection, click "Create mean value sample"
2. See that the resulting sample is selected as the Reference!
3. See that the Settings/Options for normalization and matching/Least overlap... value is set to 50
3. In your "many-samples" collection click "Test towards reference"

This will give you a list where all still not added samples (those still checked) are listed with their best and next best matching points towards the reference. If you find any sample which you really believe could be set at its right position, then double-click it and do a visual inspection. If it really fits, consider adding it to the target collection.

How to restart the addition process?

When you know that the samples cover a long time span - you may restart the addition process to create a second collection.

Building the first collection:

1. Start from an "unsorted many-samples" collection containing those samples you want to find the right offsets for.
Get a new empty target collection and start the addition process as described above.
When there are no more good looking samples to add, save your new collection, name it e.g. BEST1.fil. Close all windows.

Note: If you have to stop the process somewhere in between, you may later resume the addition process. Just open the target collection and see that it is selected as target. Then open that "unsorted many-samples" collection and click on "Add best members to target collection". CDendro will then automatically uncheck those members already within the target collection and then continue the searching for good candidates among those not yet copied.

Restarting - Building the second collection:
Now, we want to build another collection based on samples not in your BEST1.fil-collection.
2. See that BEST1.fil is your target collection.
3. Click the button Check all in that "unsorted many-samples" collection.
4. Select the command Collections/Uncheck members already in target collection.
5. Create a new target collection!
6. Click on that "unsorted many-samples" collection.
7. Click Find cross correlations.
8. Inspect the list at the end of the correlation table and select one sample which is quite long and has high correlation values in relation to other samples.
9. Find this sample in the list with samples above the report frame and double-click on it to open a sample-window with a diagram for it.
10. Add this sample to your target collection at offset 0! Close the sample window!
11. In your "unsorted many-samples" collection, click on "Check all" to make all members of that collection available in the adding process towards the new target collection. That way the new collection may grow to cover parts of the BEST1 collection.
12. Click on "Add best members to target collection" and the addition process starts up.

How to select those members not within two already created collections, BEST1 and BEST2?
1. Click "Check all" in that "many-samples" collection.
2. Open the BEST1 collection and select it as the target collection.
3. Click in that "many-samples" collection to make it the top window.
4. Select the command "Collections/Uncheck members already in target collection"
5. Open the BEST2 collection and select it as the target collection.
6. Click in that "many-samples" collection to make it the top window.
7. Select the command "Collections/Uncheck members already in target collection"
Now the members checked in your "many-samples" collection are only those not within the BEST1 or BEST2 collections.
You may save this limited collection as e.g. NOTBEST1BEST2.fil - only checked members will be written!

How to add two collections?

When  you have two collections BEST1 and BEST2 and want to add them together:
1. Click on "Create mean value sample" in both collections.
2. Select the BEST1.D12 window as the Reference sample.
3. Do a whole sample correlation analysis in BEST2. If the collections do not fit together, stop here!
4. See that the BEST1 collection is your target collection.
5. From BEST2, select the command Collections/Copy collection members to target collection. Accept the proposed offset based on your previous correlation analysis.
6. The members of BEST2 will be copied into BEST1. Save BEST1 as BEST12.fil. Members of BEST2 which are already members of BEST1 will not be copied.


Quality control

There are four buttons in a collection window for quality control calculations:

1. "Find cross correlations" which is already described above.

2. "Test towards reference" which gives you a table showing how each member matches towards the current reference. If you see that the box "With block checking" is checked, you will get a list like that shown above. If a block of a sample has a high correlation while the other blocks have low correlations, this is an indication of a missed ring. This can be checked by double-clicking on the sample in the collection list and then inspecting the diagram as shown above. Here a year-ring is obviously missing between year 80 and 90.
Note: By unchecking all and then checking certain members you can limit the length of the list when you want to inspect only those members.

3. "Test towards rest of collection". In this case each member is removed from the collection before a reference is created out of the rest of the collection and that member is compared to the reference. When there are few members in a collection this is an important tool for analysis.
Note: Also this command can be run in block checking mode. The block length and the block distance parameters for lists are set by the command Settings/Options for normalization of ring widths and for matching

4. "Show time lines" which gives you a plot showing how the samples overlap each other over time. Any zero values within a sample is marked with an "x" in the time line.

Collection reports can be saved in a file. You can copy (Ctrl-C) selected contents of a report and paste (Ctrl-V) it into a document.

Time line plots can be sent to the printer and then cut and taped together in a long strip.

Note: The "Create mean value sample" button operates only on checked members of a collection. This way it is easy to exclude a member from the reference curve of a collection to e.g. compare that member visually to the rest of the collection. To do this, uncheck that member. Click on the "Create mean value sample"-button. Double-click on the unchecked member to open it and click for a correlation analysis in its window.


Reviewing reference curves


With the automatic mechanism for adding best members to a collection in CDendro, you can quite soon rebuild an independent reference curve from its components.

An independent or freestanding curve is a curve
1. where the cutting year of the youngest sample is known and
2. where all samples date each other by matching each other well and by
3. overlapping each other in a reasonable way.
Though such a curve is freestanding, it should be crossdated towards curves from other places to check that there are no missing or extra years.

Other (published) curves which are not independent are those with e.g. a block of zero-values somewhere in the middle. Or with a younger part overlapping the older part with 30 years and a correlation coefficient of 0.14. An example of a zero block curve is the ITRDB file swed023w.rwl from Jämtland in Sweden. It covers the time 1827 - 1107 but has 25 years without values around 1327. The older part of this curve is no doubt dated towards a reference curve from a nearby place. Also the younger part of the curve must have been dated that way.

This creates a methodological problem: If many curves depend on the dating of other curves which depend on the dating of yet another curves we may run into trouble if an error is introduced somewhere on the way. This is especially true the longer back in time we are dating.

Therefore, it is essential to document how a curve is created and dated and on what material it is based and how samples are related to each other. This is especially important when some samples are short or has not so good correlation to other samples in the curve.


Building a reference curve by seeding with a known curve

If you already have a reference curve from a nearby place, you can use that curve as a basis for a new curve.

Start with a cross correlation analysis of your sample data and start building a curve with the "Add best members to target collection" button. When you have a number of overlapping samples in your new collection, check its correlation towards that curve from the nearby place. If there is a strong correlation between the two curves, you can use that nearby curve as a "seed" or starting point when building on.

Save your nearby reference curve as a normalized data file (.d12) and add that file to your new collection. Then build on. When you are ready, you can easily remove the "nearby curve" either by unchecking that member or by deleting it. Using a seeded curve makes you work much faster than otherwise. Though if  you have any errors in that nearby curve, they will surely be transmitted to your new curve - so be careful!

One more word of caution: Climate may change over years and there may be cases where two places have the same climate during one period, but not during another. Especially when going far back in time, be very careful and validate (and document) what you are doing. Guessing is not a tool to use!

Note: When trying to rebuild an already existing curve, it may be suitable to have the "Tell bad dating" checkbox checked. On the other hand, when you are building from your new own samples which surely are not already dated (or if dated, you know their dating is wrong) , the box should be unchecked to avoid unnecessary warnings.


I appreciate all comments on the usage of CDendro and any suggestions on how to make the program better!



Copyright © 2007, Cybis Elektronik & Data AB. Email: