*Revealing Climategate Programmer Notes (beyond the emails).

Here goes a sample of remarkable notes embedded in the Climategate source code data. I’m just throwing these in rough style to link back to for a wider encompassing article I’m working on:

function mkp2correlation,indts,depts,remts,t,filter=filter,refperiod=refperiod,$
datathresh=datathresh
;
; THIS WORKS WITH REMTS BEING A 2D ARRAY (nseries,ntime) OF MULTIPLE TIMESERIES
; WHOSE INFLUENCE IS TO BE REMOVED. UNFORTUNATELY THE IDL5.4 p_correlate
; FAILS WITH >1 SERIES TO HOLD CONSTANT, SO I HAVE TO REMOVE THEIR INFLUENCE
; FROM BOTH INDTS AND DEPTS USING MULTIPLE LINEAR REGRESSION AND THEN USE THE
; USUAL correlate FUNCTION ON THE RESIDUALS.
;

pro maps12,yrstart,doinfill=doinfill
;
; Plots 24 yearly maps of calibrated (PCR-infilled or not) MXD reconstructions
; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.
;

;
; Plots (1 at a time) yearly maps of calibrated (PCR-infilled or not) MXD
; reconstructions
; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.

===========================

From documents\harris-tree\recon_esper.pro:

; Computes regressions on full, high and low pass Esper et al. (2002) series,
; anomalies against full NH temperatures and other series.
; CALIBRATES IT AGAINST THE LAND-ONLY TEMPERATURES NORTH OF 20 N
;
; Specify period over which to compute the regressions (stop in 1960 to avoid
; the decline
;

=======================

; Computes regressions on full, high and low pass MEAN timeseries of MXD
; anomalies against full NH temperatures.
; THIS IS FOR THE AGE-BANDED (ALL BANDS) STUFF OF HARRY’S
;
; Specify period over which to compute the regressions (stop in 1940 to avoid
; the decline
;

======================

recon_mann.pro:

; Computes regressions on full, high and low pass MEAN timeseries of MXD
; anomalies against full NH temperatures.
; THIS IS FOR THE Mann et al. reconstruction
; CALIBRATES IT AGAINST THE LAND-ONLY TEMPERATURES NORTH OF 20 N
; IN FACT, I NOW HAVE AN ANNUAL LAND-ONLY NORTH OF 20N VERSION OF MANN,
; SO I CAN CALIBRATE THIS TOO – WHICH MEANS I’m ONLY ALTERING THE SEASON

====================

briff_sep98_e.pro:

; PLOTS ‘ALL’ REGION MXD timeseries from age banded and from hugershoff
; standardised datasets.
; Reads Harry’s regional timeseries and outputs the 1600-1992 portion
; with missing values set appropriately. Uses mxd, and just the
; “all band” timeseries
;****** APPLIES A VERY ARTIFICIAL CORRECTION FOR DECLINE*********

=====================

frs_gts_anom.PRO:

; calculate 1961-1990 synthetic normal from adjusted tmn
print,’Calculating synthetic frs normal’

for iy=nor1,nor2 do begin
tmpfl=strip(string(tmp_prefix,iy))
dtrfl=strip(string(dtr_prefix,iy))
rdbin,tmpgrd,tmpfl,gridsize=2.5,/quiet
rdbin,dtrgrd,dtrfl,gridsize=2.5,/quiet

tmn(nland)=(tmpgrd(nland)-(0.5*dtrgrd(nland)))/10.0
frssyn(nland)=frssyn(nland)+frscal(tmn(nland))
endfor

frssyn(nland)=frssyn(nland)/(nor2-nor1+1)

for im=0,11 do begin
temp=frssyn(*,*,im)
nfin=where(temp gt 0)
temp(nfin)=(temp(nfin)/100.0)*days(im)
frssyn(*,*,im)=temp
endfor
frssyn(nsea)=-999.9

; Calculate synthetic frs from tmin, convert to anomalies
; relative to synthetic mean frs, and apply to normal frs
print,’Calculating synthetic anomalies’

============================

mann/abdlowfreq2grid:

; HUGREG=Hugershoff regions, ABDREG=age-banded regions, HUGGRID=Hugershoff grid
; The calibrated (uncorrected) versions of all these data sets are used.
; However, the same adjustment is then applied to the corrected version of
; the grid Hugershoff data, so that both uncorrected and corrected versions
; are available with the appropriate low frequency variability. There is some
; ambiguity during the modern period here, however, because the corrected
; version has already been artificially adjusted to reproduce the largest
; scales of observed temperature over recent decades – so a new adjustment
; would be unwelcome. Therefore, the adjustment term is scaled back towards
; zero when being applied to the corrected data set, so that it is linearly
; interpolated from its 1950 value to zero at 1970 and kept at zero thereafter.

===================

mann/mxd_eof_rotate:

; Computes EOFs of infilled calibrated MXD gridded dataset.
; Can use corrected or uncorrected MXD data (i.e., corrected for the decline).
; Do not usually rotate, since this loses the common volcanic and global
; warming signal, and results in regional-mean series instead.
; Generally use the correlation matrix EOFs.

=======================

mann/mxd_pcr_localtemp:

; Tries to reconstruct Apr-Sep temperatures, on a box-by-box basis, from the
; EOFs of the MXD data set. This is PCR, although PCs are used as predictors
; but not as predictands. This PCR-infilling must be done for a number of
; periods, with different EOFs for each period (due to different spatial
; coverage). *BUT* don’t do special PCR for the modern period (post-1976),
; since they won’t be used due to the decline/correction problem.
; Certain boxes that appear to reconstruct well are “manually” removed because
; they are isolated and away from any trees.

==================

mann/oldprog/hovmueller_lon:

; Plots a HovMueller diagram (longitude-time) of meridionally averaged
; growing season reconstructions. Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.

=========================

mann/oldprog/calibrate_correctmxd:

; We have previously (calibrate_mxd.pro) calibrated the high-pass filtered
; MXD over 1911-1990, applied the calibration to unfiltered MXD data (which
; gives a zero mean over 1881-1960) after extending the calibration to boxes
; without temperature data (pl_calibmxd1.pro). We have identified and
; artificially removed (i.e. corrected) the decline in this calibrated
; data set. We now recalibrate this corrected calibrated dataset against
; the unfiltered 1911-1990 temperature data, and apply the same calibration
; to the corrected and uncorrected calibrated MXD data.