The Database class stores a matrix of data, together with the sample period (the class derives from the Sample class), and the names of the variables. Functions to create a database from disk files (ASCII, OxMetrics and spreadsheet formats) are provided.
In addition, the Database class has built-in support to select variables (for modelling) from the database. Variables are selected by name, optionally with a lag length, and allocated to a group (e.g. to distinguish between dependent and independent variables). A sample period for the selection can be set. This selection can then be extracted from the database. The selected sample is always adjusted so as not to include missing values (.NaN).
The Database class also supports daily and weekly data. This is different because some years have 52 and others 53 weeks. Therefore, the method of using a fixed frequency, as implemented in the Sample class from which the Database class derives, does not work. Instead, a database can be dated:
Some sample code is given in ox/samples/database.
A choice variable has type DB_CHOICE. In that case, there is an array of text labels associated with values: value zero corresponds to index zero in the array, etc. See the example in ox/samples/database/dbchoice.ox.
To use the class, link in the database code and include database.oxh. This is most easily done by adding an #import <database> statement at the top of your source code.
| Base class | Sample class |
| Construction/Information | |
|---|---|
| Database | Constructor |
| Create | Create a database (not needed when using Load...) |
| Empty | empties the database and sample |
| GetDbName | get the database name |
| GetSize | number of observations in the database |
| GetVarCount | returns number of variables |
| Info | Print summary information |
| IsEmpty | returns TRUE if the database is empty |
| SetDbName | set the database name |
| Tabulate | cross-tabulation of discrete variables |
| Data input/output | |
| Load | Load a data file |
| LoadCsv | Load a CSV spreadsheet file |
| LoadDht | Load a GAUSS .dht/.dat file |
| LoadDta | Load a Stata .dta file |
| LoadFmtVar | Load a formatted .dat file |
| LoadIn7 | Load a .in7/.bn7 file |
| LoadObs | Load a human-readable file which is ordered by observation |
| LoadVar | Load a human-readable file which is ordered by variable |
| LoadXls | Load an Excel old-format spreadsheet file |
| LoadXlsx | Load an Excel Open XML spreadsheet file |
| Save | Save the database |
| SaveCsv | Save the database as a CSV spreadsheet file |
| SaveFmtVar | Save the database as a formatted .dat file |
| SaveIn7 | Save the database as a .in7/.bn7 file |
| SaveObs | Save the database as a human-readable file which is ordered by observation |
| SaveVar | Save the database as a human-readable file which is ordered by variable |
| SaveXls | Save the database as an Excel spreadsheet file |
| SaveXlsx | Save the database as an Excel Open XML spreadsheet file |
| Adding/removing variables/observations | |
| Append | Append variables |
| Deterministic | Create deterministic variables |
| Grow | Add observations to the database sample |
| Recode | Recode observations of a variable |
| Remove | Remove the named variable from the database |
| RemoveObsIf | Remove observations from the database |
| Rename | Rename a variable |
| Renew | Renew data on variables (append if new variable) |
| RenewBlock | Renew data without checking |
| SetVar | Sets variable(s) by name |
| Shrink | Shrinks the database |
| SortBy | Sort the database by a variable |
| Extraction | |
| GetAll | Return the whole data matrix |
| GetAllNames | return all variable names |
| GetSample | Returns text with database sample |
| GetSampleByIndex | virtual function returning sample text |
| GetVar | Get the data on the named variable(s) |
| GetVarByIndex | Get the data on variable(s) by the variable index |
| GetVarIndex | Get the database indices of the named variables |
| GetVarNameByIndex | Get variable name(s) by database index |
| Database sample information: Sample class | |
| GetFrequency | data frequency |
| GetIndex | index of time point |
| GetPeriod1 | period of first observation |
| GetPeriod2 | period of last observation |
| GetSize | number of observations in sample (the database) |
| GetYear1 | year of first observation |
| GetYear2 | year of last observation |
| ObsPeriod | finds period of observation index |
| ObsYear | finds year of observation index |
| Resample | changes the sample period and frequency (but not the sample size) |
| Dated data, choice data, and variable types | |
| GetDateByIndex | get the date of a database index |
| GetDates | returns date variable or <> if undated |
| GetIndexByDate | get the index for a date (must be dated) |
| GetIndexByDates | get the indices for a date range (must be dated) |
| GetObsLabel | gets the string representation of an observation |
| GetVarChoices | gets the choice labels (array of strings) |
| GetVarChoicesByIndex | gets the choice labels (array of strings) |
| GetVarType | gets the variable type |
| GetVarTypeByIndex | gets the variable type |
| IsDated | returns TRUE if the database is dated |
| SetDates | sets the date variable |
| SetVarChoices | sets the choice labels (array of strings) |
| SetVarChoicesByIndex | sets the choice labels (array of strings) |
| SetVarType | sets the variable type |
| SetVarTypeByIndex | sets the variable type |
| Variable selection | |
| DeSelect | Remove a selection |
| DeSelectByIndex | Remove a selected variable by its index |
| DeSelectByName | Remove a selected variable by its name |
| FindSelection | Find selection index of a variable |
| GetGroup | Get a group of selected variables |
| GetGroupLag | Get a group of selected variables wich have the specified lag |
| GetGroupLagNames | Get the names of a group of selected variables wich have the specified lag |
| GetGroupNames | Get the names of a group of selected variables |
| GetMaxGroupLag | Get the highest lag in a selection group |
| GetMaxSelLag | Get the highest lag in the selection |
| GetSelInfo | Get the whole selection info |
| Select | Select named variables into a selection group |
| SelectByIndex | Select variables into a selection group by index |
| SetSelInfo | Set the whole selection info |
| Sample selection | |
| ForceSelSample | Set the selection sample without checking |
| ForceSelSampleByIndex | Set the selection sample without checking |
| GetSelEnd | Get the observation index of the selection end |
| GetSelSample | Returns a string with the selected sample text |
| GetSelSampleMode | Returns the current selection sample mode |
| GetSelStart | Get the observation index of the selection start |
| SetSelDates | select a sample by year, month, day |
| SetSelSample | Set the selection sample |
| SetSelSampleByDates | select a sample by date value |
| SetSelSampleByIndex | select a sample by datebase indices |
| SetSelSampleMode | Set the selection sample mode |
Database::Append(const mNew, const asNew); Database::Append(const mNew, const asNew, const iT1);
Appends the k new variables to the database, storing the observations and variable names. It is an error if the variable already exists (use Renew or SetVar instead), or if the new variable has more observations than the database (use Grow first or SetVar instead).
The first observation has database index iT1 (omit the third argument, use 0 if the variables start at the same sample point as the database), the last is the end of the database sample, or the end of mNew, whichever comes first.
Database::Create(const iFreq, const iYear1, const iPeriod1,
const iYear2, const iPeriod2);
Database::Create(const iFreq, const iYear1, const iPeriod1, const cT);
Database::Create(const cT);
Creates a database. Use this when the database is not to be loaded from disk. The Append member function allows adding data to the database. Use Empty first if the database is not empty.
Create(cT) is equivalent to Create(1, 1, 1, cT, 1).
Database::Database();Constructor. Calls Empty and sets the sample selection mode to SAM_ALLVALID.
Database::DeSelect(); Database::DeSelectByIndex(const iSel); Database::DeSelectByName(const sVar, const iGroup, const iLag);
DeSelect clears the current variable and sample selection completely.
The other two functions delete specific variable(s) from the current selection. The selection sample is not changed.
Database::Deterministic(const iCseason);
Appends constant, trend and seasonals to the database. These variables are named Constant, Trend and Season. Season_1, ..., Season_x, where x is the frequency.
Season has a 1 in quarter 1 (for quarterly data), and zeros elsewhere, Season_1 has a 1 in quarter 2, etc.
If iCseason is 0, normal seasonals are created. If iCseason is 1, the seasonals are centred (with quarterly observations, for quarter 1: 0.75, -0.25, -0.25, -0.25, ...), in which case the names are CSeason, CSeason_1, ..., CSeason_x. No seasonals are created if iCseason is < 0.
Database::Empty();
Database::FindSelection(const sVar, const iLag);
Returns the selection index of the specified variable with the specified lag, or -1 if it is not selected. ,
Database::ForceSelSample(const iYear1, const iPeriod1,
const iYear2, const iPeriod2);
Database::ForceSelSampleByIndex(const iT1, const iT2);
Returns the number of observations in the sample.
Sets a selection a sample for the variables previously selected with the Select function. This function does not check for missing values. Use SetSelSample() to set a sample with checking for missing values.
Database::GetAll(); Database::GetAllNames();
GetAll() returns the whole database matrix. GetAllNames() returns an array with all the variable names.
Database::GetDateByIndex(const iT1)
Returns the date at the specified index (the same as GetDates()[iT1]). This can be printed with the "%C" format, or translated using dayofcalendar. The database must be dated.
Database::GetDates();
Returns a column vector with the date variable or <> if the database is undated.
Database::GetDbName();
Returns a string with the database name.
Database::GetGroup(const iGroup); Database::GetGroupLag(const iGroup, const iLag1, const iLag2);
GetGroup returns a T x n matrix with all selected variables of group iGroup.
GetGroupLag returns only those with the specified lag length. If no database sample has been selected yet, the return value is a 0.
Database::GetGroupLagNames(const iGroup, const iLag1,
const iLag2, aasNames);
Database::GetGroupNames(const iGroup, const aasNames);
GetGroupLagNames gets the names of all selected variables of group iGroup which have a lag in iLag1 ... iLag2. GetGroupNames gets the names of all variables in the group. The selection sample period must have been set.
Database::GetIndexByDate(const dDate1) Database::GetIndexByDates(const dDate1, const dDate2)
GetIndexByDate returns the index closest to the specified date.
GetIndexByDates returns the start and end indices of the specified period as an array of two integers. This can be used, for example, as [t1,t2] = GetIndexByDates(dayofcalendar(1990,1,1), dayofcalendar(1990,12,31)).
Database::GetMaxGroupLag(iGroup); Database::GetMaxSelLag();
GetMaxSelLag returns the highest lag in all selected variables. GetMaxGroupLag returns the highest lag in selected variables of the specified group iGroup.
Database::GetObsLabel(sVar, const iT);
Return a string with the text of the observation, taking into account whether it is a choice, date or normal value.
Database::GetSample(); virtual Database::GetSampleByIndex(const iT1, const iT2)
GetSample returns a string with the database sample, e.g. "1980(1) - 1990(2)". GetSampleByIndex is called to create the text.
GetSampleByIndex writes the sample text for the sample with database indices iT1, iT2.
If iT1<0 the output is "no sample"; if iT2<0 the end-period is omitted, so only a sample date is returned.
Database::GetSelStart(); Database::GetSelEnd();
GetSelStart returns the database index of the first observation of the selected sample. GetSelEnd returns the database index of the last observation of the selected sample.
Database::GetSelInfo();
Returns a 1 x 5 array with the selection information as follows (c is the number of selected variables):
Database::GetSelSample();
GetSelSample returns a string with the text of the selected database sample, e.g. "1980(1) - 1984(2)". GetSampleByIndex is called to create the text.
Database::GetSelSampleMode();
GetSelSampleMode returns the current sample selection mode (also see findsample), one of: SAM_ALLVALID, SAM_ENDSVALID, SAM_ANYVALID.
Database::GetVar(const sName); Database::GetVarByIndex(const iVar);
Returns a matrix with the specified variable(s), or the empty matrix if the variable cannot be found. ,
Database::GetVarChoices(const sVar); Database::GetVarChoicesByIndex(const iVar);
Return an array of strings with the choice labels; if multiple variables are specified the return value is an array of arrays of strings.
Database::GetVarIndex(const asName);
Returns the database indices of the specified variable(s), or the empty matrix if none are found.
Database::GetVarCount();
Returns the number of variables in the database. ,,
Database::GetVarNameByIndex(const iVar); Database::GetVarType(const sVar); Database::GetVarTypeByIndex(const iVar);
GetVarNameByIndex returns an array with the names of the specified variable(s). If iVar is a scalar, s single string is returned.
GetVarTypeByIndex and GetVarType return the variable type, one of: DB_DOUBLE, DB_DATE , DB_CHOICE.
Database::Grow(const cTadd);
Database::Shrink(const cTdel);
Database::Info();
Prints information on the contents of the database.
Database::IsDated();
TRUE if the database is dated, FALSE otherwise.
Database::IsEmpty();
TRUE if the database is dated, FALSE otherwise.
Database::Load(const sFilename);
Database::LoadCsv(const sFilename);
Database::LoadDht(const sFilename, const iYear1,
const iPeriod1, const iFreq);
Database::LoadDta(const sFilename, const iYear1,
const iPeriod1, const iFreq);
Database::LoadFmtVar(const sFilename);
Database::LoadIn7(const sFilename);
Database::LoadObs(const sFilename, const cVar,const cObs,
const iYear1, const iPeriod1, const iFreq,
const fOffendMis);
Database::LoadVar(const sFilename, const cVar,const cObs,
const iYear1, const iPeriod1, const iFreq,
const fOffendMis);
Database::LoadXls(const sFilename);
Database::LoadXlsx(const sFilename);
Returns FALSE if the loading failed, TRUE otherwise.
Load creates the database and loads the specified data file from disk. The file type is derived from the extension. Supported are: .csv, .dat (see LoadFmtVar), .dht, .dta, .in7, .xls, .xlsx.
LoadDht creates the database and loads the specified Gauss data file from disk.
LoadDta creates the database and loads the specified Stata (up to version 14) data file from disk.
LoadIn7 creates the database and loads the specified OxMetrics file (which is the same as a PcGive 7 data file) from disk.
LoadFmtVar creates the database and loads the ASCII file with formatting information from disk. In OxMetrics this is called `Data with load info'. Such a file is human-readable, with the data ordered by variable, and each variable preceded by a line of the type:
> name year1 period1 year2 period2 frequency
LoadObs and LoadVar create the database and load the specified human-readable data file from disk. The data is ordered by observation (LoadObs), or by variable. Since there is no information on the sample or the variable names in these files, the sample must be provided as function arguments. The variable names are set to Var1, Var2, etc., use Rename to rename the variables.
LoadCsv, LoadXls and LoadXlsx create the database and load the specified spreadsheet file from disk. A .csv file is a comma-separated file, an .xls file is an old-format Excel worksheet. The modern Excel spreadsheet file has an .xlsx extension.
Database::Recode(const sVar, ...);
| recoding arguments | example |
|---|---|
| scalar old-value, scalar new value | 2, 1 |
| 1x2 matrix m, scalar new value | <-.Inf,2>, 1 |
| old value is closed interval [m[0],m[1]] |
Note that the intervals are closed, and that recoding processes the arguments from left to right.
For example
Database::Remove(const sName); Database::RemoveObsIf(const vRemove)
Removes the named variable or specified observations from the database.
Database::Rename(const sNewName, const sOldName);
Renames a database variable. To rename more than one variable at once, both most be arrays of the same size, and all old names must exist in the database.
Database::Renew(const mNew, const asName); Database::Renew(const mNew, const asName, const iT1);
Renews the observations on the named variable. The first new observation has database index iT1, the last is the end of the database sample, or the end of mNew, whichever comes first. If a non-existent variable is renewed, the variable is created first using Append. The database sample can be changed by Grow or SetVar, not by Renew or Append.
Database::RenewBlock(const mNew, const iVarIndex);
Renews the observations on the k variables starting from the first, without any checking for existence.
Database::Save(const sFilename); Database::SaveCsv(const sFilename); Database::SaveIn7(const sFilename); Database::SaveFmtVar(const sFilename); Database::SaveObs(const sFilename); Database::SaveVar(const sFilename); Database::SaveXls(const sFilename); Database::SaveXlsx(const sFilename);
Save derives the file type from the file extension (using .in7/.bn7 if no extension is given). Supported are: .csv, .dat (see SaveFmtVar), .dht, .dta, .in7, .xls, .xlsx.
SaveIn7 saves the database as a OxMetrics file.
SaveFmtVar saves the database as a formatted ASCII file. Also see under LoadFmtVar.
SaveObs and SaveVar save the database as a human-readable data file, ordered by observation, or by variable. Also see under LoadObs, LoadVar.
SaveCsv, SaveXls, and SaveXlsx save the database as a comma-separated, or old-style Excel, or Excel spreadsheet file respectively. Also see under LoadCsv, LoadXls, LoadXlsx.
Database::Select(const iGroup, const aSel);
Database::SelectByIndex(const iGroup, const iVar,
const iLag0, const iLag1);
Selects variables by name and with specified lags, and assigns the iGroup number to the selection. The aSel argument is an array consisting of sequences of three values: name, start lag, end lag. After a sample period is set, the selection can be extracted from the database.
Database::SetDates(const vDates);
If the database is not yet dated, vDates is set as the date column (the first column). Otherwise vDates replaces the current date column.
Database::SetDbName(const sName);
Sets the database name to sName.
Database::SetSelInfo(const asInfo);
Sets the selection based on the specified input array. No checking is done on the input values. The selection information should be organized as follows (c is the number of selected variables):
The last two arguments may be omitted, in which case a call to SetSelSample may be required.
Database::SetSelDates(const iYear1, const iMonth1, const iDay1,
const iYear2, const iMonth2, const iDay2)
Returns the number of observations in the sample.
This is the equivalent of SetSelSample that can be used when the database is dated.
Selects a sample for the variables previously selected with the Select function. The actually selected sample will be the largest starting from the specified starting date (but not exceeding the specified end date) without any missing values using the default selection mode. Use SetSelSampleMode to change the selection mode. Use DeSelect to deselect the current sample and variables.
Database::SetSelSample(const iYear1, const iPeriod1,
const iYear2, const iPeriod2);
Returns the number of observations in the sample.
Selects a sample for the variables previously selected with the Select function. The actually selected sample will be the largest starting from the specified starting date (but not exceeding the specified end date) without any missing values using the default selection mode. Use SetSelSampleMode to change the selection mode. Use DeSelect to deselect the current sample and variables.
Database::SetSelSampleByDates(const dDate1, const dDate2); Database::SetSelSampleByIndex(const iT1, const iT2);
Returns the number of observations in the sample.
Database::SetSelSampleMode(const iMode);
Database::SetVar(const mNew, const asName);
If any of the named variables exist in the database, the content is changed, otherwise the new variables are appended. If T is larger than the sample size of the database, the database is extended (unlike Renew); if it is shorter, the new (or changed) variable will have missing values for the remainder. If the database has not been created yet, it is created with frequency of unity (annual/undated). ,
Database::SetVarChoices(const sVar, const asChoices); Database::SetVarChoicesByIndex(const iVar, const asChoices);
Makes the variable in a choice variable (DB_CHOICE), and assigns the array of labels. The label for value i is asChoices[i]. Choice labels are only preserved in in7, xlsx and dta files. If asChoices is an empty array, the choice labels are removed and the variable is changed into a normal variable (DB_DOUBLE). ,
Database::SetVarType(const sVar, const iType); Database::SetVarTypeByIndex(const iVar, const iType);
Database::SortBy(const sVar);
Sort the database by a variable.
Database::Tabulate(const sX); Database::Tabulate(const sX, const sY); Database::Tabulate(const sX, const sY, const sOption);
Returns an array of two elements with the vector of values and corresponding counts (tabulation), or an array of three elements with the vector of x-values (row), y-values (column) and corresponding matrix of counts (cross-tabulation).
Tabulates one variable or cross-tabulates two variables. An example is given in samples/database/dbchoice.ox.
The Modelbase class derives from the Database class to implement model estimation features. Modelbase is not intended to be used directly, but as a base for a more specialized class. A range of virtual member functions allows for customization of the class. Modelbase facilitates interactive use with OxMetrics through the OxPack program. Dialogs and a test menu are easily created by overriding just a few virtual functions. Additional information is provided separately. In most cases, model estimation involves the following steps (key virtual functions are given in parentheses):
Modelbase has a few essential properties to track this procedure:
| Model status | GetModelStatus, SetModelStatus |
| Maximization method | GetMethod, SetMethod |
| Estimation result | GetResult, SetResult |
| Parameters | GetPar, SetPar, GetParCount, SetParCount |
| Fixed/Free parameters | GetFreeParCount, GetFreePar, SetFreePar, |
| FixPar, FreePar | |
| Covariance | Covar |
The samples/database/mbclass.ox example shows a minimal Modelbase implementation:
#include <oxstd.oxh>
#import <modelbase>
class Ols : Modelbase
{
decl m_mRes;
decl m_dSigmaSqr;
Ols();
GetPackageName();
GetPackageVersion();
DoEstimation(vP);
};
Ols::Ols()
{
Modelbase();
}
Ols::GetPackageName()
{
return "Ols";
}
Ols::GetPackageVersion()
{
return "1.0";
}
Ols::DoEstimation(vP)
{
decl cp = columns(m_mX);
SetParCount(cp);
olsc(m_mY, m_mX, &vP, &m_mCovar);
m_mRes = m_mY - m_mX * vP;
m_dSigmaSqr = m_mRes'm_mRes / (rows(m_mY) - cp);
m_mCovar *= m_dSigmaSqr;
SetResult(MAX_CONV);
return vP;
}
main()
{
decl ols = new Ols();
ols.Load("data/data.in7");
ols.Deterministic(FALSE);
ols.Select("Y", {"CONS", 0, 2});
ols.Select("X", {"Constant", 0, 0, "INC" , 0, 2});
ols.Estimate();
}
The program produces:
Ols package version 1.0, object created on 19-03-1999
---- Ols ----
The estimation sample is 1953 (3) - 1992 (3)
The dependent variable is: CONS
Coefficient Std.Error t-value t-prob
CONS_1 1.31039 0.07564 17.3 0.000
CONS_2 -0.352108 0.07915 4.45 0.000
Constant -2.17250 11.19 0.194 0.846
INC 0.508481 0.03606 14.1 0.000
INC_1 -0.577251 0.05816 9.92 0.000
INC_2 0.112122 0.05325 2.11 0.037
log-likelihood .NaN
no. of observations 157 no. of parameters 6
AIC .NaN AIC/T .NaN
mean(CONS) 875.78 var(CONS) 182.397
At first sight it may be somewhat surprising how much this program achives with so little coding. But, with an understanding of virtual functions, the documentation below, and the actual source code of Modelbase (in ox\src), it should be possible to implement a Modelbase derived package. Other examples of the use of Modelbase are the Arfima and DPD packages.
Functions which are used in a minimal implementation are marked as follows:
| Base class | Database class |
| General | |
|---|---|
| Modelbase | constructor |
| ClearEstimation | removes results fom previous estimation |
| ClearModel | sets model status to MS_NONE |
| FindGroup | translates variable status group string to index |
| FindMethod | translates estimation method string to index |
| GetGroupLabels* | get the array of labels for Y_VAR,X_VAR,... constants |
| GetMethod | get the estimation method |
| GetMethodLabel | get the label for the current estimation method |
| GetModelLabel | get the label for the model |
| GetModelLabels* | get the array of label for the model estimation constants |
| GetModelStatus | get the model status (MS_....) |
| GetPackageName* | returns name of the package |
| GetPackageVersion* | returns version of the package |
| GetResult | get the estimation result |
| Grow | extend database and update the deterministic terms |
| Init | resets all variables to default |
| IsUnivariate | returns TRUE if only one Y_VAR allowed |
| SetForecasts | set the number of forecasts |
| Select | Select named variables into a selection group |
| SelectByIndex | Select variables into a selection group by index |
| SetMethod | set the estimation method |
| SetModelStatus** | set the model status (MS_....) |
| SetPrint | switch printing on or off |
| SetRecursive | set the number of recursive steps |
| SetResult** | set the estimation result |
| ShowBanner | static method to switch off creation banner |
| parameter related | |
| FixPar | fixes parameters |
| FreePar | frees parameters |
| GetFreePar | get the vector of free parameters, px1 |
| GetFreeParCount | get number of free parameters p |
| GetFreeParNames | get the names of the free parameters, array of length p |
| GetPar | get the vector of all parameters, qx1 |
| GetParCount | get number of parameters q (including fixed) |
| GetParNames* | get the names of all parameters, array of length q |
| GetParStatus | returns full parameter info |
| GetParTypes | returns array of type letters for each model variable |
| MapParToFree | return the free parameters from the argument |
| ResetFixedPar | reset the values of the fixed parameters |
| SetFreePar | set the free parameters |
| SetPar** | set the full parameter vector |
| SetParCount** | set the number of parameters q (including fixed) |
| move up in model status | |
| DoEstimation | low level estimate |
| Estimate | high level estimate |
| InitData | get the data: Y,X |
| InitPar | initializes the parameter values |
| SetStartPar | set the starting values |
| covariance evaluation | |
| Covar* | sets m_mCovar |
| get model results | |
| GetCovar | returns pxp covariance matrix |
| GetCovarRobust | returns <> or pxp robust covariance matrix |
| GetLogLik | return the log-likelihood, m_dLogLik |
| GetResVar | returns residual variance, nxn |
| GetResiduals | returns residual matrix, Txn |
| GetStdErr | returns the std.errors (0 for fixed) qx1 |
| GetStdErrRobust | returns <> or robust standard errors |
| GetcDfLoss | returns degrees of freedom lost (for tailt, AIC) |
| GetcT | returns actual no of variables to use in output, m_cT |
| other get model functions | |
| GetPrint | returns current print setting |
| GetX | returns X matrix |
| GetY | returns Y matrix |
| GetcX | returns no of X variables |
| GetcY | returns no of Y variables |
| GetcYlag | returns no of lags of Y |
| post estimation | |
| DbDrawTMatrix | draws using the database sample information |
| GetForecastData | returns available data over a forecast period |
| Output | prints output |
| OutputHeader | prints output header, returns TRUE to print rest |
| OutputPar | prints parameter estimates |
| OutputLogLik | prints log-likelihood, AIC, etc. |
| OutputMax | prints maximization result and starting values |
| PrintTestVal | prints a test statistic |
| TestRestrictions | tests restrictions on the parameters |
| OxPack related, separately documented | |
| GetLongRunInfo | returns 0 or info on long-run |
| GetLongRunNames | returns 0 or names of long-run parameters |
| ReceiveData | receive the data for estimation |
| ReceiveDialog | receive output from a dialog |
| ReceiveModel | receive the model specification |
| SendDialog | send a dialog |
| SendFunctions | send specification of special functions |
| SendMenu | send a menu list |
| SendMethods | send the estimation methods |
| SendResults | send an output variable |
| SendSpecials | send the names of special variables |
| SendVarStatus | send the types of variables |
virtual Modelbase::ClearEstimation() virtual Modelbase::ClearModel();
ClearEstimation() clears the model estimation settings. ClearModel() sets the model status to MS_NONE, and calls ClearEstimation().
virtual Modelbase::Covar();
In some models, the evaluation of the variance-covariance matrix of the estimated parameters is costly, therefore, this matrix is only computed on demand: when the covariance matrix does not yet exist, Covar() is called to compute it. By default, the m_mCovar member variable of Modelbase is -1 when estimation commences (through a call to ClearEstimation()). Covar() should set m_mCovar to the variance-covariance matrix (but Estimate() or DoEstimation() may also do this, as in the example above). Covar() can optionally set m_mCovarRobust as well. If the covariance matrix does not exist, Covar() is automatically called when using GetCovar(), GetCovarRobust(), GetStdErr(), or GetStdErrRobust(). This procedure ensures that the covariance is only computed once when required, and not at all when not required (in some Monte Carlo experiments, for example).
Modelbase::DbDrawTMatrix(const iArea, const mYt, const asY, const iT1);
Modelbase::DbDrawTMatrix(const iArea, const mYt, const asY, const iT1,
const iSymbol, const iIndex);
This is equivalent to DrawTMatrix, but using sample information from the underlying database. The function will automatically draw a proper date axis if the database is dated.
virtual Modelbase::DoEstimation(vPar); virtual Modelbase::Estimate();
DoEstimation() returns:
There are two ways to implement estimation:
Modelbase::Estimate()
{
decl vpstart, vpfree, estout;
if (!InitPar()) // calls InitData() if necessary
return FALSE;
vpstart = GetFreePar(); // map pars to estimation format
estout = DoEstimation(vpstart); // do the estimation
vpfree = isarray(estout) ? estout[0] : estout;
SetFreePar(vpfree);// map estimated pars to normal format
if (m_iResult >= MAX_CONV && m_iResult < MAX_MAXIT)
m_iModelStatus = MS_ESTIMATED;
else
m_iModelStatus = MS_EST_FAILED;
if (m_fPrint)
{ Output();
if (isarray(estout))
OutputMax(estout[1],m_iResult,vpstart,estout[2]);
}
return m_iModelStatus == MS_ESTIMATED;
}
Modelbase::FindGroup(const theGroup); Modelbase::FindMethod(const theMethod);
Returns an integer (an integer argument is passed through, a string looked up in GetGroupLabels, GetMethodsLabels).
Modelbase::FixPar(const iP, const dFix);
FixPar() is used to fix a parameter at the specified value. Subsequently, this parameter is omitted from the vector returned by GetFreePar().
Modelbase::FreePar(const iP)
Frees a parameter which was previously fixed by FixPar().
virtual Modelbase::GetcDfLoss();
Returns the loss in degrees of freedom in the estimated model. The default is the number of estimated parameters.
Only override this function if the number to be used in the output is different from that number of free parameters in the estimation.
virtual Modelbase::GetcT();
Returns an integer with the actual number of observations to be used in the output.
Only override this function if the number reported in the output is different from that used in the estimation (m_cT). ,,
Modelbase::GetcX(); Modelbase::GetcY(); Modelbase::GetcYlag();
Returns respectively: no of X variables, no of Y variables, lag length of Y.
Modelbase::GetCovar(); Modelbase::GetCovarRobust();
Returns the pxp variance-covariance matrix of the free parameters.
See Covar for an explanation of the implementation.
GetForecastData(const iGroup, const mnLag, const mxLag,
const cTforc);
GetForecastData(const iGroup, const mnLag, const mxLag,
const cTforc, const iT1forc);
Returns a matrix with the available forecasts data (or an empty matrix if no there is no data).
Modelbase::GetFreePar(); Modelbase::GetFreeParCount(); Modelbase::GetFreeParNames();
GetFreePar returns the px1 vector with free parameters.
GetFreeParCount returns the free parameter count p.
GetFreeParNames returns an array of length p with the names of the free parameters.
GetFreePar returns the current values of the free parameters. Parameters are fixed with FixPar(). The value of free parameters is set with SetFreePar().
Modelbase::GetLogLik();
Returns the log-likelihood, which is the value of the m_dLogLik member variable.
Modelbase::GetMethod(); virtual Modelbase::GetMethodLabel(); virtual Modelbase::GetMethodLabels(); virtual Modelbase::GetGroupLabels();
GetMethod returns an integer representing the estimation method, which is the value of the m_iMethod member variable.
GetMethodLabel returns the text label for the current estimation method m_iMethod.
GetMethodLabels returns an array with the text labels for the available estimation. methods.
GetGroupLabels returns an array with the text labels for the available status groups used for variable specification.
virtual Modelbase::GetModelLabel(); Modelbase::GetModelStatus();
GetModelLabel returns the text label for the model.
GetModelStatus returns the model estimation status:
| value | description |
|---|---|
| MS_NONE | no model preparatory action has been taken, |
| MS_DATA | estimation data has been extracted from the database, |
| MS_PARAMS | the starting values for estimation have been set, |
| MS_ESTIMATED | the model has been estimated, |
| MS_EST_FAILED | model estimation has failed. |
virtual Modelbase::GetPackageName();
Returns the name of the modelling package.
This virtual function should be overridden by the derived class.
virtual Modelbase::GetPackageVersion();
Returns the version number of the modelling package.
This virtual function should be overridden by the derived class.
Modelbase::GetPar(); Modelbase::GetParCount(); virtual Modelbase::GetParNames();
GetPar returns the qx1 vector with the current parameter values (both fixed and free).
GetParCount returns the total parameter count q (both fixed and free parameters).
GetParNames returns an array of length q with the names of the parameters.
GetParNames should be overridden to use proper labels in the output.
Modelbase::GetParStatus();
Returns array with:
This function is infrequently used.
virtual Modelbase::GetParTypes();
Override the default to return an array of strings indicating the type of each model variable, e.g. {"Y","X","X","U"}. The default returns 0, so that no types are indicated in the output.
This function is infrequently used.
Modelbase::GetPrint();
Returns the current print setting.
virtual Modelbase::GetResiduals();
Returns the Txn matrix with residuals (n equals 1 for univariate models).
Must be overridden by the derived class to return residuals.
Modelbase::GetResult();
The estimation result (normally a value from MaxBFGS), which is the value of the m_iResult member variable.
virtual Modelbase::GetResVar();
Returns the nxn matrix with the residual variance (n equals 1 for univariate models).
Must be overridden by the derived class.
Modelbase::GetStdErr(); Modelbase::GetStdErrRobust();
Returns the qx1 vector with standard errors (0 at position of fixed parameters).
See Covar for an explanation of the implementation. ,
Modelbase::GetX(); Modelbase::GetY();
Modelbase::Grow(const cTadd);
Calls Database::Grow and updates the deterministic variables ("Constant", "Trend", "Seasonal", "CSeasonal").
virtual Modelbase::InitData();
Returns TRUE if successful.
Extracts the data for estimation from the underlying database. Sets the model status to MS_DATA if successful.
virtual Modelbase::InitPar();
Returns TRUE if successful.
Gets starting values for the estimation procedure. Sets the model status to MS_PARAMS if successful.
virtual Modelbase::IsUnivariate();
Returns TRUE if only one dependent variable (Y_VAR) is allowed.
This virtual function should be overridden by the derived class if multivariate models are implemented.
Modelbase::MapParToFree(const vPar);
Returns a px1 vector with free parameter values.
Extracts and returns the free parameter values from a full parameter vector.
Modelbase::Modelbase();
virtual Modelbase::Output();
Modelbase::OutputHeader(const sTitle);
Called by Output to print the header section.
Modelbase::OutputLogLik();
Called by Output to print the loglikelihood and other summary statistics.
Modelbase::OutputMax(const sMethod, const iResult, const vPstart,
const bNumerical);
Called by Estimate to print the starting values and method used for iterative estimation.
Modelbase::OutputPar();
Called by Output to print the parameter estimates.
static PrintTestVal(const dTest, const cR, const cTdf, const sLabel);
Prints a test statistic and its significance. If cTdf is zero, the test is assumed to have Chi2(cR) distribution, otherwise an F(cR,cTdf) distribution.
Modelbase::ResetFixedPar();
Resets the fixed parameters to their prespecified values.
Modelbase::Select(const theGroup, const aSel);
Modelbase::SelectByIndex(const theGroup, const iVar,
const iLag0, const iLag1);
Selects variables by name and with specified lags, and assigns the theGroup number to the selection. The aSel argument is an array consisting of sequences of three values: name, start lag, end lag. After a sample period is set, the selection can be extracted from the database.
virtual SetForecasts(const cForc, const bIsLessForecasts);
The Modelbase version sets m_cTforc.
Modelbase::SetFreePar(const vParFree);
Modelbase::SetMethod(const theMethod);
Modelbase::SetModelStatus(const iModelStatus);
Modelbase::SetPar(const vPar); Modelbase::SetParCount(const cPar); Modelbase::SetParCount(const cPar, const bAdd);
SetParCount() must be called for the other parameter functions to work.
Modelbase::SetPrint(fPrint);
For Monte Carlo experiments, it can be useful to switch off printing.
virtual SetRecursive(const bSet, const cInit);
The Modelbase version sets m_bRecursive and m_cTinit.
Modelbase::SetResult(const iResult);
Sets the estimation result (normally a value from MaxBFGS), which is the value of the m_iResult member variable.
virtual Modelbase::SetStartPar(const vParFree);
This is an alternative to InitPar, allowing for direct setting of the starting parameters prior to estimation.
static ShowBanner(const bSet);
virtual Modelbase::TestRestrictions(vSel); virtual Modelbase::TestRestrictions(mR, const vR);
The one-argument version tests whether one or more coefficients
are zero. The second form tests restrictions of the type R\theta=r.
Both are implemented as a Wald test with a Chi2(s) distribution.
This function requires that Covar() and GetParNames() are implemented,
and SetPar() or SetParFree() are used to set the estimated
parameters.
The PcFiml class provides part of the advanced computations available in the menu driven computer program PcFiml. The class is derived from the Modelbase class, and provides model formulation using variable names. The class allows for estimating a Vector Autoregression (VAR), cointegration analysis (`Johansen procedure'), and multivariate regression model (such as an unrestricted reduced form, URF), as well as a simultaneous equations model (2SLS, 3SLS, FIML). No identities equations are currently possible. Mis-specification tests include: vector autoregression, vector normality, vector heteroscedasticity, vector portmanteau, as well as a Chow test.
The PcFimlDgp class is a data generation process (DGP), designed for use in dynamic econometric Monte Carlo experiments. Unlike the PcNaiveDgp class, it derives from Database to formulate the DGP and store the generated data. This makes the DGP more general, but somewhat more complex. The class is used through the header file pcfimldgp.h. The design is an n-variate version DGP:
The vectors yt and ut are n x 1, wt is k x 1 so that the coefficient matrix Pi is n x k. The zt vector is q x 1, making C0 q x q. The zs can be kept fixed between experiments, or regenerated for the experiment. A distribution for ut and vt can be specified.
PcFimlDgp::Asymp();
Prints an asymptotic analysis of the current DGP.
PcFimlDgp::Create(const iFreq, const iYear1, const iPeriod1,
const cTdiscard, const mxDgpLag, const mxT);
Creates the database. After this, Select may be used to formulate the DGP, with group identifier PcFimlDgp::Y_VAR or PcFimlDgp::Z_VAR. The database name of the variables are "Ya", "Yb", ..., and "Za", "Zb", .... The Constant, Trend and normal Seasonals are automatically created.
PcFimlDgp::DiscardZ();
Discards the current zt; the next call to GenerateTo() will generate new observations on zt.
PcFimlDgp::Generate(const cT);
GenerateTo returns generated (Y:U), as a T x 2n matrix.
Generates cT observation of the current DGP.
virtual PcFimlDgp::GenerateU(const cT); virtual PcFimlDgp::GenerateV(const cT); virtual PcFimlDgp::GenerateZ(const cT, const mC0t, const mV); virtual PcFimlDgp::GenerateY(const cT, const mPit, const mU); virtual PcFimlDgp::GenerateU_t(const iT); virtual PcFimlDgp::GenerateV_t(const iT); virtual PcFimlDgp::GenerateZ_t(const iT, const mC0t); virtual PcFimlDgp::GenerateY_t(const iT, const mPit);
These virtual functions are called by GenerateTo() to generate the data. The sub-t versions are called when using a for loop for data generation. This is the case after a call to UseObsLoop(TRUE).
PcFimlDgp::GetU(); PcFimlDgp::GetV(); PcFimlDgp::GetY(); PcFimlDgp::GetZ();
GetU returns current U as a T x n matrix.
GetV returns current V as a T x q matrix.
GetY returns current Y as a T x n matrix.
GetZ returns current Z as a T x q matrix.
PcFimlDgp::PcFimlDgp(const cY, const cZ);
Constructor.
PcFimlDgp::Prepare();
Virtual function which must be called prior to data generation.
PcFimlDgp::Print();
Prints the setup of the current DGP.
PcFimlDgp::SetDistribution(const iEqn, const iDist,
mPar1, mPar2);
See under RanPcNaive.
PcFimlDgp::SetFixedZ(const fSetting);
See under RanPcNaive.
PcFimlDgp::SetInit(const iDgp, const mInit);
Specifies initial values for the DGP (default is 0).
PcFimlDgp::SetYParameter(const mPit);
Sets the parameters for the yt equation.
PcFimlDgp::SetZParameter(const mC0);
Sets the parameters for the zt equation.
PcFimlDgp::UseObsLoop(const bUseObsLoop);
By default, the data are generated using matrix expressions. Use this to generate the data in a for loop. This is considerably slower, but gives more flexibility.
The PcNaiveDgp class is a data generation process (DGP), designed for use in dynamic econometric Monte Carlo experiments. The class is used through the header file ranpcnaive.oxh. The design is an n-variate version DGP: The class derives from the RanPcNaive class. Unlike RanPcNaive, the generated data are stored inside the object, and retrieved using Get functions.
PcNaiveDgp::DiscardZ();
Discards the current zt; the next call to Generate() will generate new observations on zt.
PcNaiveDgp::Generate(const cT); PcNaiveDgp::GenerateTo(const cT);
Generate returns generated Y = (y0 ... y_T)', as a T x n matrix.
GenerateTo returns generated (Y:U), as a T x 2n matrix.
Generates cT observation of the current DGP and stores it in the current object.
PcNaiveDgp::GenerateBreakTo(const cT, const iTbreak,
const iTreset, const mA0, const mA1, const mA2,
const mA3, const mA5)
Returns generated (Y:U), as a T x 2n matrix.
Generates cT observation of the current DGP and stores it in the current object, using the alternative DGP over the specified period.
PcNaiveDgp::GetU(); PcNaiveDgp::GetY(); PcNaiveDgp::GetZ();
GetU returns current U as a T x n matrix.
GetY returns current Y as a T x n matrix (as does Generate).
GetZ returns current Z as a T x q matrix.
PcNaiveDgp::PcNaiveDgp(const cY, const cZ);
Constructor.
The RanMC class provides random number generation of specific distribution for use by the RanPcNaive, PcNaiveDgp and PcFimlDgp classes. All member functions are static, and can be used without constructing an object, for example as:
x = RanMC::Choleski(x);
static RanMC::Choleski(const mSig);
Returns Choleski decomposition of mSig. mSig may have zeros on the diagonal. The corresponding rows and columns are ignored in the decomposition, and will be zero in the return value.
static RanMC::CheckDist(const sFunc, iDist, mPar1, mPar2);
Returns an array of three values:
static RanMC::RanDist(const iDist, const cT, const cY, const mDf1,
const mDf2);
Returns a T x n matrix of random numbers from the specified distribution. The distribution parameters must be as returned from CheckDist().
static RanMC::RanDist1(const iDist, const cY, const mDf1,
const mDf2, const mUlag, const mYlag);
Returns a 1 x n matrix of random numbers from the specified distribution. The distribution parameters must be as returned from CheckDist().
static RanMC::WriteDist(const sPar, const iDist, const mDf1,
const mDf2);
Writes the used distribution. The distribution parameters must be as returned from CheckDist().
The RanPcNaive class is a data generation process (DGP), designed for use in dynamic econometric Monte Carlo experiments. The class is used through the header file ranpcnaive.oxh. The design is an n-variate version DGP:
The vectors yt,ut,et are n x 1, so that the coefficient matrices A0,A1,B0,B1 are n x n, and a3 is n x 1. The zt vector is q x 1, making a2 n x q, C0 q x q, and c1,c2 q x 1. The zs can be kept fixed between experiments, or regenerated for the experiment. A distribution for et and vt can be specified.
RanPcNaive::Asymp();
Prints an asymptotic analysis of the current DGP: companion matrix with eigenvalues, together with cointegrating space and level of integration of DGP: I(0), I(1) or I(2). Discards the current zt; the next call to Generate() will generate new observations on zt.
RanPcNaive::GenerateTo(const cT);
GenerateTo returns an array of three elements, holding the
generated {Y,Z,U}.
Generates cT observation of the current DGP.
If Z is fixed, the fixed value is used, unless none has been set.
RanPcNaive::GenerateBreakTo(const cT, const iTbreak,
const iTreset, const mA0, const mA1, const mA2,
const mA3, const mA5)
Returns returns an array of three elements, holding the generated {Y,Z,U}.
Generates cT observation of the current DGP, using the alternative DGP over the specified period.
RanPcNaive::GetFixedZValue();
Returns the current fixed Z matrix.
RanPcNaive::HasFixedZ();
Returns current TRUE if Z is fixed.
RanPcNaive::RanPcNaive(const cY, const cZ);
RanPcNaive::Print();
Prints the setup of the current DGP.
RanPcNaive::SetDistribution(const iEqn, const iDist,
mPar1, mPar2);
Specifies the distribution for the u, or z equations given above. The first argument indicates the equation, the second the distribution. The last two arguments parameterize the distribution.
RanPcNaive::SetFixedZ(const fSetting);
Specifies whether zt is fixed (fSetting equals TRUE) or not, and clears the currently stored A new value for fixed zt can be set by SetFixedZValue or generated by SetNewFixedZValue.
RanPcNaive::SetFixedZValue(const mZ);
Sets a new value for fixed zt in the object.
RanPcNaive::SetInit(const iDgp, const mInit);
Specifies initial values for the DGP (default is 0).
RanPcNaive::SetNewFixedZValue(const cT);
Generates a new Z value and stores it in the object for subsequent use.
RanPcNaive::SetUParameter(const mLagAr, const mLagMa);
Sets the parameters for the et equation.
RanPcNaive::SetYParameter(const mA0, const mA1, const mA2, const mA3);
RanPcNaive::SetYParameter(const mA0, const mA1, const mA2, const mA3,
const mA5);
Sets the parameters for the yt equation.
RanPcNaive::SetYParameterEcm(const mAlpha,
const mBeta, const mA2, const mA3);
RanPcNaive::SetYParameterEcm(const mAlpha,
const mBeta, const mA2, const mA3, const mA5);
Sets the parameters for the yt equation in equilibrium correction form.
RanPcNaive::SetZParameter(const mC0, const mC1, const mC2);
Sets the parameters for the zt equation.
RanPcNaive::SetZCustom(mCZ);
Installs a custom Z. This is added to Z after generation of Z, but before Z is used in the Y equation.
RanPcNaive::StoreInDatabase(const amYZU, const oDb, const iY0,
const iZ0, const iU0, const cTDiscard)
Stores generated data in a database object. If the index is -1, the corresponding variable is not changed in the Database. It is assumed that the Y variables are in a consecutive block, similar for Z and U.
The Sample class stores a time interval, and the frequency, e.g. 1980 (1) -- 1990 (1), with frequency 4 (i.e. quarterly observations). The Sample class is only used to derive from. It requires linking in the database code, and inclusion of database.h. This is most easily done by adding an #import <database> statement at the top of your source code.
Sample::GetFrequency();
Sample::GetIndex(const iYear, const iPeriod);
The index of the specified time point.
Sample::GetPeriod1();
The period of the first observation.
Sample::GetPeriod2();
The period of the last observation.
Sample::GetSize();
The number of observations in the sample.
Sample::GetYear1();
The year of the first observation.
Sample::GetYear2();
The year of the last observation.
Sample::ObsPeriod(iObs);
The period of the observation index.
Sample::ObsYear(iObs);
The year of the observation index.
Sample::Resample(const iFreq, const iYear1, const iPeriod1);
Changes the frequency and start year(period). The sample size is unchanged, so the end year(period) is derived from that.
The Simulator class can be used to set up Monte Carlo experiments. Derive your own simulation experimentation class from this, overriding the virtual functions. Simulator will handle the replications and storage, and print the final results. The type of data it can handle are coefficients, test statistics and p-values of test statistics. The class is used through the header file simula.oxh.
A table is printed at the end, which gives the results, for coefficients:
where M is the number of replications and rmse is the root of the mean squared error.
Note that the functions are documented as belonging to the Simulator class, but are actually mostly in SimulatorBase. The source code of both classes can be found in ox/src.
Simulator succeeds the Simulation class which was used up to Ox 6 (this is still available through simula.oxo and simula.oxh, but now deprecated).
virtual Simulator::Generate(const iRep, const cT,
const mxT);
Upon failure, Generate should return an empty matrix or array, or an array wich has integer 0 as the first element.
Upon success, Generate should return an array with four elements:
If the call to the Generate function fails, additional experiments are run in an attempt to reach the required number of replications. The number of rejected replications is reported in the output.
The derived class must override the virtual function. It is called for every replication, and must perform the actual replication.
virtual Simulator::Plot(const iRep, const iT)
Virtual plot function. The default version does nothing.
virtual Simulator::Prepare(const cT, const mxT);
Virtual function which the derived class can override if necessary. It is called just before the replications for sample size cT commence. It can be used to initialize common regressors (e.g.), and is not run in a parallel section.
Simulator::SaveIn7(const sFilename); Simulator::SaveRecIn7(const sFilename);
Saves simulation results to the named file.
SaveIn7 stores the test and coefficient values (use SetStore(TRUE); before running the experiment).
SaveRecIn7 stores: coefficients, MCSE, Bias, RMSE,
test critical values (right tail), rejection frequencies and moments (use SetRecursive(TRUE); before running the experiment).
Simulator::SetCoefNames(const asNames) Simulator::SetTestNames(const asNames)
Installs the names of tests statistics and coefficients, to determine dimensions of the collected information, and to make the report more readable.
Simulator::SetPlotRep(const iPlotRep); Simulator::SetRecursive(const bRecursive); Simulator::SetStore(const bStore);
Simulator::SetTwoSided(const mIsTwoSided);
Should be called before Prepare is called to indicate which tests are two-sided. Need not be called if all tests are one-sided. The value of st is derived from the call to SetTestNames.
Simulator::Simulate()
This is the core function. It runs the Monte Carlo experiment, and prints the results.
Simulator::Simulator(const mT, const mxT,
const cRep, const fCommon, const dSeed,
const mPvalue, const mTrueParam);
Constructor function. Calls to SetCoefNames (if coefficients are generated) and/or SetTestNames (if p-values or test statistics are generated) are also required.
Ox version 7.00. © JA Doornik This file last changed .