Ox 7: new features and fixes

New features and enhancements in version 7.0
Fixed problems in version 7.0

The Ox version number is kept in line with the OxMetrics front-end version.

Overview of new features version 7.0

Dynamic link libraries and oxo files need to be recompiled for Ox 7!

Multithreaded computation is only supported by Ox Professional

New syntax
- foreach loops
- Lambda functions (anonymous functions)
Input handling
- Open a zip file for reading. The following file functions can handle the opened zip file: fread, fscan, feof, fsize.
- Database class handles choice variables
- replace: replace text in string
- loadsheet: loads an entire sheet from xlsx file (can be any sheet, not just the first)
- Improved handling of files >2GB
Efficiency
- Improved handling of function arguments that are not labelled as const.
- Faster virtual function calls
- Lower overhead of sub matrix assignment
- Compilation of canonical for and foreach loops
Parallel programming
- Parallel loops
- Parallel simulation using Simulator class
Developer
- Renamed Appendices to Developer's manual. Created C# and Java examples.
- Java version of RanApp (usung NetBeans)

New features and enhancements in version 7.0

New syntax
- extended dot-relational and dot-equality operators for strings. Eg.
- Added [=] lambda functions
- Added foreach loop
- Added parallel for and parallel foreach loops, and serial keyword to synchronize thread access (Professional). See the new chapter in the Ox book.
- Added default values for function arguments.
- There are three new keywords: foreach, parallel and serial. Any existing code that uses these as variable names will have to be modified to use slightly different names.
- Now using .oxh as default header extension. This allows us to distinguish Ox header files from C/C++ header files. Of course, .h is still allowed. For backward compatibility, include <file.h> will try .oxh first, then, if that fails, the .h file.
- Changed meaning of a = b = c; Now leaving c on the stack.
  Previously the documentation said: `The type of an assignment is that of its left operand.' However, this is impossible, because the left-hand side in Ox may not have a type. This change is only noticeable when using matrix indices, e.g. a = b[4][] = c, and is highly unlikely to affect existing code.
- Handling of const improved:
  - Taking reference of a constant (except const argument) is now disallowed (more useful compile-time error instead of possible run-time error).
  - Function arguments are now treated as const by default (and automatically adjusted internally to non-const if required and allowed).
    So it is not necessary anymore to use const function arguments for efficiency. It is still good programming practice (but at the same time I found this a difficult feature when teaching Ox to beginners.)
New functions
- clone to clone a class object
- ftime function
- getfolders
- loadsheet to load a sheet from a xlsx file into a two-dimensional array. Can also load a .csv file.
- maxr, minr for symmetry with maxc, minc
- nans() as in zeros, ones
- Num1Derivative_parallel
- replace and find
- ranindex: same as ranshuffle(c, range(0,c-1))
- ranloopseed, clone (to clone a class object)
- union function for symmetry with intersection
- binxor, binpop and binvec
- diff and lag which default to missing values instead of zero for the gaps
Changes to existing functions
- DrawXYZ was plotting Y,X,Z instead of X,Y,Z
  This change affects existing code, and calls to DrawXYZ should be adjusted. If you write in your Ox 6 code:
```
		
			DrawXYZ(0, x, y, z)
```
  this should be changed to
```
		
			DrawXYZ(0, y, x, z')
```
  to get exactly the same graph.
- limits(zeros(0,1)) crashes
- DrawMatrix, DrawTMatrix, DrawXMatrix: symbol and index arguments can be a vector with a value for each row in the data matrix.
- fopen: Added v (print message if fails) and V (message and run-time error if fails) to fopen format.
- fread, fscan: Read from zip files
- fsize for files >2GB: now returns double
- ismissing and isdotmissing now also treat undefined (.Null) as a missing value (see ox/samples/inout/iotest13.ox)
- main: return value from main now returned to console
- Maximization functions MaxSQP(F) and SolveNLE are now fully reentrant.
- Maximization functions can use CMaxControl object (last argument) as an alternative for setting arguments, and retrieving the iteration count.
- (f)print(ln): Always printing time with "%C" using seconds: 11:00:00 instead of 11:00
- (f)print(ln): Now printing {} for empty array (as <> for matrix}
- print(ln): Default line length now 104
- string(func) returns string with name of function func, string(object) returns class name.
- vecindex/vecrindex is faster for very large vectors
- all bin.. functions now allow vector arguments
- setlower can have a vector for the second argument. In that case the lower diagonal is filled by column (from left to right) with the elements of the vector (until the vector runs out). This amounts to setting the strict lower diagonal to unvech of the vector.
- loadmat("file.xlsx", 0) did not load strings as missing values
- Getting the Julian date index of today is quite tedious: int(timing(today(), 2)).
  Added dayofcalendar() and timeofday() for current values
- Increased default sprint buffer from 2K to 16K
Classes
- Database class:
  - support for DB_CHOICE (choice variables), and related new member functions: SetVarChoicesByIndex, SetVarChoices, GetVarChoicesByIndex, GetVarChoices, GetObsLabel, GetVarType, SetVarType
  - GetSize returns 0 when empty
  - Create with one argument (size of database)
  - GetVarCount, IsEmpty, Shrink
  - Recode, Tabulate, SortBy
  - Much faster Database::Renew when the variables are new to the database
- Sample: Resample(const iFreq, const iYear1, const iPeriod1)
- Modelbase: GetPrint, GetcY, GetcX, GetcYlag, GetX, GetY
- Simulator class, which is a new version of the Simulation class (still available). The Simulator class defaults to parallel mode, requiring the Generate function to be useable in parallel.
- Split PcNaiveDgp into RanPcNaive, which has the generation facilities without storing the results internally, and PcNaiveDgp, which derives from this and stores the results internally. Both use header file ranpcnaive.oxh.
Other changes
- Define OX_PARALLEL to indicate that the parallel and serial keywords are defined.
- Handle .in7 and .fmt files that are larger than 2GB data (64-bit version). Avoiding need for transpose in saving of in7/bn7. Allow spaces in variable names when saving as in7 file.
- Member call code refactoring: faster object/virtual calls.
- Because of internal changes, all dynamic-link libraries and .oxo files must be recompiled.
- Changed oxo version
- Better multi-threaded tuning of some large matrix multiplications (Professional)
- Switching to dynamic linking of openmp (Professional): libiomp5md.dll to avoid warning message when using MKL kernel replacement.
- Indexing a vector with <> (single index): the vector has zero elements, and now it is created as 0 x 0 matrix instead of 1 x 0 or 0 x 1.
- New exported functions: OxStoreCreate, OxStoreDelete, OxValGetBlob, OxValGetVecc, OxValGetVecr, OxValSetArray, OxValSetBlob, OxValSetMat, OxValSetMatZero, OxValSetVecc, OxValSetVecr, OxValGetStaticObject, OxValGetClassName, OxValDuplicate, OxValTransfer, OxCloneObject, FOxLibAddFunctionEx
- Added #pragma _ox_stdlib(1)/#pragma _ox_stdlib(0) (for oxstd.oxh only) so that FNI optimizations and arraybase adjustments are only done for internal ox stdlib functions.
- Improved decision on when to start blocking matrix multiplication, giving good speed improvements in some cases (and marginal slow downs in a few others).
- Improved handling of import when it uses a relative path; solved crash in matrix constant when syntax error
- Spreadsheet data: slightly better at recognizing fixed frequency when using calendar dates. Improved handling of variables that mix labels and values: choice variable now constructed such that values are not reused. .xls now only load an unlabelled first column if it contains values other than dates.

Fixed problems in version 7.0:

fixed local declaration issue, e.g.: Now i3 will be undefined, and the memory for i2 cleared when the local block is exited.
Print model for pdf/postscript was still not correctly set when saving within Ox Console. Saving graphs as pdf/eps now obeys the print colour mode i as set by DrawAdjust(ADJ_COLORMODEL, -1, i);
fopen("x", "wa") crashes: added better argument check.
Cannot use class-based constant in constant expression.
crash if trying to saving empty Database
Database::Save for Stata file doesn't work.
fseek/fsize (and other file functions) with large files in 64-bit version.
Reading of field with time component in .csv: not recognized as time variable
vector indexed by vector could result in memory leak
dropr(array, <>) should return the whole array, not an empty array.
isdotfeq and isfeq now treat .NaN as the == and .== operators.
tailf(0, ...) was 1, but tailf(almost 0, ...) was 0

This file last changed .