Improving performance when using large files |
GS-Base enables you to save all the automatic search and sort indexes created when filtering tables
and when recalculating calculated formulas, especially with various inter-table aggregation formulas that
use binary searches and require such sort indexes.
This vastly improves how you use large databases because if the indexes are not saved:
(1) opening a database with active filters required waiting till the filtering is performed for the last used filter set,
(2) after opening a database, performing the first inter-table calculation with various forms of lookup
formulas required re-creating the internal sort indexes to enable binary searches. (The displayed "progress"
dialog box in the status field first shows "sorting" and much later the "calculating" status.)
Whenever you are opening a database both (1) and (2) could increase the actual total opening time
by tens of seconds for large (e.g. a few tens of millions of rows or more and multiple regex filters) tables.
If you check the two "Save indexes(...)" options in the "Options > Settings" dialog box, both the (1) and (2) steps will be skipped and the databases will be displayed instantly after the file is read.
Technically, for each table you can save up 16,384 indexes in three versions each (depending on the sorting/language parameters, like both the case-sensitive and case-insensitive versions) and each one for up to 256 million records. However, the practical limit will be lower due to the RAM/paging limitations as one index entry (one record/row) uses 4 bytes.
When entering multiple (several, more then ten etc.) field filters for very large tables, turn off automatic
filtering till you enter all filtering expressions. You can do this by unchecking the the "Search Now" check box
in the "Field Search Filter" dialog box. In this case to finally apply all entered filters use
the "Tools > Search All Now" menu command.
Otherwise accepting each subsequent filter will be triggering
repetitive filtering with expressions that you have entered so far.
If you're performing large data block operations within tables (e.g. transforming/converting entire table columns, pasting entire columns), you may want to limit the "Undo" level in the "Options > Settings" dialog box. Otherwise the "Undo" buffer will hold large amounts of the "Undo" data. (Of course, if your amount of RAM is respectively much larger than this, it won't matter.
If your database contains a very large numbers of memo/image/files/code objects (e.g. tens of thousands), you should always use the *.gsb file extension for saved GS-Base native databases, not the *.zip extension, because otherwise Windows will be trying to index any found zip file and with thousands of zip substreams saved by GS-Base, the Windows system won't be able to process such structures and might become unresponsive for minutes to hours or more.