Changes to the SmartDB and SmartDB upgrade and migration utility

We are rolling out a few changes to the SmartDB schema (logical and physical) and changes to the SmartDB Upgrade and Migration Routine that require your attention.

Dependency on PCT release 208

With the release Release 11.3 #43810@2017-10-22 of the 22nd October 2017 e're using the quickRequest argument of the PCTRun ANT task (-q parameter). This is one of the actions taken in SCL-1902 to improve the performance of the SmartDB Upgrade and Migration Routine. This parameter has only been made available in the (currently) most recent release of PCT. We're providing this PCT release in the Setup/Install/PCT folder of the SmartComponent Library release. However you may need to change the startup parameters used to invoke the upgrade utility on your end if you haven't copied that release to your ANT/lib folder. 

We recommend that you add the

-lib Install/PCT/PCT.jar

startup parameter to your command line.

Missing to use the most recent PCT release, may cause the following error:

BUILD FAILED
setup.xml:223: PCTRun doesn't support the "quickRequest" attribute

Your action may be required: Correcting the codepage of some of our CLOB fields

In SCL-1920 we have corrected the code page of some of our CLOB fields to UTF-8. As most of these CLOB fields are used to store JSON data they should have been created as UTF-8 from the beginning. This is also the only viable code page for CLOB's in an internationalization scenario.

CLOB's affected by this change are:

Table NameField Name
SmartContextStoreContextDataset
SmartProductProductDescription
SmartSchedulerJobJobCommand
SmartSchedulerJobPlanJobParamter
SmartSchedulerJobPlanContextDataSet
SmartSchedulerJobStatusJobParamter
SmartSchedulerJobStatusContextDataSet

If you are on a pretty recent release of the SmartDB schema, there's nothing to worry about - although as always we recommend you create a backup of your SmartDB first!

The schema update process, will first add new CLOB fields with UTF-8 codepage. Then a .p file will copy the data from the current ISO8859-1 codepage'd fields to the new ones. Finally we'll rename the old fields to an temporary name and the new fields to the original name. The remaining update process will then drop the temporary fields.

If your SmartDB has not been updated since February 2016 your schema might not yet contain the SmartContextStore table. If your SmartDB has not been updated since May 2016 your schema might not yet contain the Scheduler related tables. The SmartProduct table we added in April 2014 - we're not expecting that any of our customers has not yet updated since then. 

However, the conversion routines will fail, if any of those tables are missing. Since the conversion, will upgrade the SmartDB to the latest schema anyway, we require that our customers to load the required tables by applying any of the required delta.df files manually before running the SmartDB Upgrade and Migration Routine:

Implementing the basic storage area CCS spec

Following the recommendations of our partner White Star Software (WSS) we have added the basic storage area layout recommended by the CCS Basic Storage Area Specification (draft) to the SmartDB. This will basically add the following three area to the SmartDB:

  • Data
  • Index
  • LOB

It will further add storage areas to prepare the SmartDB for auditing by adding audit-areas:

SmartDB.st
#
b .
#
d "Schema Area":6,32;1 .
#
a .
a .
a .
a . 
# 
d "Data":42,128;8 .
d "Index":43,128;64 .
d "LOB":44,128;8 .

It is not mandatory to add those storage areas to your SmartDB - but recommended. When your SmartDB does not (yet) have those areas, we'll remove any storage area information from the delta.df files we'll apply to your database and tables will be added to the "Schema Area". (which is not even recommended by Progress Software).

With typical sizes of the SmartDB database, dumping your SmartDB and recreating it into a new database created with the structure above is certainly the simplest migration process.

Please contact us if you have any questions regarding the update process!