The Diagnostic log

This topic is about the Diagnostic log, the information it provides, and how to customize the settings.

In the Log settings page, you can see a table view of all the log options that you can enable or disable. On top of the table you can set the verbosity level for the Diagnostic log. Board provides a wide range of logging capabilities in order to comply with industry auditing rules and regulations, and to provide system administrators and maintenance teams with the information they require for troubleshooting, incident management, and disaster recovery.

The Diagnostic log can be configured to adjust its verbosity level so that debugging will be a lot easier, given the amount of information it can include.
To adjust the verbosity level, navigate to the dropdown menu at the top of the table and select your preferred option.

The diagnostic log has the following format: diagnosticYYYYMM_tsv.log
Rolled log files have the following format: diagnosticYYYYMM_tsv.archive###.log

After "archive" in the diagnostic rolled log file, an incremental value (indicated in the format as ###)  is added as files are rolled (001, 002, etc.)

Diagnostic log Verbosity levels

The available verbosity levels of the diagnostic log are:

• Off. The Diagnostic Log is turned off

• Error. Returns the same amount of information included in the previous Exception log

• Warning. Returns the same amount of information as the Error level plus exceptions with no impact on the user

• Information (default). Returns the same amount of information as the Warning level plus additional information about non-error actions, such as Data model loading, Dataflow executions, Data Reader executions, Data Entry actions.
  At this level, the Diagnostic log file also includes lines written in all other Board log files. Entries generated by the same request have the same "TaskId" value, while entries belonging to the same Procedure have the same "CorrelationId" value

• Debug. The highest verbosity level: includes the same amount of information as the Information level plus information about the start and end of each task

Each log line will always include the following information: timestamp, error level, action, TaskId, user, CorrelationId, additional details specific for each action. Log files are rolled at the end of each month or when they reach a file size of 50 Mb, whichever comes first. Only the last 10 rolled files are kept. Log files older than two months are automatically deleted.

In case of errors, for more details on a REST API call step configuration: at verbosity level "Information", when the remote server returns an error, the corresponding log entry will also include the URL of the request sent, the method, the Headers, and the Body (if applicable).

Diagnostic log information and format

If you are looking for logs regarding diagnostics and general system health, contact Board Support for more information.

The format of the logged information is in TSV (tab-separated values) with column headers in the first row.

If a request received by the engine is not from the Board web client - as it is the case with Board's Office Add-ins -, the corresponding entry includes a specific property (application_ID) next to the triggered engine event.

Example:

2022-12-07 11:02:54.129 +01:00  INFO    Administrator           5761e6aa-e6ae-4078-bbd2-53c36067d81a    NET_Connect [OFFICE_ADDIN]                           Starting action.

Diagnostic log Entries and format

The Diagnostic log has the following entries:

• Log entries regarding the Layout include information, as described below:

  • The Layout start event entry includes client filters (Active selection, Pagers definitions), the Layout definition (Blocks and Entities in the axes), and Block and Axes Properties. Block and Axes Properties are logged only if their settings are different from the default settings.

    Example:

    2022-12-02 12:12:20.361 +01:00    INFO     Administrator            3a43b5b9-a022-4a8a-b29a-268ccce078d9    NET_LayoutToSpread    DataEntry    Log.bcps    Home        {"ClientFilters":{"Selections":[{"Entity":"E_000201 D011 Customer", "Members":["1 C1","2 C2","3 C3"], "Mode":"Keep"},{"Entity":"E_000007 A007 Day", "Members":["1 20200101","2 20200102","3 20200103","4 20200104","5 20200105","6 20200106","7 20200107","8 20200108","9 20200109","10 20200110","11 20200111","12 20200112","13 20200113","14 20200114","15 20200115","16 20200116"], "Mode":"Keep"},{"Entity":"E_000206 D016 Custom Sort", "Members":["3 c","4 d"], "Mode":"Exclude"}], "Pages":[{"Entity":"E_000202 D012 Product", "Members":["3 P3"], "Mode":"Keep"}]}, "CoreLayout":{"ByRow":["E_000001 A001 Month","E_000204 D014 Country","E_000201 D011 Customer"], "ByCol":["E_000202 D012 Product"], "Blocks":[{"Letter":"a", "Cube":"V0001 Gross Sales", "DataEntry":{"SplitSplat":false}},{"Letter":"b", "Cube":"V0009 Gross Sales V4", "ReferTo":[{"Entity":"E_000006 A006 Year", "Members":["2020"], "Mode":"Keep"}], "TotalByEntities":"E_000203 D013 City, E_000006 A006 Year"},{"Letter":"c", "Cube":"V0001 Gross Sales", "TimeFunction":{"BlockFunction":"Value", "PreviousYear":true, "PeriodOffset":100, "PeriodCycle":200}},{"Letter":"d", "Cube":"V0009 Gross Sales V4", "Rule":{"Title":"N.A. 1"}, "RollupCube":true},{"Letter":"e", "Algo":"a + 100", "Summary":"Calculated"},{"Letter":"f", "RowTotals":false, "Rule":{"Title":"N.A. 1"}},{"Letter":"g", "Entity":"E_000006 A006 Year", "RowTotals":false, "TooltipBy":"a", "DetailBy":{"Entity":"E_000001 A001 Month", "KeepTop":12, "Sort":"Descending"}},{"Letter":"h", "RankingFunction":"Counter", "Summary":"Calculated", "RowTotals":false, "Aggregation":{"Mode":"AVG", "Entity":"E_000008 A008 Week"}},{"Letter":"i", "Cube":"V0001 Gross Sales", "AnalyticalFunction":{"Function":"Nascency", "TimeEntity":"E_000001 A001 Month", "From":0, "To":0, "IgnoreLastPeriod":false}},{"Letter":"j", "Cube":"V0001 Gross Sales", "DataEntry":{"SplitSplat":false, "PatternBasedAllocation":{"Cube":"V0009 Gross Sales V4", "PreviousYear":true}, "LockedByBlock":{"LockerBlock":"c", "LockIsZeroMode":true}, "LockByCube":{"LockerCube":"V0002 Rolap Single", "DisplayBothLockAndFree":true, "LockNotZeroMode":true}}},{"Letter":"k", "Cube":"V0001 Gross Sales", "AlertBy":"a", "DataEntry":{"SplitSplat":false}, "Nexel":{"Mode":"SingleRule", "Formula":"=100", "TotalFormula":"=10000"}}], "DifferentiateZeroNull":true}}        Layout started.

  • The Layout end event entry now includes the number of rows, columns, and cells generated as a response to the Layout query.

    Example:
    2022-12-02 12:12:21.954 +01:00    INFO     Administrator            3a43b5b9-a022-4a8a-b29a-268ccce078d9    NET_LayoutToSpread    DataEntry    Log.bcps    Home        {"Rows":9, "Columns":20, "Quota":180}        Layout completed.

• Log entries regarding data entry events include information, such as client filters (Active selection, Pagers definitions), the Layout definition (Blocks and Entities in the axes), and Block and Axes Properties. These entries also include the following details specific to data entry actions:

  • Position of the cell where the data entry action occurred (row number, column number, cell coordinates)

  • Entered value

  • Previous value

  • The header of the Block affected by the data entry action, as shown in the Layout editor (i.e. "a", "b", "c", etc.)

    Example:
    2022-11-30 11:20:15.892 +01:00    INFO     Administrator            6aab0bfb-4bb9-4cef-95d7-6f19cb9a28e4    NET_DataEntry    DataEntry    Log.bcps    Dataentry        {"ClientFilters":{"Selections":[], "Pages":[]}, "CoreLayout":{"ByRow":["E_000001 A001 Month","E_000202 D012 Product","E_000201 D011 Customer"], "Blocks":[{"Letter":"a", "Cube":"V0001 Gross Sales", "DataEntry":{"SplitSplat":true}, "Nexel":{"Mode":"SingleRule"}}], "RowShowAll":true}, "CellChanged":[{"Row":11, "Col":2, "OldValue":300, "NewValue":3000, "Coordinates":"1 202001, 2 P2, 2 C2", "Block":"a"}]}

• Log entries regarding Data readers now include additional information, as described below:

  • The Data reader start event entry now includes the protocol name and the path to the file that should be read (only for text file Data readers)

    Example:
    2022-12-02 11:38:32.933 +01:00    INFO     537bd1ed-e8be-40a6-b6aa-92e185571ad0            d8a4898d-1ac4-440c-ae03-b9940afb5996    DataRead    sync                {"ClientFilters":{"Selections":[], "Pages":[]}, "DataReader":{"Title":"SyncToDataModelTemporary", "Path":"C:\\Windows\\TEMP\\tmp2117.tmp"}}        Data reader started.

  • The Data reader end event entry now includes the number of successfully processed rows, discarded rows, and the error message (if any)

    Example:
    2022-10-25 12:40:18.241 +02:00    INFO     Administrator    537bd1ed-e8be-40a6-b6aa-92e185571ad0    DataRead    ce788c9f-bf3d-48ef-938f-12253ae40704    ASCII Data Read    Echo                {"Stats":{"ReadRecords":2, "DiscardedRecords":5, "ErrorMessage":"Error : RBT1825 New Item Found on column 1 [Joan \"the bone\", Anne]"}}        Data reader ended.

• Log entries regarding Dataflows now include additional information, as described below:

  • The Dataflow start event entry now includes the client filters (Active selection, Pagers definitions) sent with the Procedure step.

    Example:
    2022-11-25 17:47:03.887 +01:00    INFO     Administrator    e03b5b6a-d11e-4cb2-b2f9-26d1862a8a82    dataflow    68dea0cd-a9cc-4fa7-8fc7-d3b0037f4530    3. Data flow    Echo                {"ClientFilters":{"Selections":[{"Entity":"E_000001 A001 Month", "Members":["1 201001"], "Mode":"Keep"}], "Pages":[]}}        Dataflow started.

  • The Dataflow end event entry now includes various additional information about its configuration, such as  the target Cube, the algorithm used for the calculation, the details of the "extend" option configured by the user, the details of the "extend" option applied by the system, and others.

    Example:
    2022-11-25 17:47:06.140 +01:00    INFO     Administrator    e03b5b6a-d11e-4cb2-b2f9-26d1862a8a82    dataflow    68dea0cd-a9cc-4fa7-8fc7-d3b0037f4530    3. Data flow    Echo                {"DataFlow":{"Target":"V156 Sales pacl", "ContainingPlus":true, "Algo":"Tuples.[Union].InnerOuter", "UserExtend":"E_000001 A001 Month, E_000011 D011 Customer", "InternalExtend":"E_000017 D017 Product", "RollOver":"V226 Gross Sales", "BackCompatible":false}}        Dataflow ended.

  • If a Cube configured in the "Limit calculation to tuples of the Cube" option is not found when the step is executed, the Procedure will fail and a specific error message will be displayed. This may happen, for example, when the Cube is configured in the "Limit calculation to tuples of the Cube" option and that same Cube is later deleted from the Data model.

    Example:
    2023-03-22 10:47:03.058 +01:00  WARN  Administrator  f04dfa9d-4e0f-427e-856b-eeb527813ea2  Rollover  7d6e289a-f71a-4094-aa00-81f3952d3e6b  3. Data flow  Echo  Unable to find rollover cube. Rollover cube will be ignored. Dataflow revert to default calculation domain option

• Log entries regarding Procedure steps (start event, end event, additional action events), now include the step number as it is displayed in the Procedure designer.

  Example:
  2022-12-02 10:38:28.011 +01:00  INFO    Administrator   1e4b6e48-0672-4cb4-a7f2-69901d767b42    select based on 27b14850-6886-4110-bf79-8f8a09fb70ea    5. Data flow    DP              {"ClientFilters":{"Selections":[{"Entity":"E_000036 D036 Currency", "Members":[], "Mode":"Invalid"},{"Entity":"E_000027 D027 Merch_Zone", "Members":["10 10","11 11"], "Mode":"Keep"}], "Pages":[]}}      Dataflow started. (bold added)

• If one or more dimensions of a Cube are missing in the Data reader mapping configuration, a new specific log entry is created for each missing dimension in the Diagnostic Log set to verbosity level "Warning".

  The Log entry would look like the following example:
  2023-03-16 12:44:12.931 +01:00 WARN Administrator d6f45ad2-4359-4963-9467-6892b8b8bf9f Import in Gross Sales a8db4810-f525-4993-b369-3a4ea12e2d5a 2. ASCII Data Read Echo {"Cube":"V001 Gross Sales", "MissingEntityIdx":3} Cube dimension is missing in mappings

If a log entry mentions a Data model resource (Entities, Entity members, Cubes, and Sparsities), the reference to the corresponding resource is now presented as described below:

• Entities → "{Physical name} {Internal identifier} {Extended Name}".

  Example: "E_000036 D036 Currency"

• Entity members → "{Internal identifier} {Code}".

  Example: "10 202212"

• Cubes → "{Physical name}  {Name}".

  Example: "V156 Sales EMEA"

• Sparsities → "{Sparsity code} {Internal identifier}".

  Example: "SP001 D0014#"

If one of the resources mentioned above cannot be found, the related information in the corresponding entry is logged as follows: "N.A. {Internal identifier}".

Example: "N.A. 13"