Saturday, June 6, 2020

Expanding the Stock Severity Levels

Original Severity Levels

The original Pantheios C/C++ library defined eight severity levels (and their integral values and string forms), as follows:


Manifest constant Integral value String form (1)
PANTHEIOS_SEV_EMERGENCY 0 "Emergency"
PANTHEIOS_SEV_ALERT 1 "Alert"
PANTHEIOS_SEV_CRITICAL 2 "Critical"
PANTHEIOS_SEV_ERROR 3 "Error"
PANTHEIOS_SEV_WARNING 4 "Warning"
PANTHEIOS_SEV_NOTICE 5 "Notice"
PANTHEIOS_SEV_INFORMATIONAL 6 "Informational"
PANTHEIOS_SEV_DEBUG 7 "Debug"

1. as obtained by pantheios_getStockSeverityString()

For anyone familiar with Syslog, these level names and values will be very familiar, as they exactly match:


Manifest constantIntegral value
Emergency0
Alert1
Critical2
Error3
Warning4
Notice5
Informational   6
Debug7

Experiences and Evolution

However, over the decade and a half of use of Pantheios (and its other language bindings - .NET, Ruby, etc.) experience and further theoretical work has raised a number of issues, as discussed throughout the rest of this post.

"Error" -> "Failure"

As discussed in a 2013 instalment of Quality Matters in ACCU's Overload, the term error is fraught with ambiguity: almost the whole industry uses the term incorrectly to mean a failure - "throw an error" - whereas an error is something that a human does in designing or building a system.

So, we've started to use the term failure instead. Newer Pantheios projects, such as Pantheios.Ruby, already make use of this term, and the extant projects will evolve in that direction over the coming months.

"Emergency" -> "Violation"

Just as the term error is ambiguous so is the term bug. The issues around both terms are dealt with more formally in Contract Programming / Design by Contract, which is beyond the scope of this post, but you can read up on it in Meyer's Object-Oriented Software Construction or Wilson's Imperfect C++.

Contract Programming makes a formal distinction between a failure of a system and a violation of a software contract. When a contract violation occurs the program has behaved in contradiction to its design so by definition it cannot "correct". The only sensible course of action in this case is to (attempt to) terminate the process, yielding control to the operating environment in the hope that the process <-> operating system abstraction can handle the situation.

Over the lifetime of Pantheios we have taken to using the Emergency severity level exclusively for contract violations, and are starting to use the name Violation instead. Again, newer projects already have this, and we're evolving others.

Later in this post we will discuss the meanings we ascribe to the severity levels.

More debug levels

Extensive use in recent years of the Pantheios C/C++ library into sophisticated commercial C++ codebases with precise runtime controls over enabling/disabling severity levels has taught us that having a single debug level is insufficient. For example, when handling network traffic it is useful to be able to control the logging of the number of bytes received on a channel separate to the logging of all of those bytes.

As such, we've tended to have a number of debugging levels: Debug0Debug1, and so on, where each level involves deeper call levels and/or more extensive logged information. In the commercial projects we've provided custom severities with such additional levels, but plan to evolve the main Pantheios project along these lines soon.

"Trace"

There is one (new) particular debugging level that stands out from the others: Trace. This is useful for tracing function/method entry (including arguments received) and exit. Because tracing can lead to heavy amounts of logging, we have found it useful to give it a separate debugging level.

Again, Pantheios.Ruby has this level already, and other projects will be getting it soon. The current tracing facilities provided by the Pantheios C/C++ library defined in pantheios/trace.h  will be adjusted to use the new severity level.

"Benchmark"

Finally, there is one (new) level that has emerged, for the exclusive purpose of benchmarking.

Pantheios.X 202x levels

Hence, what we might call the 202x levels for all Pantheios projects has emerged, as follows:

Level name String form Former name Integral value
Violation "Violation" Emergency 1
Alert "Alert" Alert 2
Critical "Critical" Critical 3
Failure "Failure" Error 4
Warning "Warning" Warning 5
Notice "Notice" Notice 6
Informational "Informational" Informational 7
Debug0 "Debug-0" Debug 8
Debug1 "Debug-1" - 9
Debug2 "Debug-2" - 10
Debug3 "Debug-3" - 11
Debug4 "Debug-4" - 12
Debug5 "Debug-5" - 13
Trace "Trace" - 14
Benchmark "Benchmark" - 15


These levels already exist within Pantheios.Ruby and will be included in the next version of the Pantheios C/C++ library (albeit with a compile-time option to use the prior levels) and in the next release of Pantheios.NET. (Each of these is likely to be within a month or two of this post.)

Value changes

As well as the addition of new levels, and the renaming of two extant levels, you may also note that the values of the levels corresponding to the original eight have changed. Specifically, they are no longer 0-based, but are now 1-based. The reasons for this are:

  • to help catch programming errors in some languages where a default-initialised level variable/parameter would be interpreted as Violation(/Emergency); and
  • to allow sophisticated front-ends to have an "all levels" value to facilitate group control. (We'll discuss this more fully in a future post.)
Note that all levels still fall within 4-bits, so the extended severity level information mechanism is still supported by the the Pantheios C/C++ library. (See this post about extended severity information. Also, the next release will include C and C++ examples of how to use extended severity information.)

Ascribing meaning to levels

A previous post on this blog discusses the meanings we tend to ascribe to the 8 Syslog-derived levels. Naturally, if we're expanding the stock levels we need to take care to update this picture. In summary, we can say that we interpret them as follows, and offer it as a guide to you:


Level Meaning
Violation Used exclusively in the attempt to record a contract violation
Alert Used to report a practically-unrecoverable runtime condition, i.e. one that will require imminent process termination but is not a fault
Critical Serious failure to achieve normative behaviour
Error Failure to achieve normative behaviour
Warning Warning condition
Notice Information to be logged in normal operation, meaning when there has been no need to increase the logging information above a normal condition
Informational Logging information that is useful when actively monitoring the health of a system, but that is not necessarily displayed as part of "normal" operation 
Debug-0 Highest conceptual level and/or most terse form of debugging logging
Debug-1 Next lower conceptual level and more verbose than Debug-0
Debug-2 Next lower conceptual level and more verbose than Debug-1
Debug-3 Next lower conceptual level and more verbose than Debug-2
Debug-4 Next lower conceptual level and more verbose than Debug-3
Debug-5 Lowest conceptual level and most verbose form of debugging logging
Trace Used exclusively to record function/method entry/exit (along with, optionally, function arguments)
Benchmark Used exclusively to emit terse statements - usually only a literal string, to cause least impact to system performance - for the express purpose of tracking system performance

Friday, June 5, 2020

Custom Severity Levels with Pantheios C/C++ project

Been a bit of demand for this one, so here we go, in as few words as possible.

Stock Severity Levels

Out of the box, up to the present version (1.0.1 beta 218) the Pantheios C/C++ library comes with eight stock severity levels, as follows:


Manifest constantIntegral valueString form (1)
PANTHEIOS_SEV_EMERGENCY0"Emergency"
PANTHEIOS_SEV_ALERT1"Alert"
PANTHEIOS_SEV_CRITICAL2"Critical"
PANTHEIOS_SEV_ERROR3"Error"
PANTHEIOS_SEV_WARNING4"Warning"
PANTHEIOS_SEV_NOTICE5"Notice"
PANTHEIOS_SEV_INFORMATIONAL   6"Informational"   
PANTHEIOS_SEV_DEBUG7"Debug"

1. as obtained by pantheios_getStockSeverityString()

These are defined in the file pantheios/pantheios.h and are available for both C and C++ compilation units. Additionally, for C++ compilation units the following pseudo-constants are available:


C++ pseudo-constant (2)Integral ValueCorresponding constant
pantheios::emergency0PANTHEIOS_SEV_EMERGENCY
pantheios::alert1PANTHEIOS_SEV_ALERT
pantheios::critical2PANTHEIOS_SEV_CRITICAL
pantheios::error3PANTHEIOS_SEV_ERROR
pantheios::warning4PANTHEIOS_SEV_WARNING
pantheios::notice5PANTHEIOS_SEV_NOTICE
pantheios::informational   6PANTHEIOS_SEV_INFORMATIONAL
pantheios::debug7PANTHEIOS_SEV_DEBUG

2. these are defined 

These C++-only pseudo-constants also facilitate easily the use of extended severity information, but that's a topic for another time.

The string forms' C-style strings and string lengths are obtained, respectively, via the pantheios_getStockSeverityString() and pantheios_getStockSeverityStringLength() API functions.

Custom Severity Levels

If you want to use a different set of severities, then you must do the following:
  1. #define the preprocessor symbol PANTHEIOS_NO_STOCK_LEVELS;
  2. link to a back-end of your own that understands your custom severity levels.

Here's a C-program illustrating how to do this (with as much detail as possible elided for brevity):



#define PANTHEIOS_NO_STOCK_LEVELS /* suppress stock severity levels */

#include ⟨pantheios/pantheios.h⟩

/* some log4j-like levels */

enum CustomSeverityLevel_t
{
  CSL_Fatal = 0,
  CSL_Error = 3,
  CSL_Warn  = 4,
  CSL_Info  = 6,
  CSL_Debug = 7,
};

/* the application */

int main()
{
  int retCode = EXIT_SUCCESS;
  int res;

  res = pantheios_init();

  if(0 != res)
  {
    fprintf(stderr, "Failed to initialise the Pantheios libraries: %s\n", pantheios_getInitCodeString(res));

    retCode = EXIT_FAILURE;
  }
  else
  {
    pantheios_logputs(CSL_Info, PANTHEIOS_LITERAL_STRING("pantheios_logputs() can output a single C-style string"));


    . . . //rest of program logic, including more pantheios_logputs() / pantheios_logprintf() statements


    pantheios_uninit();
  }

  return retCode;
}

/* the custom back-end */

#include ⟨pantheios/backend.h⟩
#include ⟨pantheios/init_codes.h⟩
#include ⟨pantheios/strdup.h⟩

PANTHEIOS_CALL(int) pantheios_be_init(
  PAN_CHAR_T const* processIdentity
, void*             reserved
, void**            ptoken
)
{
  ((void)reserved);

  *ptoken = pantheios_util_strdup_nothrow(processIdentity);

  return (NULL == *ptoken) ? PANTHEIOS_INIT_RC_OUT_OF_MEMORY : PANTHEIOS_INIT_RC_SUCCESS;
}

PANTHEIOS_CALL(void) pantheios_be_uninit(void *token)
{
  pantheios_util_strfree((PAN_CHAR_T*)(token));
}

PANTHEIOS_CALL(int) pantheios_be_logEntry(  
  void*             feToken
, void*             beToken
, int               severity
, PAN_CHAR_T const* entry
, size_t            cchEntry
)
{
  PAN_CHAR_T const* sevString;
  PAN_CHAR_T const* processId = (PAN_CHAR_T const*)(beToken);

  ((void)feToken);

  switch (severity)
  {
  case CSL_Fatal:
    sevString = PANTHEIOS_LITERAL_STRING("Fatal");
    break;
  case CSL_Error:
    sevString = PANTHEIOS_LITERAL_STRING("Error");
    break;
  case CSL_Warn:
    sevString = PANTHEIOS_LITERAL_STRING("Warn");
    break;
  case CSL_Info:
    sevString = PANTHEIOS_LITERAL_STRING("Info");
    break;
  case CSL_Debug:
    sevString = PANTHEIOS_LITERAL_STRING("Debug");
    break;
  default:
    sevString = PANTHEIOS_LITERAL_STRING("");
    break;
  }

#ifdef PANTHEIOS_USE_WIDE_STRINGS
  return fwprintf(
        stderr
      , L"[%s; %s]: %.*s\n"
      , processId
      , sevString
      , (int)cchEntry, entry
      );
#else /* ? PANTHEIOS_USE_WIDE_STRINGS */
  return fprintf(
        stderr
      , "[%s; %s]: %.*s\n"
      , processId
      , sevString
      , (int)cchEntry, entry
      );
#endif /* PANTHEIOS_USE_WIDE_STRINGS */
}

Tuesday, February 5, 2019

Coming in 2019: Pantheios 1 (non-beta); Pantheios 2

Ok, so it's been a heck of a long while since Pantheios' first beta was released. The last released beta was 219 (!), on 28th January 2017. There's been a fair amount of useful work on the library since then, but not released.

(As has been obvious since the outset of its release into the world, having the first version number as 1.0.1 and then just bumping the beta was crazy optimistic, and in practice just crazy. This very aspect has hindered speed of progress, as it's worked against the principle of release early, release often.)

There's also been a lot of change in "the ideal conception of Pantheios in C/C++" in this time. There are no massive changes, but there are some important differences that will require a bump in major number, which will be outlined in posts in the near future.

Both of the above (as yet unreleased) works have arisen due to use and commercial customisations I've been doing for a client, in which Pantheios has seen tremendous use in exacting circumstances. (I say this to reassure that the project is not dead, not even resting, stunned, or pining for the fjords. It is alive, and it will be sprouting wings again into the world this year.)

So, without much further ado, the coming changes of the Pantheios project are:
  • The existing Pantheios project (1.0.1-bXXX) will be mothballed;
  • The current state of Pantheios will be effectively rendered in another project, Pantheios1;
  • The new, improved version will be rendered in another project, Pantheios2, sometime before the middle of the year. Subsequent posts will explain the new/changed features.


Sunday, September 27, 2015

Use of the showPlus format flag

Pantheios 1.0.1 beta 215 introduces the showPlus format flag, which causes leading pluses to be applied to integers (just as leading minuses will appear for negative numbers).

It is used as follows:

#include <pantheios/pan.hpp>
#include <pantheios/inserters/i.hpp>

int main()
{
  pan::log_INFORMATIONAL("log -10:");
  pan::log_INFORMATIONAL("without showPlus: '", pan::i(-10), "'");
  pan::log_INFORMATIONAL("with    showPlus: '", pan::i(-10, 0, pan::fmt::showPlus), "'");
  pan::log_INFORMATIONAL("----------\n");

  pan::log_INFORMATIONAL("log +10:");
  pan::log_INFORMATIONAL("without showPlus: '", pan::i(+10), "'");
  pan::log_INFORMATIONAL("with    showPlus: '", pan::i(+10, 0, pan::fmt::showPlus), "'");
  pan::log_INFORMATIONAL("----------\n");


  pan::log_INFORMATIONAL("log -10 into 10-width:");
  pan::log_INFORMATIONAL("without showPlus: '", pan::i(-10, 10, 0), "'");
  pan::log_INFORMATIONAL("with    showPlus: '", pan::i(-10, 10, pan::fmt::showPlus), "'");
  pan::log_INFORMATIONAL("----------\n");

  pan::log_INFORMATIONAL("log +10 into 10-width:");
  pan::log_INFORMATIONAL("without showPlus: '", pan::i(+10, 10, 0), "'");
  pan::log_INFORMATIONAL("with    showPlus: '", pan::i(+10, 10, pan::fmt::showPlus), "'");
  pan::log_INFORMATIONAL("----------\n");


  pan::log_INFORMATIONAL("log -10 into 10-width left-justified:");
  pan::log_INFORMATIONAL("without showPlus: '", pan::i(-10, -10, 0), "'");
  pan::log_INFORMATIONAL("with    showPlus: '", pan::i(-10, -10, pan::fmt::showPlus), "'");
  pan::log_INFORMATIONAL("----------\n");

  pan::log_INFORMATIONAL("log +10 into 10-width left-justified:");
  pan::log_INFORMATIONAL("without showPlus: '", pan::i(+10, -10, 0), "'");
  pan::log_INFORMATIONAL("with    showPlus: '", pan::i(+10, -10, pan::fmt::showPlus), "'");
  pan::log_INFORMATIONAL("----------\n");


  return 0;
}

PANTHEIOS_EXTERN_C PAN_CHAR_T const PANTHEIOS_FE_PROCESS_IDENTITY[] = PANTHEIOS_LITERAL_STRING("test.scratch.showPlus.integer");

which gives the output:

[test.scratch.showPlus.integer...]: log -10:
[test.scratch.showPlus.integer...]: without showPlus: '-10'
[test.scratch.showPlus.integer...]: with    showPlus: '-10'
[test.scratch.showPlus.integer...]: ----------

[test.scratch.showPlus.integer...]: log +10:
[test.scratch.showPlus.integer...]: without showPlus: '10'
[test.scratch.showPlus.integer...]: with    showPlus: '+10'
[test.scratch.showPlus.integer...]: ----------

[test.scratch.showPlus.integer...]: log -10 into 10-width:
[test.scratch.showPlus.integer...]: without showPlus: '       -10'
[test.scratch.showPlus.integer...]: with    showPlus: '       -10'
[test.scratch.showPlus.integer...]: ----------

[test.scratch.showPlus.integer...]: log +10 into 10-width:
[test.scratch.showPlus.integer...]: without showPlus: '        10'
[test.scratch.showPlus.integer...]: with    showPlus: '       +10'
[test.scratch.showPlus.integer...]: ----------

[test.scratch.showPlus.integer...]: log -10 into 10-width left-justified:
[test.scratch.showPlus.integer...]: without showPlus: '-10       '
[test.scratch.showPlus.integer...]: with    showPlus: '-10       '
[test.scratch.showPlus.integer...]: ----------

[test.scratch.showPlus.integer...]: log +10 into 10-width left-justified:
[test.scratch.showPlus.integer...]: without showPlus: '10        '
[test.scratch.showPlus.integer...]: with    showPlus: '+10       '
[test.scratch.showPlus.integer...]: ----------


GitHub access

Pantheios GitHub access now at:




Fork away!

Pantheios 1.0.1 beta 215 released

Pantheios is an open source C/C++ Logging API library, offering an optimal
combination of 100% type-safety, efficiency, genericity and extensibility.
It is simple to use and extend, highly-portable (platform and
compiler-independent) and, best of all, it upholds the C tradition of you
only pay for what you use.

Pantheios supports logging of message statements of arbitrary complexity,
consisting of heterogeneous types.

Pantheios supports filtering of log messages based on severity level
including (but not limited to) the eight levels defined by the SysLog
protocol.

Pantheios supports back-end output, individually and in combination, to
stderr/stdout, SysLog (including a custom implementation of the SysLog
protocol for Windows), Windows debugger, Windows event log, COM Error
Object, Speech, or any custom back-end extension you care to write.
Importantly, Pantheios is readily extended to use the existing transport
mechanisms of feature-rich logging libraries such as ACE, log4cpp,
log4cplus, log4cxx.

Pantheios does not contain any compiler-specific or platform-specific
constructs. It supports UNIX (including Linux and Mac OS-X), and Windows,
and should work with any operating system. It is known to be compatible with
Borland (5.6+), Clang, Comeau (4.3.3+), Digital Mars (8.45+), GCC (3.4+), Intel
(7+), Metrowerks (8+), Microsoft Visual C++ (6.0+), and should work with any
reasonably modern C++ compiler.

Pantheios is completely free and includes source released under a BSD-style
license. Commercial servics, including bespoke customisations, are available
from Synesis Software Pty Ltd; http://synesis.com.au/contact.html

Pantheios Training is provided by Synesis Software Pty Ltd; details at
http://synesis.com.au/training.html

Release 1.0.1 beta 215 incorporates:
* added interval inserter
* added stream_character inserter
* Clang-compatibility
* VC++ 11-compatibility
* VC++ 12-compatibility
* VC++ 14-compatibility
* showPlus format flag support in pantheios::integer (pan::i)
* makefiles customisable to specify non-bundled locations of b64, shwild,
xContract, xCover, and xTests libraries (useful to getting head from GitHub)
* NOTE: Now requires STLSoft 1.9.121


Download from:
https://sourceforge.net/projects/pantheios/files/Pantheios%20%28C%20and%20Cxx%29/

Discuss at: https://sourceforge.net/projects/pantheios/forums/forum/475313

Pantheios website: http://pantheios.org/

Note: this release of Pantheios requires STLSoft 1.9.121, or later; download
from http://stlsoft.org/

Monday, May 21, 2012

Extended Radio Silence - ending in Q3 2012

To anyone who's still following any of my public works - FastFormat, Pantheios, STLSoft, Breaking Up The Monolith, Quality Matters, VOLE, etc. - and wondering whether these activities are permanently moribund, I want to let you know that I'll soon be free of a very intense and overwhelmingly consuming commercial engagement over the last 2.5 years, and the second half of this year should see much activity in open-source, commercial, and writing activities.

Cheers

Matt