scrobj(n) 1.0 "scripted Tcl_Objects"


scrobj - scripted Tcl_Objects


package require Tcl 8.5
package require scrobj 1.0

scrobj register typeName updateCode parseCode
scrobj convert typeName string ?extraArg?
scrobj value typeName intRep ?extraArg?
scrobj eval typeName code
scrobj info ?typeName?


Tcl's fundamental mantra says that "everything is a string" and much of Tcl's simplicity derives from this rule. Strings can, however, carry with them an internal representation as some other data type, and this is one reason for Tcl's astonishing performance. For example, Tcl maintains such an internal representation for integers, lists, bytecodes, and many other types.

Extension writers have long been able to add new custom types using Tcl's C interface. The scrobj package allows you to do the same from the script level.

Unfortunately, in order to achieve this the package has to violate a basic assumption of Tcl's bytecode engine, namely that it is not getting invoked recursively from a call to Tcl_GetString. A consequence is that scrobj can be used to crash the application, although this does require a rather contrived setup.

To prevent such crashes in the legitimate cases the package uses a dedicated Tcl interpreter for every registered type. Both Tcl 8.5 and Tcl 8.6 seem to tolerate recursive invocations of the bytecode engine, as long as they execute in different interpreters.


scrobj register typeName updateCode parseCode
To register a new data type you have to write code to transform the string value to the internal representation and vice versa. This code then needs to be registered using scrobj register.

scrobj register mytype {
    intRep {
	# code to generate the string
	# from the internal representation
	set dummy "<$intRep>"
} {
    stringVal {
	# code to generate the internal 
	# representation from the string
	regexp -- {^<([0-9]+)>$} $stringVal -> irp
	set irp

The call to scrobj register implicitly creates a new Tcl interpreter where the conversion routines will be executed.

scrobj convert typeName string ?extraArg?
Return the internal representation corresponding to string for the type typeName. If the internal representation is already available, that value will be returned directly. Otherwise the internal representation will first be generated by applying the type's parseCode to the given string.

# convert "<4711>" and return the internal representation "4711"
scrobj convert mytype <4711>

scrobj value typeName intRep ?extraArg?
Generate a value from the internal representation.

# generate a value for type mytype
set a [scrobj value mytype 123]

# the next call will not invoke the 
# parseCode because $a already has an internal
# representation for type "mytype"
set b [scrobj convert mytype $a]

# the next line will run the updateCode for mytype
set c "a is $a" ;# -> "a is <123>"

scrobj eval typeName code
Execute some code in the interpreter for type typeName.

scrobj info ?typeName?
Return the list of all registered types, or information about one specific type.

Parametrized types

The convert and value subcommands allow to specify an optional extra argument. This can be used to parametrize a data type by a value:


# register a parametrized type that uses a configurable
# regular expression for the conversion
scrobj register rexp {
    irep {
        # string is kept in internal rep
        lindex $irep 1
} {
    {inString regExp} {
        regexp -- $regExp $inString -> res
        list $res $inString

set pat1 {^foo([0-9]+)$}
set pat2 {^foo([0-9]+)$}
set pat3 {^fo(o[0-9]+)$}

set a [scrobj value rexp [scrobj convert rexp foo17 $pat1] $pat1]

# The next line will use the internal representation
# of $a that's already there
set b [scrobj convert rexp $a $pat1] 

# The next line will re-generate the internal
# representation, because $pat1 and $pat2
# are not represented by the same Tcl_Object.
set c [scrobj convert rexp $a $pat2]

# The next line also re-generates the internal
# representation. This time the return value
# is actually different.
set d [scrobj convert rexp $a $pat3]

Application examples

scrobj comes equipped with two packages, scrobj::expression and scrobj::rational, which illustrate the use of scrobj types.

scrobj::expression implements a parser for simple arithmetic expressions.

package require scrobj::expression 1.0

scrobj convert scrobj::expression {23*foo(bar(12),3+$a/7)} {
    opns ::operatorns::
    funcns ::funcns::

# the result of the conversion is Tcl code that 
# implements the computation:
    upvar 1 a 3 
    set 1 [::funcns::bar [eval {set dummy 12}]]
    set 2 [::funcns::foo [eval {set 1}] [eval {
	set 4 [::operatorns::/ $3 7]
	set 5 [::operatorns::+ 3 $4]
	set 5
    set 6 [::operatorns::* 23 $2]
    set 6
} {23*foo(bar(12),3+$a/7)}

The parser is implemented in Tcl using regular expressions. It doesn't have to be fast, because the script will be cached in the internal representation once it is generated.

The scrobj::rational package implements rational number arithmetic. A rational number p/q is represented internally as the list [list $p $q].

package require scrobj::rational 1.0

scrobj::rational numerator 17272/9128 ;# -> 17272
scrobj::rational + 1727/162 8348/1731 ;# -> 1447271/93474

scrobj::rational uses scrobj::expression to implement its own version of the expr command:

package require scrobj::rational 1.0

# register some custom functions
namespace eval rational {
    namespace eval func {
	proc f1 {a b} {
	    ::scrobj::rational expression {3*$a-17/93*$b}
	proc f2 {x} {
	    ::scrobj::rational expression {2*$x-1}

set a 72/841

scrobj::rational expression {15*f1($a,3-$a) - f2(17/18+$a)/9}
# -> -8988239/2111751


Tcl_Object, expression, rational, scrobj