Excel formatted file

Comparing Tcl to Jacl/TJC



The following timing results compare execution times for the C implementation of Tcl to Jacl code compiled with TJC.

System:
Win XP (sp2)
1.8 GHz CPU
256 Megs RAM

Tcl:
Tcl 8.4.12
Built with default optimizations:
gcc  -O2 -fomit-frame-pointer

Jacl
From CVS on 6/21/2006
JDK 1.4.2_05 from Sun
Server Hotspot JVM

Invoked Tcl Bench via:

export JAVA_FLAGS="-server -Xbatch"

tclsh runbench.tcl jacl2+tjc *.bench

This executes java via:

java -ms5m -mx32m -server -Xbatch tcl.lang.Shell ...

These results show how long in ms is took for Tcl to execute the named test vs Jacl code compiled with TJC.

TEST Tcl 8.4 Jacl/TJC Delta
Faster (+-2 ms) X-Faster
append-array-ten-n-words 30 58 28
Tcl 1.93
append-array-var-key-ten-n-words 53 60 7
Tcl 1.13
append-inline-string 6 7 1


append-n-words 10 7 -3
Jacl 1.43
append-n-words-big 16 31 15
Tcl 1.94
append-one-words 9 8 -1


append-one-words-big 16 65 49
Tcl 4.06
append-strings 14 9 -5
Jacl 1.56
append-ten-n-words 27 19 -8
Jacl 1.42
append-ten-n-words-big 245 808 563
Tcl 3.3
append-ten-one-words 23 21 -2


append-ten-one-words-big 215 1260 1045
Tcl 5.86
data-access-array 527 370 -157
Jacl 1.42
data-access-list 220 183 -37
Jacl 1.2
data-create-array 1520 789 -731
Jacl 1.93
data-create-list 177 388 211
Tcl 2.19
expr-big-expression 10 5 -5
Jacl 2
expr-bigsqrt1-1 24 22 -2
Jacl 1.09
expr-bigsqrt1-2 24 23 -1


expr-bigsqrt2-1 11 14 3
Tcl 1.27
expr-bigsqrt2-2 11 13 2


expr-braced 5 8 3
Tcl 1.6
expr-empty-not-string 6 4 -2


expr-empty-string 6 4 -2


expr-empty-string-eq-loop 275 192 -83
Jacl 1.43
expr-empty-string-loop 296 192 -104
Jacl 1.54
expr-empty-string-two-obj-eq-loop 296 293 -3


expr-empty-string-two-obj-loop 344 240 -104
Jacl 1.43
expr-equal-constant-int 5 4 -1


expr-equal-constant-int-loop 255 178 -77
Jacl 1.43
expr-equal-not-same-string-two-obj-eq-loop 341 193 -148
Jacl 1.77
expr-equal-not-same-string-two-obj-loop 873 439 -434
Jacl 1.99
expr-equal-same-big-constant-string 13 4 -9
Jacl 3.25
expr-equal-same-char-string 11 6 -5
Jacl 1.83
expr-equal-same-char-string-index 13 6 -7
Jacl 2.17
expr-equal-same-char-string-parse 17 6 -11
Jacl 2.83
expr-equal-same-constant-string 8 4 -4
Jacl 2
expr-equal-same-string-two-obj-eq-loop 296 173 -123
Jacl 1.71
expr-equal-same-string-two-obj-loop 588 425 -163
Jacl 1.38
expr-evaluate-add-fifty_double_easy 10 14 4
Tcl 1.4
expr-evaluate-add-fifty_double_hard 11 13 2


expr-evaluate-add-fifty_double_obj 8 15 7
Tcl 1.88
expr-evaluate-add-fifty_fifty_double_hard 478 505 27
Tcl 1.06
expr-evaluate-add-fifty_fifty_int_hard 255 213 -42
Jacl 1.2
expr-evaluate-add-fifty_int_easy 7 8 1


expr-evaluate-add-fifty_int_hard 6 8 2


expr-evaluate-add-fifty_int_obj 7 12 5
Tcl 1.71
expr-evaluate-add-ten_double 3 6 3
Tcl 2
expr-evaluate-add-ten_int 2 5 3
Tcl 2.5
expr-evaluate-one_double 2 4 2


expr-evaluate-one_int 2 4 2


expr-if-branch-loop 325 73 -252
Jacl 4.45
expr-incr-with-expr 2 4 2


expr-incr-with-incr 2 4 2


expr-inline 5 40 35
Tcl 8
expr-math-funcs 796 653 -143
Jacl 1.22
expr-streq-a-eq-b 2 6 4
Tcl 3
expr-streq-a-eq-b-eq-len 7 9 2


expr-streq-a-eq-b-neq-len 6 7 1


expr-streq-a-neq-b 2 6 4
Tcl 3
expr-streq-a-neq-b-neq-len 6 9 3
Tcl 1.5
expr-strneq-a-neq-b-eq-len 7 12 5
Tcl 1.71
expr-unary-not-double-literal 264 177 -87
Jacl 1.49
expr-unary-not-double-literal-subexpr 375 193 -182
Jacl 1.94
expr-unary-not-double-tclobject 298 190 -108
Jacl 1.57
expr-unary-not-double-tclobject-parse 311 207 -104
Jacl 1.5
expr-unary-not-int-literal 197 135 -62
Jacl 1.46
expr-unary-not-int-literal-subexpr 266 183 -83
Jacl 1.45
expr-unary-not-int-tclobject 233 117 -116
Jacl 1.99
expr-unary-not-int-tclobject-parse 244 127 -117
Jacl 1.92
expr-unary-not-string-literal 516 297 -219
Jacl 1.74
expr-unary-not-string-literal-subexpr 876 300 -576
Jacl 2.92
expr-unary-not-string-tclobject 536 318 -218
Jacl 1.69
expr-unbraced 38 22 -16
Jacl 1.73
bench_incr_array_constant_50 29 33 4
Tcl 1.14
bench_incr_array_nonconstant_50 28 33 5
Tcl 1.18
bench_incr_scalar_const_50 7 7 0


bench_incr_scalar_default_50 7 8 1


bench_incr_scalar_nonconst_50 6 8 2


invoke_1_to_10 166 539 373
Tcl 3.25
invoke_1_to_10_loop 8120 13897 5777
Tcl 1.71
invoke_empty 34 11 -23
Jacl 3.09
bench_list_append 43 130 87
Tcl 3.02
bench_list_constant 7 6 -1


bench_list_index_constant 12 9 -3
Jacl 1.33
bench_list_index_nonconstant 14 11 -3
Jacl 1.27
bench_list_llength 11 16 5
Tcl 1.45
bench_list_loop_lappend_array1 490 713 223
Tcl 1.46
bench_list_loop_lappend_array2 493 768 275
Tcl 1.56
bench_list_loop_lappend_scalar1 368 516 148
Tcl 1.4
bench_list_loop_lappend_scalar2 1155 676 -479
Jacl 1.71
bench_list_tostring 15 18 3
Tcl 1.2
loops-foo-num-500 104 77 -27
Jacl 1.35
loops-for-list 218 187 -31
Jacl 1.17
loops-for-list-ten 433 303 -130
Jacl 1.43
Loops-for-string-index 475 90 -385
Jacl 5.28
loops-foreach-list 98 62 -36
Jacl 1.58
loops-foreach-list-ten 303 275 -28
Jacl 1.1
loops-foreach-split-char 2003 1275 -728
Jacl 1.57
loops-foreach-split-char-def 2488 1677 -811
Jacl 1.48
loops-foreach-split-every-char 129 82 -47
Jacl 1.57
loops-split-char 8510 5786 -2724
Jacl 1.47
loops-split-char-def 10975 7581 -3394
Jacl 1.45
loops-split-every-char 246 345 99
Tcl 1.4
loops-while-1-500 92 102 10
Tcl 1.11
loops-while-1-ten-500 305 123 -182
Jacl 2.48
loops-while-num-500 105 72 -33
Jacl 1.46
loops-while-num-ten-500 304 87 -217
Jacl 3.49
bench_set_array_and_scalar_names 27 37 10
Tcl 1.37
bench_set_array_get_same_element 9 7 -2


bench_set_array_global_import 8 5 -3
Jacl 1.6
bench_set_array_same_element 9 7 -2


bench_set_array_variable_keys 28 21 -7
Jacl 1.33
bench_set_different_array_elements 26 20 -6
Jacl 1.3
bench_set_different_scalars 6 7 1


bench_set_get_same_scalar 6 5 -1


bench_set_global_import 12 9 -3
Jacl 1.33
bench_set_same_scalar 5 4 -1


bench_set_six_arguments 3 12 9
Tcl 4
bench_string_compare 5 3 -2


bench_string_equal 5 4 -1


bench_string_index 4111 1942 -2169
Jacl 2.12
bench_string_index_foreach 874 869 -5


bench_string_length1 5 4 -1


bench_string_length2 5 7 2


switch-constant-3 6 3 -3
Jacl 2
switch-constant-az-char 7 3 -4
Jacl 2.33
switch-constant-az-compare 7 4 -3
Jacl 1.75
switch-constant-az-compare-big 13 10 -3
Jacl 1.3
switch-constant-az-first 7 3 -4
Jacl 2.33
switch-constant-az-len 7 4 -3
Jacl 1.75
switch-dummy 6 3 -3
Jacl 2
switch-glob 7 10 3
Tcl 1.43
switch-loop-constant-10 101 14 -87
Jacl 7.21
switch-loop-constant-az-char 127 7 -120
Jacl 18.14
switch-loop-constant-az-compare 136 23 -113
Jacl 5.91
switch-loop-constant-az-compare-big 390 327 -63
Jacl 1.19
switch-loop-constant-az-first 127 8 -119
Jacl 15.88
switch-loop-constant-az-len 148 7 -141
Jacl 21.14


Overview of Results


append.bench:

Tcl is faster for most of the append tests cases. Jacl is faster for a few of these test that append very small strings.

data.bench:

Jacl is significantly faster that Tcl for all tests except data-create-list.

expr.bench:

Most execution times for Tcl and Jacl expr code are very close. When Tcl is faster, it tends to be about 2 to 5 ms faster which is not much faster. When the Jacl version executes faster, it tends to be by a larger margin since the test is in a loop.

incr.bench:

Tcl is slightly faster for a few tests, but most are a tie.

invoke.bench:

Tcl shows a significant advantage in command invocation unless the method take no arguments.

list.bench:

Tcl is faster for most lappend and llength operations. Jacl's lindex implementation is faster as is an lappend to a scalar variable with multiple arguments.

loops.bench:

Jacl looping implementation is significantly faster that Tcl's in all tests. Jacl's split command is also significantly faster.

set.bench:

Jacl has a slight advantage in a few set operations, but most are about the same.

string.bench:

These tests are about the same. Jacl's optimized string index command is about twice as fast as Tcl's.

switch.bench:

Jacl's switch implementation is 2 to 20 times faster that Tcl's in all constant switch cases.

All in all, these results indicate that most TJC compiled Jacl code executes just as quickly as Tcl code run in the C version. In many cases, Tcl code running in Jacl outperforms the same code running in the C version.

The following charts split test results into 3 groups based on how long a test took to execute.

Tests with times from 1 to about 30



TEST ID Tcl 8.4 Jacl/TJC
append-inline-string 1 6 7
append-n-words 2 10 7
append-n-words-big 3 16 31
append-one-words 4 9 8
append-strings 5 14 9
append-ten-n-words 6 27 19
append-ten-one-words 7 23 21
expr-big-expression 8 10 5
expr-bigsqrt1-1 9 24 22
expr-bigsqrt1-2 10 24 23
expr-bigsqrt2-1 11 11 14
expr-bigsqrt2-2 12 11 13
expr-braced 13 5 8
expr-empty-not-string 14 6 4
expr-empty-string 15 6 4
expr-equal-constant-int 16 5 4
expr-equal-same-big-constant-string 17 13 4
expr-equal-same-char-string 18 11 6
expr-equal-same-char-string-index 19 13 6
expr-equal-same-char-string-parse 20 17 6
expr-equal-same-constant-string 21 8 4
expr-evaluate-add-fifty_double_easy 22 10 14
expr-evaluate-add-fifty_double_hard 23 11 13
expr-evaluate-add-fifty_double_obj 24 8 15
expr-evaluate-add-fifty_int_easy 25 7 8
expr-evaluate-add-fifty_int_hard 26 6 8
expr-evaluate-add-fifty_int_obj 27 7 12
expr-evaluate-add-ten_double 28 3 6
expr-evaluate-add-ten_int 29 2 5
expr-evaluate-one_double 30 2 4
expr-evaluate-one_int 31 2 4
expr-incr-with-expr 32 2 4
expr-incr-with-incr 33 2 4
expr-streq-a-eq-b 34 2 6
expr-streq-a-eq-b-eq-len 35 7 9
expr-streq-a-eq-b-neq-len 36 6 7
expr-streq-a-neq-b 37 2 6
expr-streq-a-neq-b-neq-len 38 6 9
expr-strneq-a-neq-b-eq-len 39 7 12
bench_incr_array_constant_50 40 29 33
bench_incr_array_nonconstant_50 41 28 33
bench_incr_scalar_const_50 42 7 7
bench_incr_scalar_default_50 43 7 8
bench_incr_scalar_nonconst_50 44 6 8
invoke_empty 45 34 11
bench_list_constant 46 7 6
bench_list_index_constant 47 12 9
bench_list_index_nonconstant 48 14 11
bench_list_llength 49 11 16
bench_list_tostring 50 15 18

 1 to 30 Chart



Tests with times 30 to 1000

TEST ID Tcl 8.4 Jacl/TJC
append-array-ten-n-words 1 30 58
append-array-var-key-ten-n-words 2 53 60
append-one-words-big 3 16 65
append-ten-n-words-big 4 245 808
append-ten-one-words-big 5 215 1260
data-access-array 6 527 370
data-access-list 7 220 183
data-create-list 8 177 388
expr-empty-string-eq-loop 9 275 192
expr-empty-string-loop 10 296 192
expr-empty-string-two-obj-eq-loop 11 296 293
expr-empty-string-two-obj-loop 12 344 240
expr-equal-constant-int-loop 13 255 178
expr-equal-not-same-string-two-obj-eq-loop 14 341 193
expr-equal-not-same-string-two-obj-loop 15 873 439
expr-equal-same-string-two-obj-eq-loop 16 296 173
expr-equal-same-string-two-obj-loop 17 588 425
expr-evaluate-add-fifty_fifty_double_hard 18 478 505
expr-evaluate-add-fifty_fifty_int_hard 19 255 213
expr-if-branch-loop 20 325 73
expr-math-funcs 21 796 653
expr-unary-not-double-literal 22 264 177
expr-unary-not-double-literal-subexpr 23 375 193
expr-unary-not-double-tclobject 24 298 190
expr-unary-not-double-tclobject-parse 25 311 207
expr-unary-not-int-literal 26 197 135
expr-unary-not-int-literal-subexpr 27 266 183
expr-unary-not-int-tclobject 28 233 117
expr-unary-not-int-tclobject-parse 29 244 127
expr-unary-not-string-literal 30 516 297
expr-unary-not-string-literal-subexpr 31 876 300
expr-unary-not-string-tclobject 32 536 318
expr-unbraced 33 38 22
invoke_1_to_10 34 166 539
bench_list_append 35 43 130
bench_list_loop_lappend_array1 36 490 713
bench_list_loop_lappend_array2 37 493 768
bench_list_loop_lappend_scalar1 38 368 516
bench_list_loop_lappend_scalar2 39 1155 676
loops-foo-num-500 40 104 77
loops-for-list 41 218 187
loops-for-list-ten 42 433 303
Loops-for-string-index 43 475 90
loops-foreach-list 44 98 62
loops-foreach-list-ten 45 303 275
loops-foreach-split-every-char 46 129 82
loops-split-every-char 47 246 345
loops-while-1-500 48 92 102
loops-while-1-ten-500 49 305 123
loops-while-num-500 50 105 72
loops-while-num-ten-500 51 304 87
bench_set_array_and_scalar_names 52 27 37
bench_set_array_variable_keys 53 28 21
bench_set_different_array_elements 54 26 20
bench_string_index_foreach 55 874 869
switch-loop-constant-10 56 101 14
switch-loop-constant-az-char 57 127 7
switch-loop-constant-az-compare 58 136 23
switch-loop-constant-az-compare-big 59 390 327
switch-loop-constant-az-first 60 127 8
switch-loop-constant-az-len 61 148 7

 30 to 1000 Chart



 Large Times

TEST ID Tcl 8.4 Jacl/TJC
data-create-array 1 1520 789
invoke_1_to_10_loop 2 8120 13897
loops-foreach-split-char 3 2003 1275
loops-foreach-split-char-def 4 2488 1677
loops-split-char 5 8510 5786
loops-split-char-def 6 10975 7581
bench_string_index 7 4111 1942

 Large Times Chart