1

At first glance (see the evidence below), it looks like while a Tcl_CmdProc has control, the interpreter is waiting for it to return and can't accept any other calls in the meantime.

So, how do I make any calls into Tcl before returning like e.g. a user-defined function would do? I guess I may need to set up a new call stack frame in the interpreter or something (and unwind it later). Tcl_CreateCommand man page says nothing on this matter.

The big picture is like this:

I'm fixing https://bugs.python.org/issue33257 . The TkinterHandlers.py example uses Python event handlers that are implemented as custom Tcl commands under the hood. Currently, their implementation releases the "Tcl lock" (a Python-specific lock that it wraps all Tcl calls with) while executing Python code and reacquires it to Tcl_SetObjResult at the end -- thus allowing other calls to the same interpreter in the meantime.

Now, if another call into the interpreter is actually made during this time frame, Tcl aborts shortly with a message on stderr: TclStackFree: incorrect freePtr. Call out of sequence?

And if I make the custom command hold on to the Tcl lock, it later freezes trying to acquire the lock again because it itself also needs to make a Tcl call sometimes. Now, I can make the lock reentrant, but without knowing how to handle the interpreter right, I'll probably break it, too.

To keep this question on topic, I'm specifically asking about how to handle the interpreter, and make Tcl calls in particular, from a Tcl_CmdProc. The specific situation is solely for exposition to illustrate my needs. If this is actually explained in some doc that I couldn't find, linking to it and reciting some key points would be sufficient.

Improve this question
4
  • I suggest you show code or point to it. This will help clarify what you mean by "thus allowing other calls to the same interpreter in the meantime." The Tcl threading model maintains one interpreter per (user-space) thread. If you have interleaving Tcl_CmdProc executions hammering onto one and the same interp, there is something fundamentally wrong. Event handlers as commands map to Tcl events managed by the Tcl event loop. Better watch out for Tcl_DoWhenIdle, Tcl_QueueEvent Commented Apr 15, 2018 at 21:35
  • @mrcalvin the C code is Python's _tkinter module that runs as a part of a Python installation. The custom command implementation there is Tkapp_PythonCmd, regular Tcl calls from Python are made via Tkapp_Call, and the event loop is _tkinter_tkapp_mainloop_impl. The linked .py file is Python code that demonstrates the aforementioned crash. Commented Apr 15, 2018 at 22:38
  • 1
    DO NOT call a Tcl interpreter from any OS thread other than the one it was created in; Tcl's implementation uses thread-local storage quite extensively to avoid global locks, but it means that multithreaded usage will definitely blow up every time. Commented Apr 15, 2018 at 22:46
  • 1
    Those are also not the only place where the single-threaded-ness of an interpreter is assumed; it sounds like you hit problems in one of our high-performance memory allocators. That's an absolute “what you are doing is unsupported; stop doing it” failure. Commented Apr 15, 2018 at 23:00

1 Answer 1

Reset to default
1

To call a Tcl command from C code, you've got a choice between two API function families. One is Tcl_EvalObjv, and the other is Tcl_Eval. Each has a number of variants, but the only variant I'll mention is Tcl_EvalObjEx.

Tcl_EvalObjv

This function invokes a single Tcl command, with no processing of substitutions in arguments (unless the command itself does them, of course). It has this signature:

int Tcl_EvalObjv(Tcl_Interp *interp,
                 int objc,
                 Tcl_Obj *const objv[],
                 int flags);

It takes the description of what command to call and what arguments to pass to it as a C array of Tcl value references (in argument objv) where the array is of length objc; Tcl guarantees to not modify the array itself, but might transform the values if it does type conversions. The values must all have a non-zero reference count (and all values start with a zero reference count from their birthing Tcl_NewObj call). The interp is the interpreter context, and flags can usually be zero.

The result is a Tcl exception code; if it is TCL_OK, the result of the call can be retrieved from the interpreter using Tcl_GetObjResult, and if the exception code is TCL_ERROR then there was an error and you should usually pass that on out (perhaps adding to the stack trace with Tcl_AddErrorInfo). Other exception codes are possible; it's usually best to just pass those straight on out without doing any further processing (unless you're making something loop-like, when you should pay attention to TCL_BREAK and TCL_CONTINUE).

Tcl_Eval

This function evaluates a Tcl script, not just a single command, and that includes processing substitutions in arguments. It has this signature:

int Tcl_Eval(Tcl_Interp *interp,
             const char *script);

The script is any old C string; Tcl won't modify it, but it will parse, bytecode-compile, and execute it. It's up to you to provide the script in a form that will execute a single command without surprises. The interp argument and the result of the function call are the same as for Tcl_EvalObjv.

If you're interested in using this for running a single command, you're actually better off using Tcl_EvalObjv or…

Tcl_EvalObjEx.

This is like Tcl_Eval except it takes the script as a Tcl value reference (and takes flags too).

int Tcl_EvalObjEx(Tcl_Interp *interp,
                  Tcl_Obj *objPtr,
                  int flags);

Again, make sure the objPtr has a non-zero reference count before passing it into this function. (It may adjust the reference count during execution.) Again, interp and the result are as documented for Tcl_EvalObjv, and flags is too.

The advantage of this for calling single commands is that you can call Tcl_NewListObj (or any other list-building function) to make the script value; doing so guarantees that there will be no surprise substitutions. But you could also go directly to invoking the command with Tcl_EvalObjv. But if you want to process anything more complex than a single simple call to a command, this is a good place to start as it has a key advantage that plain Tcl_Eval doesn't: it can make the type of the script passed in via objPtr be one that caches the compiled bytecode, allowing quite a reasonable performance gain in some circumstances.

Note that Tcl_EvalObjv is effectively the API that Tcl calls internally to invoke all user code and perform all I/O. (“Effectively” because things get more complex in Tcl 8.6.)

Within a Tcl_CmdProc, all these functions can be called as usual, no special processing or "handling of the interpreter" is needed. If this doesn't work for you, causing crashes or whatever, the interpreter is not at fault, something else must be wrong with your code.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Okay, but how do these calls work while a Tcl_CmdProc has control? Can I just call them from that proc at any time, without any special processing?
The Tcl_CmdProc can call them as much as it wants. Other threads must not. Tcl's assumed thread model keeps each interpreter within a single thread, and yes, this is very different to that of Python.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.