I’ve always loved Tcl because of its absolute purity and simplicity at the language level. I love how you can implement your own control flow operators that look like built-in ones. I love how easily you can make DSLs.
I just wish there was a Tcl that was redesigned from a modern perspective. Give me first-class lambdas. Give me unnamed objects that get destroyed automatically (with a callback) when out of scope[1] so I don’t have to use a weird namespace hack to make the object and rely on the caller to invoke the destroy method. Give me a stdlib that offers more functional stuff. Heck, give me async/await. And more too.
But even without all of that, Tcl is still a wonderful language and I often lament how it’s mostly died out.
[1] There’s at least one library I’ve used that actually does this, but AFAIK there’s no way to do this in pure Tcl so it must be a feature of the C API. Also I’m not sure if it actually tracks the object or just the variable it’s assigned to.
Tcl is one of the most underrated programming languages. I have a friend who is working at Intel as tool development engineer (equivalent to software engineer), and Tcl is being used extensively at Intel for chip tooling and design since a very long time.
The modern version of Tcl will be great and a seamless integration with the OS shell will probably the killer niche application. Hopefully Oil Shell can modernize Tcl to some extent and incorporate the best features of Tcl with excellent OS shell integration capabilities.
I haven't gotten around to looking into Oil Shell. Is it actually drawing any inspiration from Tcl? The "A Tour of the Oil Language" page[1] references Ruby and Python but not Tcl.
Not yet but I've mentioned to Andy (Oil Shell author) regarding the importance of adding command features into Oil Shell and he is actually looking into it [1]. Hopefully he will draw some inspirations from Tcl. Imagine a Cisco like networking command wrapper on top of Linux OS with eBPF backend. It will turn Linux into a ready-made formiddable open source networking OS without the need for NX-OS [2].
If you are looking into a modern version of Tcl, there is TIL [3]. It's a Tcl inspired new scripting language on top of D language. By having D language as the foundation it can perform all the features that you've requested including lambdas, async and then some more [4].
TIL looks interesting, but also both too simple and too complicated. What's the difference between a SubList and a SimpleList? Why does it require SimpleLists instead of using variadic arguments (e.g. why [math ($a + $b)] instead of [math $a + $b])? And why does the author explicitly disallow multiple spaces in a row outside of indentation when this is a purely artificial limitation and does not simplify anything?[1]
But also, why can't I write modules in TIL? It's based on D, sure, but I shouldn't need to write and compile something in D just to be able to have some means of namespacing things. This very much feels like "I want to write most of my stuff in D but I just want to be able to whip up short scripts that for whatever reason I don't want to write in D".
And of course it seems to be ditching the pure simplicity of Tcl. It's like Tcl in that it's a command language and borrows syntax from Tcl, but it seems to be missing the fundamental concepts of Tcl.
[1] The author seems to hate the idea of people lining up equals signs in code or things like that. Ok sure, you can have a style preference. But that's what a code formatter is for, not an artificial and unprecedented level of whitespace significance.
Tcl is deeply rooted in EDA and chip design industry, too many code were written with it. It's like C++, no matter how fancy/modern rust is, there is no way it's going to replace C++ for the next few decades.
One way is to provide Python(python stdlib might be enough for simplicity) gradually, e.g. providing a Python binding for Tcl so for the future, you can write all new piece in Python, and it is either used directly or converted to Tcl behind-the-scene.
I don't think that opening is there. Tcl may be a terrible scripting language, its usage is not a factor in the value of this or that EDA tool.
If a startup wants to displace an entrenched tool from Synopsys or Cadence, the underlying algorithms (millions of lines of C++ code) must be better. (And once that happens, they'll be acquired by one of these 2.)
I think you are probably right, it is not a make or break factor. Especially when it comes to back-end flows, PPA/TAT/capacity/design closure are the key metrics and the big 2.5 have large moats in these areas.
On the front-end side though, areas like verification, debug, and various lint checks, I think there is more opportunity for smaller players to introduce point tools with better customizability & scripting interfaces.
Ultimately I think the industry is stuck in a local maximum, where the frictional costs of rethinking overall methodology, e.g. from SystemVerilog to Chisel, is too high to justify the change, even if the end result would be marginally (or greatly) better. (Not an endorsement of Chisel, just an example.)
There is Python support in some EDA frameworks, but especially in the digital domain, Tcl is still the gold standard. On the one side this means there are tons of Tcl code around and any experienced engineer is well versed in Tcl.
Also, electrical engineers are usually very focussed on electric engineering. While being being very intelligent, they are often not into programming, so keeping things very simple is an advantage to them.
May be true of analog EEs, but more than half of EEs I work with, and all the good ones, can script well. They use perl/python for day to day, and tcl to interact with tools. The number who can use tcl well is a lot lower than those who can script in general though
python never seemed like a good fit for me, because it's just not suited for DSLs. It's too imperative and has too much built in syntax, you'll end up in callback hell as soon as you need an actually descriptive DSL.
I used a lot of TCL for stuff in the legacy networking space and while the language always seemed to be easy to reason about I seem to remember my main frustration was tracking down syntax errors as runtime problems. Dunno if I was just using the tooling wrong though.
Tcl's usage at Intel is not particularly special. Perl/Tcl were the workhorses of most (hardware) engineering companies. Back in 2010, a close friend who was working for one such company said that putting Python on your resume won't help you as none of the managers had heard of it. When I joined work around that time, I discovered he wasn't exaggerating much.
While Tcl may be a particularly good language for somethings, well over 90% of the teams I've found using it do so for legacy reasons, and they correlate very highly with poor SW engineers with even poorer SW practices. When I interview for jobs, I steer clear of any that lists Tcl. Not because of the language, but because it's a fairly useful signal about the quality of the team.
All EDA uses tcl, so no surprise Intel would too. A while back I saw some vendors tried to make perl or python an option, but tcl is the de facto language in this space those vendors often abandoned the having multiple scripting interfaces and fell back on tcl. It's quirky, but functional. I like the one line composability
You can't really get full-fledged closures without ditching the fundamental concept of Tcl, which is that "everything is a string". It's the same problem as OOP, or rather, object lifetime management - you can have that in Tcl, but only as long as you manage the lifetimes manually. For closures, this negates most of the benefits they provide.
It would work if they basically copy the current value of the identifiers (i.e. by-value, not by-reference). Or by capturing the procedure context. You can synthesize the variable capture by hand but that would indeed force everything into a string, which is a potential performance issue and would also break any sort of automatic lifetime management.
Even just making it easier to have lambdas reference variables from their creation scope while the creation scope is still on the stack would be an improvement. Today you can use [info level] and then construct a lambda body that uses [upvar #$level …] to reference variables from the original scope, but it's a PITA. Or you could cram the level into a temporary namespace variable to avoid having to manually construct a body string, but it's still a PITA.
[list apply {args body} $value1 $value2 ...] can be used to capture values without actually generating a string because even though conceptually a list is a string, it's actually stored internally as a list.
The basic problem is, how do you even know that something is a variable reference? Remember, everything is a string. This means that the body of the lambda is also a string. Now you can require that it's an eval'able string at the point of creation, and parse it to identify any $var references. But you can only do that for the topmost level of the lambda, because it can contain nested strings - and you don't know which of those strings are code, and which aren't (since it's really the command to which the string is passed that decides to treat something as code or not).
Furthermore, if you process $ like that, this won't work for any other command that takes a variable name as an argument, and does something to it.
"upvar" is explicit, there's no guesswork involved there. But how would you handle a lambda with a body like this?
if {$foo == 1} { puts $bar } else { puts $baz }
given that all the {}s are just non-expanded string literals, and what makes them be interpreted as code is the implementation of "if"? Sure, you can hardcode "if"; but the whole point of Tcl is that syntax constructs like that can be easily implemented as a library, which breaks if you have to special-case them in lambdas for them to work.
Now if you used upvar, this kinda sorta works, but only so long as the lambda is immediately invoked by the function that it was passed to. If you want to pass it on, you'd have to wrap it in another lambda. And, of course, this only works for lambdas that don't escape.
I'm not sure that's an appropriate phrasing. The word "choice" implies that alternatives are available, and there a decision was part of the selection. I think "Has been used for the language that momentum has demanded" might be more accurate. ;)
Synopsys used to have their own scripting language that was so limited you couldn't define functions. You had to resort to setting variables as pretend arguments and continually including another script for any code you needed to invoke multiple times without duplication.
Checkout Scala. It allows you to do pretty much the same, i.e. the syntax is so flexible that you can even make 'puts "Hello World!"' work and implement your own "if" that looks like the builtin one. It also has first-class lambdas and the other stuff that you asked for, but obviously there are also differences.
Scala runs on the JVM, which makes it not at all suited for scripting. Scala also seems to have something of a racism/white supremacy/misogyny problem.
> Scala runs on the JVM, which makes it not at all suited for scripting
The JVM is one target. You can also generate and run javascript code or create a native binary. But it will not be as smooth as e.g. python, so I can't really recommend that.
> Scala also seems to have something of a racism/white supremacy/misogyny problem.
I'm a member of that community for a long time and I really don't think so. I always thought the Scala community was a rather welcoming one. There certainly have been incidents in this community, just like they have been in other communities, but I don't think "Scala" has more of a problem here than any other language.
Re [1]: the libraries that delete an object when it goes out of scope use Tcl's built-in "trace" command, which lets you execute code when a command terminates or when a variable is deleted, among other events. So no, it's all done in pure Tcl.
Oh it looks like I misremembered. I thought the API I had used was something like `set doc [foo parse $input]` and the document would go away when the variable does. I found it, it's the tDOM library, and it was actually `dom parse $xml doc` that does the automatic freeing, whereas `set doc [dom parse $xml]` requires manual deletion. So it probably is using trace.
I love the language as well, we did lots of cool stuff with Tcl in 1998 - 2003, we were already doing our own Rails, but a tiny startup in Lisbon does not rock the world the same way.
However that experience lived on and gave birth to OutSystems.
> I just wish there was a Tcl that was redesigned from a modern perspective. Give me first-class lambdas. Give me unnamed objects that get destroyed automatically (with a callback) when out of scope[1] so I don’t have to use a weird namespace hack to make the object and rely on the caller to invoke the destroy method.
If you take a look at jimtcl (minimalistic reimplementation of tcl, started by antirez - he of redis fame), it did take some steps towards the above.
- unified arrays with dicts
- proper lambdas
- (sort of) closures
In many ways, I personally feel its a shame that jimtcl has felt the need to tie itself down to tcl-compatibility as much as it has. A few more warts could have been fixed along the way.
Anyway, check it out. It's manpage is quite comprehensive
I believe a lot of what makes Tcl the way it is is precisely because it does things different from most other languages with regards to closures, lambdas, scoping etc.
If Tcl had the other stuff, would it need upvar/uplevel or its funny braces and substitution rules? I'm no expert at Tcl but when I learnt it (mainly to use tkinter) I really had to rewire my brain to think about something as simple as _variables_ so differently
Imagine the timeline where Tcl, Scheme, xlisp, or any of the dozen other existing, mature, available, non-turrible programming languages was chosen for DHTML.
Versus the poorly conceived, poorly implemented, from scratch rush job based on a fundamental misunderstanding of Self.
In fact, we could have ended up with Self. Imagine that timeline.
I’ve always loved Tcl because of its absolute purity and simplicity at the language level. I love how you can implement your own control flow operators that look like built-in ones.
Have you ever had to debug code written by someone who implemented their own control flow operators? Not fun. I've seen it done with C macros. Don't do that.
A poor choice in C indeed, but languages like Haskell or (it seems, don't know much about it myself) TCL are practically made for it. Debugging "own control flow operators" is nothing very special there, mostly because control flow operators themselves aren't very special in those languages.
From another way to look at it, nobody seems to bat an eye when using things like items.forEach() in, say, Node.js, even if this sort of thing is arguably much uglier to do in JavaScript vs. a "first class" functional language where common control flow operators are often just plain functions themselves.
Dynamic scoping (`local $foo`) is nothing special in Perl. It's also a construct which, like novel flow control operators, by its nature leads to code which is complicated to reason about and difficult to debug.
There are many programming language concepts which may be innovative and beautiful, but whose popularity wanes because of practical flaws. (EIAS is another.)
There is a world of difference between writing a control structure in C using macros... and Tcl. And even more so when you look at a proper Lisp. Being able to construct new control structures is amazing; I miss it when in other languages.
I totally got turned around on TCL. I hated it when I started as I was mostly programming in c#, but now after year of sporadic use I absolutely love it for it's simplicity when manipulating text files and strings.
>I’ve always loved Tcl because of its absolute purity and simplicity at the language level. I love how you can implement your own control flow operators that look like built-in ones.
The Tcl wiki is quite strange, it has interesting content but most pages are discussions like this one and it's hard to find what's relevant or even correct.
What's weird about the Tcl syntax is that there is barely any. It's very dynamic so only the top-level of a file is parseable as a list of commands, anything else like a procedure depends on the implementation of the command which can be overridden at any time.
It's even used by built-in commands like expr and if which essentially implement a mini expression language which is different from Tcl.
Emacswiki is one active one still like that and the effort to replace it with MediaWiki was ultimately abortive. C2 is also big but I think mostly dead.
That is true but doesn't fully explain the OP's "what's relevant or even correct" bit. The truth is that OG wikis needed a special kind of moderators---affectionally called WikiGardener or even WikiMaster---who clean up things and deal with conflicts without removing values provided by prior authors but possibly removing the contents themselves. If this sounds absurd that's because it does; it is waaay harder to maintain than typical forums and even encyclopedic wikis popularized by Wikipedia. And without those kind of hard and thankless moderation non-encyclopedic wikis generally end up with being a dump of uncurated informations.
I have been writing some Tcl for the last couple of years, mostly in place of shell scripts. Going to Tcl wiki is always a surreal experience. I think this old comment captured it perfectly:
Really. It's been a couple of decades, but when I used to do a lot of Tcl programming, I too noticed that coding in Tcl had a certain Lispish quality to it.
I’ve seen Tcl used to write testing systems a long time ago, but never really worked with it.
Tangentially, I found that the primary author of SQLite and Fossil SCM, D Richard Hipp, was also a core team member of Tcl. The server side of Fossil uses a minimal dialect of Tcl and the build/test system is built with Tcl. [1]
My most miserable programming experience was with Tcl. It was used as a plugin language for a heat transfer application.
Maybe it allows for some "clever" code or "powerful" concepts but for its use in that application, it was just not straightforward. It was used among a pretty large team and it just felt so messy and chaotic. At that scale I would want type checking and the ability to not have to think "okay is this a variable or is it getting interpolated in some other code, or what?"
As embeddable scripting languages go it's pretty straightforward IME? I agree that you probably want a safer language for building large systems (Tcl is significantly older than OCaml, which I'd argue is the language that popularised non-cumbersome type systems), but considered as a language in the Perl/Python/Ruby/Lua kind of space, Tcl is reasonably clean IMO.
While we're at it - Tcl-like languages make for pretty decent embedded scripting solutions if performance is not a major requirement. There's one called LIL that's literally just a single .c file (and the accompanying header) for ease of embedding:
So far as I can tell, the reason why they're so small is because they implement "everything is a string" literally - so e.g. appending to a list is basically concat. This is good enough as a proof of concept, and neatly showcases just how simple the language itself really is; but perf would be abysmal even by Tcl standards as soon as you get more than a few dozen items per list.
LIL tries to be a minimalist but practical embeddable implementation, so it has basic optimizations like representing lists more efficiently internally. The price, of course, is code size, but also C API complexity.
Tcl is a better shell scripting language. Tcl's escaping is simple and consistent, while bash is almost impossible to grok -- just too many implicit rules to fit in my head.
> Tcl is a macro language, it derives from macro-assemblers, the C preprocessor, and older, text-oriented languages like TRAC
Then why doesn’t it have macros? This is one of my biggest complaints with tcl, the other being that you have to escape newlines in forms. This makes much less pleasant then lisp.
For example if I have this Tcl code
[set foo [foofn bar [barfn baz [bazfn quz 1]]]]
I would love to write it like this
[set foo
[foofn bar
[barfn baz
[bazfn quz 1]]]]
But I can’t without escaping every newline (you better hope you didn’t forget one). If tcl had macros I could write it like this
I dislike that too about most of the languages which are in some way white-space sensitive. (Python as the go-to worst example, but other modern languages where ; or a statement termination value is optional also fit.)
Newlines only can work, but then the question revolves around implicit newline escapes (if there's an unclosed statement) versus a required escape and things can quickly get out of hand and readability if really long strings need to be passed around; like with nightmare long SQL queries or default / example config file blocks.
The hardest part to understand when writing TCL "macros", is that it's all strings. You might be used to "inputs are s-expressions" from lisp, or "inputs are closure objects with pointers to code and local variables", but not in TCL. {} produces a string parameter. It is interpreted as an expression to be evaluated in parent context when passed to "uplevel". So "proc" takes 3 strings as arguments, and syntactically no different from "regexp".
It will work because the value passed to proc multiline within the curly braces is treated as a string without substitutions, that value is then passed to the regsub command which replaces all newlines in the string with spaces, then the result is passed to uplevel, which is basically "eval this string in the scope context where multiline was called". It's uplevel that does all the variable substitution, using any values defined in the scope where multiline was executed.
Second, since your regsub isn't really replacing newlines (except the first), the example isn't so good - your "set foo ..." works without the multiline proc because the opening square brackets are on the same line as the commands. So this also works: jdoodle.com/ia/jnc
Third, if you do change the example so that square brackets are on their own lines, as in the original request, then your version would work (with "regsub -all") but it might strip too much - like newlines inside nested quoting. See, for example: jdoodle.com/ia/jne
In Tcl newlines only have special meaning at the end of a command -- at the beginning of a command they're just whitespace. So you can break up a long line as follows:
Thanks for that great puzzle! Took me a few minutes to figure out what was going on with it. Indeed very clever--and amusing. I concur Tcl is often awesome and at times also terribly irksome. I suppose the same could be said of Lisps in general. (Like many others I'd say Tcl belongs in the family of Lisp and its direct descendants.)
I clicked through to these comments expecting that it would be full of discussion about eggdrop bots, as that was where I experienced tcl. To learn that it is heavily used at places such as Intel for tooling controls was very interesting to me.
I still maintain a bot that began life as a mirc script, then a long time as an eggdrop bot, then ported to python based slackbot, and now most recently a python based discord bot.
Over the last 25 years it really has been a huge part of the way my group of friends interact. I recall the sheer wonder that I experienced when I learned about eggdrop bots and the huge amount that I learned while flailing around with tcl.
> To learn that it is heavily used at places such as Intel for tooling controls was very interesting to me.
Same! It is quite interesting, I have not heard of it before.
My first experience with Tcl was through Eggdrop. I was really young at the time! I remember those mIRC scripts and different mIRC clients or something, they may have been through plugins? I don't recall, but I remember after installing them my mIRC looked very different and so on.
After Eggdrop came Supybot which was really popular at the time, then after that I just made my own. I have a Discord bot as well, I made it myself, although it is written in Lua because I made it around the time I was tinkering around with the language.
A more recent experience with Tcl was modifying Tclsh to make it suitable to be used as a replacement for bash. It works quite well now, I would say. Actually, a friend of mine did most of the job but still!
Figured this thread would be full of eggdrop comments. So many good memories writing TCL scripts back in the 90's. Really happy to see the project is still developed, definitely a part of internet history for me.
The CMake language has the same absence of syntax, and everything is a command as well - like assignments with set(). Oh, and all variables are strings, too like in Tcl. (In CMake, even lists of strings are strings. Don't ask.)
The CMake language was created in a haphazard way as well, with some other important problems that Tcl probably doesn't have.
> The CMake language has the same absence of syntax
Ironically, if I recall correctly, cmake came about from a Kitware medical imaging project that was heavily leaning on Tcl, yet when their build system needed a DSL they decided to hack something up from scratch :/
Article doesn't do a good job of explaining why Tcl syntax isn't weird.
I had to code in it once, can't remember what, something trivial; but I had to learn Tcl. It was basically an obstacle to my work. I never needed to use it again, so I've forgotten it, so I wasted my time learning that language.
How so? Tcl has namespaces, a standard lib, runtime introspection, an OO system, threads, asynchronous programming capabilities via event loop, a GUI framework and is cross platform. Have you actually used it?
Tclsh is not really suitable for use directly as a general-purpose shell. TkCon gets a bit closer - https://wiki.tcl-lang.org/page/Tkcon - but still not great, e.g. if you start a long-running program you will only see its output when it terminates.
I wrote a semi-graphical general-purpose shell in Tcl/Tk for my own use - https://wiki.tcl-lang.org/page/gush . I used this as my main shell at work for 12 years, and only stopped because I retired last year.
I’ve always loved Tcl because of its absolute purity and simplicity at the language level. I love how you can implement your own control flow operators that look like built-in ones. I love how easily you can make DSLs.
I just wish there was a Tcl that was redesigned from a modern perspective. Give me first-class lambdas. Give me unnamed objects that get destroyed automatically (with a callback) when out of scope[1] so I don’t have to use a weird namespace hack to make the object and rely on the caller to invoke the destroy method. Give me a stdlib that offers more functional stuff. Heck, give me async/await. And more too.
But even without all of that, Tcl is still a wonderful language and I often lament how it’s mostly died out.
[1] There’s at least one library I’ve used that actually does this, but AFAIK there’s no way to do this in pure Tcl so it must be a feature of the C API. Also I’m not sure if it actually tracks the object or just the variable it’s assigned to.
Tcl is one of the most underrated programming languages. I have a friend who is working at Intel as tool development engineer (equivalent to software engineer), and Tcl is being used extensively at Intel for chip tooling and design since a very long time.
The modern version of Tcl will be great and a seamless integration with the OS shell will probably the killer niche application. Hopefully Oil Shell can modernize Tcl to some extent and incorporate the best features of Tcl with excellent OS shell integration capabilities.
I haven't gotten around to looking into Oil Shell. Is it actually drawing any inspiration from Tcl? The "A Tour of the Oil Language" page[1] references Ruby and Python but not Tcl.
[1] http://www.oilshell.org/release/latest/doc/oil-language-tour...
Not yet but I've mentioned to Andy (Oil Shell author) regarding the importance of adding command features into Oil Shell and he is actually looking into it [1]. Hopefully he will draw some inspirations from Tcl. Imagine a Cisco like networking command wrapper on top of Linux OS with eBPF backend. It will turn Linux into a ready-made formiddable open source networking OS without the need for NX-OS [2].
If you are looking into a modern version of Tcl, there is TIL [3]. It's a Tcl inspired new scripting language on top of D language. By having D language as the foundation it can perform all the features that you've requested including lambdas, async and then some more [4].
[1]https://news.ycombinator.com/item?id=28552998
[2]https://en.wikipedia.org/wiki/Cisco_NX-OS
[3]https://news.ycombinator.com/item?id=27167762
[4]https://dlang.org/spec/expression.html
TIL looks interesting, but also both too simple and too complicated. What's the difference between a SubList and a SimpleList? Why does it require SimpleLists instead of using variadic arguments (e.g. why [math ($a + $b)] instead of [math $a + $b])? And why does the author explicitly disallow multiple spaces in a row outside of indentation when this is a purely artificial limitation and does not simplify anything?[1]
But also, why can't I write modules in TIL? It's based on D, sure, but I shouldn't need to write and compile something in D just to be able to have some means of namespacing things. This very much feels like "I want to write most of my stuff in D but I just want to be able to whip up short scripts that for whatever reason I don't want to write in D".
And of course it seems to be ditching the pure simplicity of Tcl. It's like Tcl in that it's a command language and borrows syntax from Tcl, but it seems to be missing the fundamental concepts of Tcl.
[1] The author seems to hate the idea of people lining up equals signs in code or things like that. Ok sure, you can have a style preference. But that's what a code formatter is for, not an artificial and unprecedented level of whitespace significance.
As a tool development engineer, it's only normal that your friend is using Tcl extensively: there is no other option.
The real question is: would they still be using it when given the choice?
For me, Tcl is a cancer that just doesn't want to die.
I feel like there is an opening for an EDA company or startup to really embrace Python and provide a Python interface / command line to their tooling.
Tcl is deeply rooted in EDA and chip design industry, too many code were written with it. It's like C++, no matter how fancy/modern rust is, there is no way it's going to replace C++ for the next few decades.
One way is to provide Python(python stdlib might be enough for simplicity) gradually, e.g. providing a Python binding for Tcl so for the future, you can write all new piece in Python, and it is either used directly or converted to Tcl behind-the-scene.
I don't think that opening is there. Tcl may be a terrible scripting language, its usage is not a factor in the value of this or that EDA tool.
If a startup wants to displace an entrenched tool from Synopsys or Cadence, the underlying algorithms (millions of lines of C++ code) must be better. (And once that happens, they'll be acquired by one of these 2.)
I think you are probably right, it is not a make or break factor. Especially when it comes to back-end flows, PPA/TAT/capacity/design closure are the key metrics and the big 2.5 have large moats in these areas.
On the front-end side though, areas like verification, debug, and various lint checks, I think there is more opportunity for smaller players to introduce point tools with better customizability & scripting interfaces.
Ultimately I think the industry is stuck in a local maximum, where the frictional costs of rethinking overall methodology, e.g. from SystemVerilog to Chisel, is too high to justify the change, even if the end result would be marginally (or greatly) better. (Not an endorsement of Chisel, just an example.)
There is Python support in some EDA frameworks, but especially in the digital domain, Tcl is still the gold standard. On the one side this means there are tons of Tcl code around and any experienced engineer is well versed in Tcl. Also, electrical engineers are usually very focussed on electric engineering. While being being very intelligent, they are often not into programming, so keeping things very simple is an advantage to them.
May be true of analog EEs, but more than half of EEs I work with, and all the good ones, can script well. They use perl/python for day to day, and tcl to interact with tools. The number who can use tcl well is a lot lower than those who can script in general though
Replace one cancerous language with another one. Please no.
python never seemed like a good fit for me, because it's just not suited for DSLs. It's too imperative and has too much built in syntax, you'll end up in callback hell as soon as you need an actually descriptive DSL.
Tcl is also at the heart of some F5 Networks products
My team and I write TCL based iRules for F5 devices. We do it a couple of times a week. They are a very nice tool.
Sometimes we've had to use it like an emergency super power to patch http applications on the wire.
The reason why some financial institutions are working right now is because of some iRules we wrote a couple for years ago.
If your company uses F5 boxes, you should befriend your F5 admins.
I used a lot of TCL for stuff in the legacy networking space and while the language always seemed to be easy to reason about I seem to remember my main frustration was tracking down syntax errors as runtime problems. Dunno if I was just using the tooling wrong though.
Tcl's usage at Intel is not particularly special. Perl/Tcl were the workhorses of most (hardware) engineering companies. Back in 2010, a close friend who was working for one such company said that putting Python on your resume won't help you as none of the managers had heard of it. When I joined work around that time, I discovered he wasn't exaggerating much.
While Tcl may be a particularly good language for somethings, well over 90% of the teams I've found using it do so for legacy reasons, and they correlate very highly with poor SW engineers with even poorer SW practices. When I interview for jobs, I steer clear of any that lists Tcl. Not because of the language, but because it's a fairly useful signal about the quality of the team.
All EDA uses tcl, so no surprise Intel would too. A while back I saw some vendors tried to make perl or python an option, but tcl is the de facto language in this space those vendors often abandoned the having multiple scripting interfaces and fell back on tcl. It's quirky, but functional. I like the one line composability
You can't really get full-fledged closures without ditching the fundamental concept of Tcl, which is that "everything is a string". It's the same problem as OOP, or rather, object lifetime management - you can have that in Tcl, but only as long as you manage the lifetimes manually. For closures, this negates most of the benefits they provide.
It would work if they basically copy the current value of the identifiers (i.e. by-value, not by-reference). Or by capturing the procedure context. You can synthesize the variable capture by hand but that would indeed force everything into a string, which is a potential performance issue and would also break any sort of automatic lifetime management.
Even just making it easier to have lambdas reference variables from their creation scope while the creation scope is still on the stack would be an improvement. Today you can use [info level] and then construct a lambda body that uses [upvar #$level …] to reference variables from the original scope, but it's a PITA. Or you could cram the level into a temporary namespace variable to avoid having to manually construct a body string, but it's still a PITA.
[list apply {args body} $value1 $value2 ...] can be used to capture values without actually generating a string because even though conceptually a list is a string, it's actually stored internally as a list.
The basic problem is, how do you even know that something is a variable reference? Remember, everything is a string. This means that the body of the lambda is also a string. Now you can require that it's an eval'able string at the point of creation, and parse it to identify any $var references. But you can only do that for the topmost level of the lambda, because it can contain nested strings - and you don't know which of those strings are code, and which aren't (since it's really the command to which the string is passed that decides to treat something as code or not).
Furthermore, if you process $ like that, this won't work for any other command that takes a variable name as an argument, and does something to it.
> The basic problem is, how do you even know that something is a variable reference?
Isn't that by convention? Like there is a "upvar" to grab values of the prior stack.
"upvar" is explicit, there's no guesswork involved there. But how would you handle a lambda with a body like this?
given that all the {}s are just non-expanded string literals, and what makes them be interpreted as code is the implementation of "if"? Sure, you can hardcode "if"; but the whole point of Tcl is that syntax constructs like that can be easily implemented as a library, which breaks if you have to special-case them in lambdas for them to work.
Now if you used upvar, this kinda sorta works, but only so long as the lambda is immediately invoked by the function that it was passed to. If you want to pass it on, you'd have to wrap it in another lambda. And, of course, this only works for lambdas that don't escape.
Tcl has been used for decades as the language of choice in the electronic design automation world.
I often lament how it has not died out. It's demise is long overdue...
> as the language of choice
I'm not sure that's an appropriate phrasing. The word "choice" implies that alternatives are available, and there a decision was part of the selection. I think "Has been used for the language that momentum has demanded" might be more accurate. ;)
Synopsys used to have their own scripting language that was so limited you couldn't define functions. You had to resort to setting variables as pretend arguments and continually including another script for any code you needed to invoke multiple times without duplication.
Checkout Scala. It allows you to do pretty much the same, i.e. the syntax is so flexible that you can even make 'puts "Hello World!"' work and implement your own "if" that looks like the builtin one. It also has first-class lambdas and the other stuff that you asked for, but obviously there are also differences.
Scala runs on the JVM, which makes it not at all suited for scripting. Scala also seems to have something of a racism/white supremacy/misogyny problem.
>Scala also seems to have something of a racism/white supremacy/misogyny problem.
lol what
> Scala runs on the JVM, which makes it not at all suited for scripting
The JVM is one target. You can also generate and run javascript code or create a native binary. But it will not be as smooth as e.g. python, so I can't really recommend that.
> Scala also seems to have something of a racism/white supremacy/misogyny problem.
I'm a member of that community for a long time and I really don't think so. I always thought the Scala community was a rather welcoming one. There certainly have been incidents in this community, just like they have been in other communities, but I don't think "Scala" has more of a problem here than any other language.
Why do you say that the JVM is not suitable for scripting?
Re [1]: the libraries that delete an object when it goes out of scope use Tcl's built-in "trace" command, which lets you execute code when a command terminates or when a variable is deleted, among other events. So no, it's all done in pure Tcl.
Oh it looks like I misremembered. I thought the API I had used was something like `set doc [foo parse $input]` and the document would go away when the variable does. I found it, it's the tDOM library, and it was actually `dom parse $xml doc` that does the automatic freeing, whereas `set doc [dom parse $xml]` requires manual deletion. So it probably is using trace.
This is how the "defer" [0] command (from tcllib) in Tcl works.
[0] https://core.tcl-lang.org/tcllib/doc/tcllib-1-19/embedded/ww...
I love the language as well, we did lots of cool stuff with Tcl in 1998 - 2003, we were already doing our own Rails, but a tiny startup in Lisbon does not rock the world the same way.
However that experience lived on and gave birth to OutSystems.
> I just wish there was a Tcl that was redesigned from a modern perspective. Give me first-class lambdas. Give me unnamed objects that get destroyed automatically (with a callback) when out of scope[1] so I don’t have to use a weird namespace hack to make the object and rely on the caller to invoke the destroy method.
If you take a look at jimtcl (minimalistic reimplementation of tcl, started by antirez - he of redis fame), it did take some steps towards the above.
- unified arrays with dicts
- proper lambdas
- (sort of) closures
In many ways, I personally feel its a shame that jimtcl has felt the need to tie itself down to tcl-compatibility as much as it has. A few more warts could have been fixed along the way.
Anyway, check it out. It's manpage is quite comprehensive
I believe a lot of what makes Tcl the way it is is precisely because it does things different from most other languages with regards to closures, lambdas, scoping etc.
If Tcl had the other stuff, would it need upvar/uplevel or its funny braces and substitution rules? I'm no expert at Tcl but when I learnt it (mainly to use tkinter) I really had to rewire my brain to think about something as simple as _variables_ so differently
It's not dead yet. There's still AOL Server [1] and there's at least one game [2] that uses TCL that is online. :)
1. http://www.aolserver.org/ 2. http://www.carnageblender.com/
Long live aolserver, Naviserver [1] is the fork that's still in very active development.
1. https://wiki.tcl-lang.org/page/NaviServer
Imagine the timeline where Tcl, Scheme, xlisp, or any of the dozen other existing, mature, available, non-turrible programming languages was chosen for DHTML.
Versus the poorly conceived, poorly implemented, from scratch rush job based on a fundamental misunderstanding of Self.
In fact, we could have ended up with Self. Imagine that timeline.
Worst is better.
The Web would definitely not be better had Tcl been the language of the Web instead of JavaScript. At least JS has working closures.
TCL would evolve a lot.
I’ve always loved Tcl because of its absolute purity and simplicity at the language level. I love how you can implement your own control flow operators that look like built-in ones.
Have you ever had to debug code written by someone who implemented their own control flow operators? Not fun. I've seen it done with C macros. Don't do that.
That sounds hilarious though
A poor choice in C indeed, but languages like Haskell or (it seems, don't know much about it myself) TCL are practically made for it. Debugging "own control flow operators" is nothing very special there, mostly because control flow operators themselves aren't very special in those languages.
From another way to look at it, nobody seems to bat an eye when using things like items.forEach() in, say, Node.js, even if this sort of thing is arguably much uglier to do in JavaScript vs. a "first class" functional language where common control flow operators are often just plain functions themselves.
Dynamic scoping (`local $foo`) is nothing special in Perl. It's also a construct which, like novel flow control operators, by its nature leads to code which is complicated to reason about and difficult to debug.
There are many programming language concepts which may be innovative and beautiful, but whose popularity wanes because of practical flaws. (EIAS is another.)
Dynamic scoping, combined with variable naming conventions, is a beautiful way to achieve “dependency injection” with a minimal amount of complexity.
Like any feature, it can be abused, but it can be very elegant.
There is a world of difference between writing a control structure in C using macros... and Tcl. And even more so when you look at a proper Lisp. Being able to construct new control structures is amazing; I miss it when in other languages.
I re-implemented the wheel in Forth, too, before! :)
I totally got turned around on TCL. I hated it when I started as I was mostly programming in c#, but now after year of sporadic use I absolutely love it for it's simplicity when manipulating text files and strings.
>I’ve always loved Tcl because of its absolute purity and simplicity at the language level. I love how you can implement your own control flow operators that look like built-in ones.
Then you will like Lisp, Smalltalk and Forth!
Just [trace add variable unset]. Can be done from a script.
The Tcl wiki is quite strange, it has interesting content but most pages are discussions like this one and it's hard to find what's relevant or even correct.
What's weird about the Tcl syntax is that there is barely any. It's very dynamic so only the top-level of a file is parseable as a list of commands, anything else like a procedure depends on the implementation of the command which can be overridden at any time.
It's even used by built-in commands like expr and if which essentially implement a mini expression language which is different from Tcl.
That's what wikis originally were. Wikipedia's curated ontology is an exception that only looks like the rule because most OG wikis have died off.
Emacswiki is one active one still like that and the effort to replace it with MediaWiki was ultimately abortive. C2 is also big but I think mostly dead.
That is true but doesn't fully explain the OP's "what's relevant or even correct" bit. The truth is that OG wikis needed a special kind of moderators---affectionally called WikiGardener or even WikiMaster---who clean up things and deal with conflicts without removing values provided by prior authors but possibly removing the contents themselves. If this sounds absurd that's because it does; it is waaay harder to maintain than typical forums and even encyclopedic wikis popularized by Wikipedia. And without those kind of hard and thankless moderation non-encyclopedic wikis generally end up with being a dump of uncurated informations.
I have been writing some Tcl for the last couple of years, mostly in place of shell scripts. Going to Tcl wiki is always a surreal experience. I think this old comment captured it perfectly:
https://news.ycombinator.com/item?id=9098859
Equal time:
https://news.ycombinator.com/item?id=9102916
It's an accessible Lisp with square brackets.
Not really. References are pretty fundamental to Lisp, but are (at best) bolted on in Tcl.
Really. It's been a couple of decades, but when I used to do a lot of Tcl programming, I too noticed that coding in Tcl had a certain Lispish quality to it.
I wrote a blog post last year trying to explain the under-appreciated radical minimalism of Tcl, and in particular the decoupling of syntax and semantics - https://colin-macleod.blogspot.com/2020/10/why-im-tcl-ish.ht...
Great read, thanks! It also received 100+ comments on HN: https://news.ycombinator.com/item?id=24897326
I’ve seen Tcl used to write testing systems a long time ago, but never really worked with it.
Tangentially, I found that the primary author of SQLite and Fossil SCM, D Richard Hipp, was also a core team member of Tcl. The server side of Fossil uses a minimal dialect of Tcl and the build/test system is built with Tcl. [1]
[1]: https://fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki
My most miserable programming experience was with Tcl. It was used as a plugin language for a heat transfer application.
Maybe it allows for some "clever" code or "powerful" concepts but for its use in that application, it was just not straightforward. It was used among a pretty large team and it just felt so messy and chaotic. At that scale I would want type checking and the ability to not have to think "okay is this a variable or is it getting interpolated in some other code, or what?"
As embeddable scripting languages go it's pretty straightforward IME? I agree that you probably want a safer language for building large systems (Tcl is significantly older than OCaml, which I'd argue is the language that popularised non-cumbersome type systems), but considered as a language in the Perl/Python/Ruby/Lua kind of space, Tcl is reasonably clean IMO.
While we're at it - Tcl-like languages make for pretty decent embedded scripting solutions if performance is not a major requirement. There's one called LIL that's literally just a single .c file (and the accompanying header) for ease of embedding:
http://runtimeterror.com/tech/lil/
My favorite implementations are partcl[1] (600 lines) and Picol[2] (500 lines by Salvatore Sanfilippo).
[1] https://github.com/zserge/partcl
[2] http://oldblog.antirez.com/post/picol.html
So far as I can tell, the reason why they're so small is because they implement "everything is a string" literally - so e.g. appending to a list is basically concat. This is good enough as a proof of concept, and neatly showcases just how simple the language itself really is; but perf would be abysmal even by Tcl standards as soon as you get more than a few dozen items per list.
LIL tries to be a minimalist but practical embeddable implementation, so it has basic optimizations like representing lists more efficiently internally. The price, of course, is code size, but also C API complexity.
Tcl is a better shell scripting language. Tcl's escaping is simple and consistent, while bash is almost impossible to grok -- just too many implicit rules to fit in my head.
> Tcl is a macro language, it derives from macro-assemblers, the C preprocessor, and older, text-oriented languages like TRAC
Then why doesn’t it have macros? This is one of my biggest complaints with tcl, the other being that you have to escape newlines in forms. This makes much less pleasant then lisp.
For example if I have this Tcl code
[set foo [foofn bar [barfn baz [bazfn quz 1]]]]
I would love to write it like this
[set foo
[foofn bar
[barfn baz
[bazfn quz 1]]]]
But I can’t without escaping every newline (you better hope you didn’t forget one). If tcl had macros I could write it like this
[set foo
But instead I end up create a bunch of temporary variables so that my lines don’t get super long. It just feels less elegant then it could have been.
I dislike that too about most of the languages which are in some way white-space sensitive. (Python as the go-to worst example, but other modern languages where ; or a statement termination value is optional also fit.)
Newlines only can work, but then the question revolves around implicit newline escapes (if there's an unclosed statement) versus a required escape and things can quickly get out of hand and readability if really long strings need to be passed around; like with nightmare long SQL queries or default / example config file blocks.
It's a bit odd, because you can skip newlines in many cases.
This works:
TCL does not need macros because every function can be a macro. Here is an answer to your question:
Use it as following:
try it in action: https://jdoodle.com/ia/jka
The hardest part to understand when writing TCL "macros", is that it's all strings. You might be used to "inputs are s-expressions" from lisp, or "inputs are closure objects with pointers to code and local variables", but not in TCL. {} produces a string parameter. It is interpreted as an expression to be evaluated in parent context when passed to "uplevel". So "proc" takes 3 strings as arguments, and syntactically no different from "regexp".
But that won't work if I am using variables correct? because {} does not interpolate.
sure it will, "uplevel" will interpolate for you.
There is a link to online playground, I encourage you to visit it and try various changes. No installation or signin required.
I stand corrected, it works just as you describe.
It will work because the value passed to proc multiline within the curly braces is treated as a string without substitutions, that value is then passed to the regsub command which replaces all newlines in the string with spaces, then the result is passed to uplevel, which is basically "eval this string in the scope context where multiline was called". It's uplevel that does all the variable substitution, using any values defined in the scope where multiline was executed.
This is not quite right...
First off I think you want "regsub -all ...".
Second, since your regsub isn't really replacing newlines (except the first), the example isn't so good - your "set foo ..." works without the multiline proc because the opening square brackets are on the same line as the commands. So this also works: jdoodle.com/ia/jnc
Third, if you do change the example so that square brackets are on their own lines, as in the original request, then your version would work (with "regsub -all") but it might strip too much - like newlines inside nested quoting. See, for example: jdoodle.com/ia/jne
In Tcl newlines only have special meaning at the end of a command -- at the beginning of a command they're just whitespace. So you can break up a long line as follows:
set foo [
foofn bar [
]
Elegant!
I find this Tcl quine quite amazing:
I think it tells a lot about the weirdness and awesomeness of the language at the same time.
Thanks for that great puzzle! Took me a few minutes to figure out what was going on with it. Indeed very clever--and amusing. I concur Tcl is often awesome and at times also terribly irksome. I suppose the same could be said of Lisps in general. (Like many others I'd say Tcl belongs in the family of Lisp and its direct descendants.)
Who remembers Eggdrop, the IRC bot?
https://en.wikipedia.org/wiki/Eggdrop
https://www.eggheads.org
https://www.tclarchive.org (Blast from the past!)
I was surprised to see that it is still under development.
After Eggdrop, it was Supybot that got really popular, although it is written in Python.
I clicked through to these comments expecting that it would be full of discussion about eggdrop bots, as that was where I experienced tcl. To learn that it is heavily used at places such as Intel for tooling controls was very interesting to me.
I still maintain a bot that began life as a mirc script, then a long time as an eggdrop bot, then ported to python based slackbot, and now most recently a python based discord bot.
Over the last 25 years it really has been a huge part of the way my group of friends interact. I recall the sheer wonder that I experienced when I learned about eggdrop bots and the huge amount that I learned while flailing around with tcl.
> To learn that it is heavily used at places such as Intel for tooling controls was very interesting to me.
Same! It is quite interesting, I have not heard of it before.
My first experience with Tcl was through Eggdrop. I was really young at the time! I remember those mIRC scripts and different mIRC clients or something, they may have been through plugins? I don't recall, but I remember after installing them my mIRC looked very different and so on.
After Eggdrop came Supybot which was really popular at the time, then after that I just made my own. I have a Discord bot as well, I made it myself, although it is written in Lua because I made it around the time I was tinkering around with the language.
A more recent experience with Tcl was modifying Tclsh to make it suitable to be used as a replacement for bash. It works quite well now, I would say. Actually, a friend of mine did most of the job but still!
Figured this thread would be full of eggdrop comments. So many good memories writing TCL scripts back in the 90's. Really happy to see the project is still developed, definitely a part of internet history for me.
Yeah! I was surprised to see it is still under development after so long! Pleasant memories tied to all of it. :)
The CMake language has the same absence of syntax, and everything is a command as well - like assignments with set(). Oh, and all variables are strings, too like in Tcl. (In CMake, even lists of strings are strings. Don't ask.)
The CMake language was created in a haphazard way as well, with some other important problems that Tcl probably doesn't have.
CMake is the modern MUMPS.
> The CMake language has the same absence of syntax
Ironically, if I recall correctly, cmake came about from a Kitware medical imaging project that was heavily leaning on Tcl, yet when their build system needed a DSL they decided to hack something up from scratch :/
It's amusing how that pattern was adopted in Javascript (well, C/Java really I guess) as an optimisation.
But your JS code has a bug in it while the Tcl code does not.
I love Tcl. The community is awesome as well.
Article doesn't do a good job of explaining why Tcl syntax isn't weird.
I had to code in it once, can't remember what, something trivial; but I had to learn Tcl. It was basically an obstacle to my work. I never needed to use it again, so I've forgotten it, so I wasted my time learning that language.
Tcl is too much like shell script programming. It's one of those "take one basic concept and use it for everything" systems.
Could be worse. Could be Forth.
>Tcl is too much like shell script programming.
How so? Tcl has namespaces, a standard lib, runtime introspection, an OO system, threads, asynchronous programming capabilities via event loop, a GUI framework and is cross platform. Have you actually used it?
Do any of you use tclsh as your shell?
Tclsh is not really suitable for use directly as a general-purpose shell. TkCon gets a bit closer - https://wiki.tcl-lang.org/page/Tkcon - but still not great, e.g. if you start a long-running program you will only see its output when it terminates.
I wrote a semi-graphical general-purpose shell in Tcl/Tk for my own use - https://wiki.tcl-lang.org/page/gush . I used this as my main shell at work for 12 years, and only stopped because I retired last year.