Why not? The majority of engineering professions say use the right tool for the job. Whatever is most cost-effective. Programming is one of few things where people try to do the opposite. Fortunately, many in the profession do right tool for the job philosophy at least for whole domains like OS code, web apps, and so on. Still an insistence on sticking to same language too much.
Python has incredible productivity and versatility with ease of learning. It's readable. It's easier to prototype things like verification tools in it than C. There's also tools like Hypothesis to test that. Then, there's tools to speed up the code like Cython.
If anything, Python would be a safer, faster way to write tooling for modifying or testing C code than C itself. One could also convert Python generated tests to a C library of tests with automatic validation of data types, coverage, etc. So, I think it would make sense if someone did anything from prototyping C in Python to using it to test C code.
And do note Ive enjoyed reading your experimentation with macros in C. I learned about defer through it. Just commenting on that one point.
How is python safer than C? I deal with a code base that’s 10x more C++ than python, and when I’m on call to babysit the test clusters, >99% of the type errors are in python. With the occasional oddball exception to break the rule, the C++ type/memory errors are easier to track down to boot. (Python has its doozies too, BTW)
I think this is because writing python code is like writing C++ without templates, but all the integer types declared “auto” and all the pointers void.
The main difference is that the C++ compiler still* does more type checking than python even after you abuse the warning flags enough to get it to stop gently telling you to change professions!)
Reading it reminds me of when I accidentally got my emacs syntax highlighting stuck at “white on white” for a bunch of stuff, but I digress.
I guess I find it to be a completely unproductive, unreadable, and unmaintainable language. It’s OK for throw away prototypes, but I’ve never seen a team throw away a python prototype and rewrite it in another language. I know of python projects that failed and were [repeatedly] rewritten by the same team in python, and I know of python teams that failed and got swapped out for another language, but that doesn’t count. Part of the problem with the python prototyping story is that python wants to own the event loop and asynchrony / thread model, and changing those are often the main reason to swap languages — the next step is always a rewrite in my experience.
Wow. Long rant. One too many ‘assert “” != None’ errors this week. :-)
Its basic operations don't often crash a system or lead to hacks. It also promotes readable, concise code with a lot of FOSS utilities for things like testing (esp see Hypothesis). This both increases development pace and reduces defect according to about every study that's ever pitted a HLL against C. The best one I've seen in terms of apples to apples was this Ada and C comparison:
Note that Python isn't the main point or even what I brought up, though. mort's point read like you shouldn't develop C-related tooling in high-level languages like Python. I'm countering saying you should do as much work out of C as you can with even Python being beneficial. I give much better examples in my other response here:
TLDR: because you need to test the real thing, and using Python to test C code causes too many important differences between tests and production.
To test C code you need to compile it, and there’s no C compiler inside Python. Also if you have C++, sometimes you can abuse templates for similar effects, and Python doesn’t have C++ AST either.
C toolchains largely depend on environment: included & linked files, environment variables, compiler & linker switches, they all make a huge difference. You’ll spend much time replicating that in Python, and it’ll break when you’ll upgrade any part of toolchain (compiler/IDE/build automation/etc).
To a lesser extent, same applies to native binaries as well. If you’ll manage to use Python to produce tests, C compiler to build them, then Python to run them — the runtime environment will be different from the real one.
I think nickpsecurity has introduced two ideas, and you are only countering the more complex one. The ideas were,
* Use python as a preprocessor
* Use python to call into C.
To the first idea: the language you choose to use as your preprocessor. Instead of using C macros, you could have an alternate DSL that you transform into C. Then you compile this. Given what an awkward thing the C preprocessor is, I am surprised that it continues to hold mindshare against options like this. Awk is a better powerful transform tool, and easily compiled for any platform it is not already on.
To the other point. The complications you raise are real, but you can manage them away by setting your C project up to create a shared-object build. For example, it is trivial to use Racket to write tests against shared object files (once you know Racket). This still doesn't address the runtime issue you raise.
Separate issue. There is a set of debugging C-preprocessor macros with similar intent to the ones posted in OP published in "Learn C the Hard Way".
It’ll become harder to find developers, and longer for new ones to start being productive. Also it’ll become harder to do cross platform development because you’ll have to port, and then support, that custom pre-processing tools/DSL for every platform.
> Use python to call into C
Last time I’ve checked Python can’t import C headers. To call into C, you somehow need to negotiate function prototypes & calling conventions between the two languages. Regardless on how you do it manually or automatically, it’s very expensive to support. People do it when the have to (e.g. to interop Python with C libraries), but for tests, the approach is way too expensive for most project.
> it is trivial to use Racket to write tests against shared object files (once you know Racket)
I don’t know Racket but I doubt it’s trivial. SO libraries export C functions, they use C structures for arguments and return values, and these can be very complex. Pointers graph, function pointers, pointers to pointers, variable length structures, other advanced features are pain to use from any other language (besides languages specifically designed to be backward compatible, like C++, objective C, and maybe D). And if you need to test C++ code it’s even harder, unlike C it doesn’t have standardized ABI, i.e. the ABI changes between compilers and their versions.
You introduce a strong point with the issue moving structs over the barrier. When I do it, I create constructor functions in C, and only send pointers over the bridge between the non-C and C. I'm happy to do this, but recognise that it is boilerplatey, and should be seen as a different tradeoff choice rather than a trivial alternative.
Agree regarding C++ ABI also. I don't have much experience with C++. My usual approach is to use C for system calls and bare-minimum surrounds, and then use the wrapping approaches described above to get to a high-level language. This is why the struct issue didn't come to mind for me.
You nailed it. Also note that Python isn't my recommendation so much as what mort brought up that I'm countering with examples showing even it can help. If it was my choice, I'd pick a better HLL for these goals. Let's keep looking at this a bit, though, where I'll introduce those where they're useful.
PHP was a great example I didn't think of on preprocessor. It versus C's is either the 1st or 2nd most used one out there. On the high end, some "term-rewriting languages" can easily handle jobs like refactoring: TXL, Rascal, OMeta, Ohm. Alan Kay et al in STEPS project did a whole OS with graphics stack and all in a mere tens of thousands of lines of code using such a language. One can, as people do with Prolog and STEPS did, pick a general language that can be extended with meta facilities for DSL's or rewriting where opportunistic. Then, where that doesn't work out, you at least have general-purpose language to fall back on.
(Use Control-F to go to anything that says "STEPS" to find the reports. Work from the bottom up since it's chronological series.)
On your other point, your Racket example is one way it could work. I was also advocating in this thread just using Python itself to build tooling for its own benefits and ecosystem. A lot is already built. Use C for low-level software that strictly needs it if nothing else is available with HLL's like Python for stuff that doesn't need it. The language I've been telling C programmers to check out for scripting-like use is Nim. It looks like Python, has real macros, can call C, and compiles to C. Here's a comparison I just dug up:
I also doubt it will be harder to find C developers if tooling is written in HLL's. For one, they just have to use the tooling rather than write it. I doubt most C# developers extend Visual Studio, most Java developers extend NetBeans, and so on. If they do have to learn, using a language like Python or Nim should make low barrier to entry since even folks with no experience pick Python up quickly. A C-like subset of Nim or something similar will be just a lot like C with easier syntax and less stuff to worry about. If anything, productivity will go up over C like shown in about every language study ever done.
I have encouraged people to do C-like languages with safe defaults, cleaner semantics, a REPL, better macros, easy integration of C, and outputting C. That way, one can program in a cleaner language avoiding most headaches and accidents that come with C being designed for 1970's hardware. Aside from Wirth's languages with C syntax, a good picture of what this might look like is Julia language which was femtolisp on the inside. Especially its seemless interoperation with C and Python.
Python has incredible productivity and versatility with ease of learning. It's readable. It's easier to prototype things like verification tools in it than C. There's also tools like Hypothesis to test that. Then, there's tools to speed up the code like Cython.
If anything, Python would be a safer, faster way to write tooling for modifying or testing C code than C itself. One could also convert Python generated tests to a C library of tests with automatic validation of data types, coverage, etc. So, I think it would make sense if someone did anything from prototyping C in Python to using it to test C code.
And do note Ive enjoyed reading your experimentation with macros in C. I learned about defer through it. Just commenting on that one point.