I enjoy both languages quite a bit, but a 4x improvement is more of a testament to python than it is to rust imo!
What I get from this is that the python interpreter is better than I would have guessed.
That is a lot of text for not determining why the new solution is faster. The only relevant part:
> Before our migration, the old pipeline utilized a C library accessed through a Python service, which buffered and bundled data. This was really the critical aspect that was causing our latency.
How much speed up would there have been if they moved to a Rust wrapper around the same C library?
Using something other than Python is almost always going to be faster. This Reddit post does not give any insights into which aspects of Python lead to small/large performance hits. They show that it was the right solution for them with ample documentation which is great, but they don't provide any generalizable information.
Noticed similar scale of improvement when testing the two on cloud functions.
Just comes down to whether you need speed of building it or speed of program
Could be renamed as 120ms to 30ms: Python to "AOT compiled language".
Am I the only one thinking "why wasn't it better?"
Guess I'm never satisfied
use the right tool for the right job. your software requirements and design document should drive the language and the tools you use.
Almost always when I start prototyping something in python, I wish that I stopped half-way where I am now and switched to something else.
Most recent example - converting huge amount of xml files to parquet. I started very fast with python + pyarrow, but when I realized that parallelizing execution would help enormously, I hit GIL or picking/unpickling/multiprocessing costs.
It did work in python, in the end, but I feel that writing that in Rust/C# (even if I don't know Rust besides tutorials) in the end would be much more performant.