Python is not compiled and start & load pandas (which is comparable to the libraries loaded in the article) in ~0.4" on my computer, and that's a notoriously slow language.
If I were to use e.g. Rust with polars, load time would be virtually none. And when I have to process ~50k different datasets, I can't afford 0.85" per file, which would translate to ~11 hours of overhead.
> If I were to use e.g. Rust with polars, load time would be virtually none.
Because you're compiling...
And if you need to do the same in Julia, you should also pre-compile or some other method like https://github.com/dmolina/DaemonMode.jl (their demo shows loading a database, with subsequent loads after the first one taking roughly ~0.2% of the first)
No it's not, because I will not architecture my whole pipeline & program around Julia inability to start in maybe a second in a year or 1.7" now, I will just use another language.
Python is not compiled and start & load pandas (which is comparable to the libraries loaded in the article) in ~0.4" on my computer, and that's a notoriously slow language.
If I were to use e.g. Rust with polars, load time would be virtually none. And when I have to process ~50k different datasets, I can't afford 0.85" per file, which would translate to ~11 hours of overhead.