> Sort of. Accessing private Canadian financing is a lot harder for the most part, our financial sector is notoriously risk averse for any sort of personal or business loans unless there's a path of guaranteed returns and you have a track record of getting those returns. Last time one of my bosses tried to get something like $2 million in raising funds with Canadian investors, they said that we needed to show I think at least $2 million in net profits for 4 or 5 years at least.
If someone's businesses is already working, then why would they go and get investment money?
It seems like the investors don't realize that if someone does have that much net profit, they actually don't have that much leverage over them to get good equity deals. So they are essentially just paying high and selling low later.
I am impressed that Polar is close to DuckDB near the top. It's surprising that a Python library would often out perform everything but DuckDB. DuckDB is very impressive but DataFrames and Python is too useful to give up on.
DuckDB interoperates with polars dataframes easily. I see DuckDB as a SQL engine for dataframes.
Any DuckDB result is easily converted to Pandas (by appending .df()) or Polars (by appending .pl()).
The conversion to polars is instantaneous because it’s zero copy because it all goes through Arrow in-memory format.
So I usually write complex queries in DuckDB SQL but if I need to manipulate it in polars I just convert it in my workflow midstream (only takes milliseconds) and then continue working with that in DuckDB. It’s seamless due to Apache Arrow.
Wow, what a cool workflow. I looks like the interop promise of Apache Arrow is real. It's a great thing when your computer works as fast as you think as opposed to sitting around waiting for queries to finish.
I mean polars is great, but there's nothing fundamentally impossible about polars providing similar performance to DuckDB, polars is written in rust, and really a lazy dataframe just provides an alternative frontend (sql being another frontend).
There's nothing in the architecture that would make it so that performance in one OLAP engine is fundamentally impossible to achieve in another.
I didn't know that Polars was implemented in Rust. In fact, it's very neat that Rust can interop with Python so cleanly, but that shouldn't be surprising since Python is basically a wrapper around C libraries.
But I still think it's surprising how much legs Python model of wrapping around C/C++/Rust libraries has. I would assume that if you have Python calling the libraries, you can't do lazy evaluation and thus you hit a wall such as Pandas.
But we seen with compiling Pytorch and Polars that you can have your cake and eat it too. Still have the ease of use of Python while having performance with enough engineering.
The biggest lesson I took away from all of these hype cycles is how unpredictable they are in their arc. Some trends like AV seem like they were going to take over the world, but they died down as quickly as they rose when we realize how hard safety is.
VR seem like it was dead with Google Glasses and Meta. But recent releases show that the [vision of VR is more real than I thought](https://www.youtube.com/watch?v=MVYrJJNdrEg). Apple releasing their VR definitely provides more legitimacy to the idea.
Technology is always hard to actually implement in the real world. I think have expectations that the world would flip over with new innovations in a matter of months. But it actually takes years of hard work to get people to adopt it. The internet and mobile phones are good examples where the promise was clear but the journey being universally adopted was not easy. *
* For the internet, you have the massive struggle of web standards, getting implementations web browser to align, and achieving acceptable performance. Internet Explorer still haunts the dreams of many web devs. It's remarkable how many fully featured web apps there are like Onshape and Google Maps.
* For mobile phones, they require a massive amount of technology investment and unit economics to get the price cheap enough for normal people. It's amazing how wireless technology gotten to the point where we can stream videos on phones.
I think the time to value between the Internet’s hype cycle and real utility was relatively short compared to most other technology hype cycles, because it was like the OG computer hype cycle and so was naturally a bit more delayed than subsequent hype cycles that had as their rationale “better not miss out in case this as big as the Internet”. It only took about 5-10 years to go from the beginning of the bubble to the modern incarnation of the internet and early versions of the products that now underpin trillion dollar businesses.
But, a lot of internet standards had been in the works for a long time at that point. Not so for AV and VR. I think we’re looking at 2+ decades for a lot of technology like that. It’s such a long time it really shakes out the hypemen with short attention spans.
If you can put javascript and animations in pdf, what's stopping you from making a frontend in it? I wonder what are the frontiers of things you can do in pdf.
Honestly, it seems like only malware authors benefit from the complexity of pdfs.
Yea, I am a struggling to figure out what the secret sauce of this library and if that sauce is introducing foot guns down the line.
Multiprocessing std uses fork in linux distros already. I once ran a multiprocess code on Linux and Windows and there was a significant improvement in performance when running Linux.
The confusion here is parallelism vs concurrency. Parallelism is executing multiple tasks at once and concurrency is the composition of multiple tasks.
For example, imagine there is a woodshop with multiple people and there is only one hammer. The people would be working on their projects such as a chair, a table, etc. Everyone needs to use the hammer to continue their project.
If someone needed a hammer, they would take the single hammer and use it. There are still other projects going on but everyone else would have to wait until the hammer is free. This is concurrency but not parallelism.
If there are multiple hammers, then multiple people could use the hammer at the same time and their project continues. This is parallelism and concurrency.
The hammer here is the CPU and the multiple projects are threads. When you have Python concurrency, you are sharing the hammer across different projects, but it's still one hammer. This is useful for dealing with blocking I/O but not computing bottlenecks.
Let's say that one of the projects needs wood from another place. There is no point in this project to hold on to the hammer when waiting for wood. This is what those Python concurrency libraries are solving for. In real life, you have tasks waiting on other services such as getting customer info from a database. You don't want the task to be wasting the CPU cycles doing nothing, so we can pass the CPU to another task.
But this doesn't mean that we are using more of the CPU. We are still stuck with a single core. If we have a compute bottleneck such as calculating a lot of numbers, then the concurrency libraries don't help.
You might be wondering why Python only allows for a single hammer/CPU core. It's because it's very hard to get parallelism properly working, you can end up with your program stalling easily if you don't do it correctly. The underlying data structures of Python were never designed with that in mind because it was meant to be a scripting language where performance wasn't key. Python grew massive and people started to apply Python to areas where performance was key. It's amazing that Python got so far even with GIL IMO.
As an aside, you might read about "multiprocessing" Python where you can use multiple CPU cores. This is true but there are heavy overhead costs to this. This is like building brand-new workshops with single hammers to handle more projects. This post would get even longer if I explained what is a "process" but to put it shortly, it is how the OS, such as Windows or Linux, manages tasks. There is a lot of overhead with it because it is meant to work with all sorts of different programs written in different languages.
A lot of problem with Python packages is the fact that a lot of Python programs is not just Python. You have a significant amount of C++, Cython, and binaries (like Intel MKL) when it comes to scientific Python and machine learning. All of these tools have different build processes than pip so if you want to ship with them you end up bring the whole barn with you. A lot of these problems was fixed with python wheels, where they pack the binary in the package.
Personally, I haven't ran into a problem with Python packaging recently. I was running https://github.com/zyddnys/manga-image-translator (very cool project btw) and I didn't ran into any issues getting it to work locally on a Windows machine with Nvidia GPU.
If someone's businesses is already working, then why would they go and get investment money?
It seems like the investors don't realize that if someone does have that much net profit, they actually don't have that much leverage over them to get good equity deals. So they are essentially just paying high and selling low later.