Very happy to see Timescale making more features available in the community edition.
We first started evaluating time series databases a month or two ago, some features like continuous aggregation (rollups) were enterprise only. Perhaps their strategy is to drive adoption and letting people try their feature out, hoping that some of these adoptors will end up using their managed solution. I checked their pricing, and the delta between their pricing and the underlying AWS instance seems quite reasonable.
We ended up testing Influx first, because it seems to be a safe choice with wider adoption and extensive documentation.
With Influx, it was very easy to put together a prototype quickly. But once we started throwing some real workload at it, it would lose writes under load. But it makes sense that it failed, because according to Influx's documentation (https://docs.influxdata.com/influxdb/v1.8/guides/hardware_si...), we would need cluster to make it work. Influx is very transparent in their documentation that writes and queries will fail immediately when a server is unavailable without cluster.
This isn't to say that Influx wouldn't work for other use cases. But at least in our use case, their open source offering isn't suitable for us, and it's unclear how much better the cluster version is.
Timescale, on the other hand, was able to handle the same workload under stress. As we are unable to backfill some of the ingressing data, it's quite vital that the system can degrade more gracefully.
For my use case, one feature that still need some work in Timescale is their real time aggregation. It is currently impossible to define a rollup on top of another rollup, which means that if you are ingesting a lot of data into the raw table, and you down sample into a wide time bucket (e.g. a day, or week), queries against these wider buckets will potentially ended up having to query a lot of data points, slowing the system down considerably. Granted, it is a new feature that just got released about a month ago. Hopefully, with multi-node nearing completion, continuous aggregation will get a bit more love.
I spoke with their engineers about this over Slack, and their suggestion was to manually modify the rollup materialized view to aggregate over a combination of the materialized buckets (currently handled by the continuous aggregation) + real time aggregation from a higher resolution bucket.
We are still testing out Timescale, of course. But so far, it's been holding up its end of the bargain. The fact that Timescale is "just an extension" built for Postgres also makes it a less risky choice and offers a lot of flexibility; if Timescale doesn't work out, we could still work with Postgres, and that IMHO is a very nice thing.
Thanks for the feedback. Really glad to hear that your experience is going well with TimescaleDB! Feel free to ping me directly ajay (at) timescale.com if there's anything I can do to help.
When I first learned about Prolog in AI class, the first thing that struct me was how beautiful Prolog/logic programming solutions are.
In non-declarative languages like Java, you have to build up the different pieces of computations, while you keep maintaining a mental picture of how the various pieces fit together.
In Prolog, you declare the entities and the relationships/constraints between them, and the system will build the solution for you through inference.
David Nolen has done some awesome work writing the core.logic[1] library, which makes those Prolog gooodies accessible to Clojure programmers.
I seem to recall that one of the issue with Prolog is that, above a certain level of complexity, you can't really write purely declarative Prolog "progams" - you have to start considering the procedural aspect as well which (in certain cases) can be non-trivial.
You indeed have to understand how the Prolog engine works in order to structure the declarative statements of your code in a way that will lead to an efficient execution.
A way to see a Prolog programs is as a sequence of facts and statements, and at least one query. The Prolog engine will then search for a solution that fits the given facts and requirements and answer the query. In a way the Prolog engine will search a space of possible answers to the query to find the one(s) that match the given facts and requirements. The key to speed is to structure the facts and statements that the search will fail as early as possible when a wrong path is taken. This amount to pruning the useless parts of the search space as aggressively as possible, so that the Prolog engine does not waste time evaluating options that are doomed in the end.
Before getting this I was often frustrated with apparently nice and correct Prolog programs that took forever and in effect just looked stuck (at some point you just stop waiting and abort the execution). I guess it's a pretty common frustration when beginning in Prolog. But once you get it, it's possible to come up with efficient code. It's still scary to see that some small changes in statements ordering can lead to dramatic difference in runtime. You can have big differences in performance for imperative programming too, but it's rare that it's so bad that a first implementation is completely useless. In Prolog it's quite common. And the way to optimize Prolog performance is very specific, you need to learn to anticipate how the engine walks the search space. I guess it's one of the big roadblock in the practical use of Prolog.
A book -- in fact, the book -- that covers this sort of thing is Richard O'Keefe's The Craft of Prolog, which, like Sterling & Shapiro's The Art of Prolog, is put out by MIT Press.
Mercury[1], a derivative of Prolog does away with these cuts. So I guess that these declarative short-circuits in Prolog are just a fault that arose because some things were not thought through in the creation of Prolog.
I've found the Clojure community at large to be well reasoned in general and very intelligent. KirinDave's posts may be a little hot, but I think it is a sign of growth for Clojure. Ruby/Rails also had (has? ;-) ) a problem with smugness as it was gathering rabid following. Let's hope that this is a sign that Clojure is also building up a gathering of all sorts of interesting characters :-)
> by and large Clojure basically obsoletes Ruby and Python
Care to qualify how Clojure obsoletes Ruby or Python? Both of those languages have great tools (rails, for example) that those in the Clojure communities have great respect for. As a Clojure and Ruby/Rails programmer, I share your enthusiasm for Clojure and lisp, but I can't see Clojure pushing either Ruby or Python out of the water any time soon.
> Both of those languages have great tools (rails, for example)
I am surprised a Clojure programmer would have much respect for Rails. Its aggressive share-nothing approach leads to inconsistency, poor performance, awkward architectures, and over-reliance on external state management.
Perhaps one might admire it for making a bold claim about configuration vs. convention, but a valid criticism of Rails is that they went so far along the convention line in rebellion of bad tools like Spring that now they get accused of excessive magic.
> But I can't see Clojure pushing either Ruby or Python out of the water any time soon.
Like I said, success is fashion. But then, people said this to me about Git Vs. SVN after I gave a tech talk to a bunch of ruby folks about how git would take over the world. Two people left that talk and went on to found Github, so... "High Fashion."
Clojure has all the power of Ruby's dynamism, but with a better solution to the expression problem (protocols are namespace contained, allowing for monkey-patching that doesn't taint the world). It has access to a lot of fantastic high-performance libraries. It can safely deal with high degrees of parallelism with a degree of ease few environments can claim. It has a minimalist syntax which can compete with even the most terse and convention-driven ruby syntax (i.e., Compojure vs. Sinatra).
Similar complaints can be leveled at Python with less of a performance angle and more of a power angle; Clojure gives you access to many of those features that the python community feels are "too awkward" to use in the OO world, but are entirely natural in the Lisp world.
So yeah, every time someone writes a server in Node.js I think to myself, "You could have done it in Clojure and had just as much dynamism during development, just as easy a deployment story, and also perform easily an order magnitude faster under load, and you don't have to run like a scared child from parallelism."
> I am surprised a Clojure programmer would have much respect for Rails. Its aggressive share-nothing approach leads to inconsistency, poor performance, awkward architectures, and over-reliance on external state management.
When evaluating what tool to use for a project, there are many things to take into account other than the technical superiority of the tools. Relevance Inc, one of the strongest backer of Clojure, still make use of Rails for client projects. Rails' huge web dev ecosystem really made many common tasks as simple as importing a gem here and there and making a few method calls. It all comes down to the right tool for the right job. I'm interested in your take on how Rails' "share-nothing approach", awkward architectures etc.
> Clojure has all the power of Ruby's dynamism
Yes, and then some. Without repeating what you have already said, I would add efficient persistent data structure to the Clojure's list of awesome tricks. Doing functional programming in ruby and python, for example, will force you to trade value immutability with efficiency -- a penalty that one does not have to pay in Clojure.
Clojure is definitely gathering a lot of following. Like many others, I have picked it up, played with it, and never looked back.
But going back to what was said before. Yes, Clojure is awesome. No, Clojure is not obsoleting Ruby or Python any more than those obsoleted PHP, at least not now.
I enjoyed programming in Clojure, but chose Python once more for a new project because w/ Clojure I don't know which projects are doing well. (Templating? Web app library? Postgresql?).
Wrt Python I know I'll do fine w/ Django, Flask, Jinja2, Psycopg2, Tornado...
You have successfully gotten me to commit myself to looking into Clojure. Any good resources on learning/setting up, or should I go by the seat of my pants?
I find Clojure Programming (http://www.clojurebook.com/) to be an easy and practical introduction of the language. It often compare and contrast the Clojure way of doing things vs scripting languages like Ruby. After that, Joy of Clojure (http://joyofclojure.com/) dives deep into the whys of the language. Even though it was written against Clojure 1.2, it covers advanced topics like continuation passing style that were left out of Clojure Programming.
Clojure arithmetic is fast enough if you add optional type annotations or otherwise take advantage of unboxed values. Ordinary calculation-intense code will be hundreds of times faster than in Python which might be enough in most cases.
If you need deeper support for vector math, scientific calculations, and the like then Incanter is probably what you want. It does all the complicated fast math stuff that always benefits from carefully planned and hand-tuned libraries.
To be clear... the benefit of Scipy / Numpy / Matplotlib that I'm alluding to is not (necessarily) runtime efficiency -- though, given that many of the linked libraries are standards (eg. LAPACK) and compiled C / Fortran, it is efficient for many operations.
The benefits to scientists are the developer efficiency (rapid prototyping), comprehensive libraries (scientific functions and plotting), interfaces similar to MatLab (eg. slicing), and its standing in the scientific community. It would sort of defeat the purpose to spend valuable research time to reinvent the wheel in another language -- though I would switch to a LISP variant in a heartbeat if there was comparably functionality.
I've been seriously spoiled by paredit. It is a brilliant, absolutely brilliant way to edit code. Editing code written in those so called free form readable languages is absolute pain compared to structured editing in paredit. It is a reason to program in Lisps by itself. You don't need that filter on your data? Easy, you just need one keystroke (M-r) to raise the inner expression. Everyone should experience it, it certainly makes me want to write more and more code in Clojure.
Since 1.0.0rc1 that was released 5 months ago (1.0.0 went gold 2 months ago), the custom attributes used for by AngularJS can be turned into valid html5 attributes by prepending the "ng-*" AngularJS attributes with "data-". For example,
<html ng-app>
can be turned into a valid attribute by making it
<html data-ng-app>
I'm really glad to see how quickly the AngularJS team is addressing concerns like these. AngularJS is moving very fast, and there is a lot of room for improvements in documentation. But a quick check with the changelog would have showed that AngularJS supports the use of valid custom attributes for directives almost half a year ago.
For me, the best take-away from the post is this one line:
"I didn’t care how much I ran, as long as I went out and did it."
I have seen many people starting a new exercise regime (among other things) by trying too hard and expecting quick results, and that leads to the inevitable burn out that most people experience soon enough. I've been there.
Whether it's learning a new programming language or getting fit, it's more important to keep at doing it each day than to focus only on beating yesterday's record.
Tomorrow, I'm going to put on my running shoes and go out to run. I don't care how much I can run, but I'm going to time myself for record keeping, and I'm just going to feel good that I did it :-)
We first started evaluating time series databases a month or two ago, some features like continuous aggregation (rollups) were enterprise only. Perhaps their strategy is to drive adoption and letting people try their feature out, hoping that some of these adoptors will end up using their managed solution. I checked their pricing, and the delta between their pricing and the underlying AWS instance seems quite reasonable.
We ended up testing Influx first, because it seems to be a safe choice with wider adoption and extensive documentation.
With Influx, it was very easy to put together a prototype quickly. But once we started throwing some real workload at it, it would lose writes under load. But it makes sense that it failed, because according to Influx's documentation (https://docs.influxdata.com/influxdb/v1.8/guides/hardware_si...), we would need cluster to make it work. Influx is very transparent in their documentation that writes and queries will fail immediately when a server is unavailable without cluster.
This isn't to say that Influx wouldn't work for other use cases. But at least in our use case, their open source offering isn't suitable for us, and it's unclear how much better the cluster version is.
Timescale, on the other hand, was able to handle the same workload under stress. As we are unable to backfill some of the ingressing data, it's quite vital that the system can degrade more gracefully.
For my use case, one feature that still need some work in Timescale is their real time aggregation. It is currently impossible to define a rollup on top of another rollup, which means that if you are ingesting a lot of data into the raw table, and you down sample into a wide time bucket (e.g. a day, or week), queries against these wider buckets will potentially ended up having to query a lot of data points, slowing the system down considerably. Granted, it is a new feature that just got released about a month ago. Hopefully, with multi-node nearing completion, continuous aggregation will get a bit more love.
I spoke with their engineers about this over Slack, and their suggestion was to manually modify the rollup materialized view to aggregate over a combination of the materialized buckets (currently handled by the continuous aggregation) + real time aggregation from a higher resolution bucket.
We are still testing out Timescale, of course. But so far, it's been holding up its end of the bargain. The fact that Timescale is "just an extension" built for Postgres also makes it a less risky choice and offers a lot of flexibility; if Timescale doesn't work out, we could still work with Postgres, and that IMHO is a very nice thing.