Transformer is a patterning probabilistic machine for a sequence of identities[1]. These identities are fed to the transformer in lanes. The transformer is conditioned to shift lanes one position to the left until they make it to the output, and make a prediction in the right-most lane that got freed up. Attention adds an exponential amount of layer interconnectivity, when we compare it with a simple densely connected layers. The attention mask serves as a high-dimensional dropout, without which it would be extremely easy for the Transformer to simply repeat the inputs (and then fail to generalize when making the prediction). Each layer up until the vertical middle of the Transformer works with a higher contextual representation than the previous one, and this is again being unwound back to lower contexts from the middle layer back to the original identities (integers) on the outputs. This means that you have raw identities on the input and output which span a certain width/window of the input sequence, but in comparison the middle-most layer has a sequence of high level contexts spanning extreme lengths of the original input sequence, knowledge-wise. [1]It's important to know that modification (learning) by the Transformer, of the vector embeddings which represent the input/output identities/integers that the Transformer works with, constitute big portion of the Transformer's power, and the practical implication of that is that it's impractical to try to tell the Transformer that e.g. some of our identities are similar or there's some logical system in their similarity, because all the Transformer really cares about is the occurrence of these identities in the sequence we train the Transformer on, and the Transformer will figure out the similarities or any kind of logic in the sequence by itself.
Life (aging) is a direct cause of cancer. And we can't change that unless we have some sort of nanobots constantly repairing DNA damage in every single cell of our body.
You realize the human body has a process thst cleans out damaged and misbehaving cells. Autophagy. This process is kicked off during periods of not eating.
We both expose ourselves to too Many toxins, eat too often, and don't get the right balance of nutrients over the long term.
If you can positively impact these things, then you can lengthen health span and lifespan.
Yes and no. While some cells may be impossible to rejuvenate otherwise, there are few reasons why we cannot just exchange many organs for dish grown ones, i.e artificially grown kidneys. For others like blood stem cells, we many extinguish the existing ones and add replace them by fresh ones that lack any pathogenic somatic mutations every so often. Even parts of the brain may be replaced one by one, though I assume such a technology is quite far away (compared to the other two that may be possible in the next 2-3 decades).
How's that related to this story? One outcome might have multiple factors affecting its probability of ocurrence. Just because you can't change life doesn't mean you can't stop alcohol. Life isn't 1 and 0 dude.
I can imagine aging being a technically solvable problem and maybe some of us will live to see it.
But if people didn't age, then there would still be a base rate of mortality from illness and accidents, and so for any arbitrary probability (99.9%, etc) there would be a maximum age that one could reasonably expect to live to.
For instance, if the only thing people ever died of was car accidents, at a rate of 20 per 100,000 people per year, then it would still be very unlikely for anyone to live more than 20,000 years. That's a long time, but it's not millions or billions of years, much less eternity.
Furthermore, if nothing but accidents killed people we might become extremely conservative and do practically nothing for thousands of years. Do I want to use the stairs and risk 99% of my potential lifespan?
So maybe one could say that biology and even physics don't ensure death, probability does.
I know I'd never be able (or wanting) to live with having left my dogs behind. I don't fear my own death _that_ much. I'm here for them until they cross, only after that I can do whatever I want with my life.
Damn, so I really need to follow up on my plans on moving to e.g. Thailand (and getting happy by following my dreams in terms of work/endeavors).
I don't have IBS but more like a colitis and I was really good for months but it came back strong after one of my dogs passed away and I got terribly sad.
What I'm getting at is that more than a vitamin D, I suspect a strong link to happiness and life satisfaction.
1. When people are financially incentivized to run servers, they always will.
2. If a crypto protocol doesn't evolve at the pace of available innovation, that particular blockchain will be superseded by a new one. That said, a (truly democratic) evolutionary process is a core part of every blockchain specification.
3. You can get blockchain data via public (and federated/proxied) API, but you can always cryptographically verify its veracity, and your edge device (e.g. your smartphone) can do that. The same the other way around, you cryptographically sign the inputs you send to the networks, so that no federated API can tamper them, because the secret key stays on your device. This is referred to as the "trust-less model".
It's interesting to see how human understanding differs when it comes to complex, yet clearly defined topics, like machine-learning/Transformers.
For comparison, my understanding of Transformers, after going through Peter Bloem's "Transformers from scratch" [1], implementing/understanding the code and the actual flow of the mathematical quantities, my understanding is that:
- Transformers consist of 3 main parts: 1. Encoders/Decoders (I/O conversion), 2. Self-attention (Indexing), 3. Feed-forward trainable network (Memory).
- The Feed-forward is the most simple kind of (an input->single-layer) neural net, actually often implemented by a Conv1d layer, which is a simple matrix multiply plus a bias and activation.
- The most interesting part is the Multi-head self-attention, which I understand as [2] a randomly-initialized multi-dimensional indexing system where different heads focus on different variations of the indexed token instance (token = initially e.g. a word or a part of a word) with respect to its containing sequence/context. Such encoded token instance contains information about all other tokens of the input sequence = a.k.a. self-attention, and these tokens vary based on how the given "attention head" was (randomly) initialized.
The part that really hits you is when you understand that for a Transformer, a token is not unique only due to its content/identity (and due to all other tokens in the given context/sentence), but also due to its position in the context -- e.g. to the Transformer, the word "the" at the first position is a completely different word to the word "the" on e.g. the second position (even if the rest of the context would be the same). (Which is obviously a massive waste of space if you think about it, but at the same time, at the moment, the only/best way of doing it, because it moves a massive amount of processing from inference time to the training time - which is what our current von-Neumann hardware architectures require.)
Forget all that. If we tried to swich half the cars to electric overnight, the power grid would collapse into a permanent blackout. Has anyone estimated the cost and time of upgrading the power grid for electric cars? Is hydrogen delivery from power plants to electric charging stations a viable option to bridge the gap until the grid is upgraded?
Please stop downvoting the above comment. Sure, it is wrong thinking imho, but it is a doubt that a lot of people have, and it needs to be seen and addressed.
If we just downvote questions like this into oblivion, it kills the discussion and people continue on with their misconceptions, which never get addressed.
OP here. The second sentence came out all wrong and the downvotes are deserved. What I was trying to ponder, was - given the summer blackouts (in certain areas) due to people running their ACs (more than on average) - how do we plan to cope with the advent of EVs.
So I did some googling[1] and apparently, until 2030 there's only 1 million EVs planned to hit the roads in the US, which is quite shocking given the dire trends of climate change and toxic pollution on this planet.
Another interesting fact is that the costs of extending the power grid will be passed on to all its consumers in the form of increased rates (as it should, because if you don't drive an EV, you should be "taxed" on the extra pollution your car creates).
Can confirm that this is a real concern In The Wild—even if it's not a good one—my (boomer, yes) uncle was just complaining to me a couple weeks ago that, "no-one's thought about how we're going to have enough electricity for all these electric cars".
In my experience, many of the people who say things like this don't just have misconceptions, they have an axe to grind. In that case, preventing the spread of their misinformation may have more value than trying to 'educate' them.
It's not so much to educate those who have an ax to grind, and I tend to agree on where (many of) these people are coming from.
It's to allow everyone else to see under the open light how weak the ax is, and what a dull blade it has, and what poor metal it was forged out of, and how it was never a good ax to begin with.
Instead of just saying (not that you directly said this, but downvotes kind of say it): "let's not talk about that."
I think I agree with you, but I may have less faith in the ability of random people to get past the first argument and make it to the counterpoints. I'm probably being uncharitable; especially somewhere like HN.
No they wouldn’t. Steam turbines can take days to ramp up and connect to the grid. Several dams in my state barely reach full capacity and are not running 24/7. The co-op that supplies my power offers a flex plan that gives you free power from 11pm-6am. You’re supposed to schedule your car to charge during that time. If half the cars in my county did this nightly it would be easier to predict the nightly loads.
For a forum focused on meaningful and substantive discussion, the amount of incorrect information around EVs and the power grid is shocking.
GP's point is a good one although there is work happening to address that very problem. The point stands, though. EVs are a significant bump in both power and energy and will require significant investment in shoring up grid capacity. Sad this is downvoted.
>Steam turbines can take days to ramp up and connect to the grid
How is this relevant here? Also, the typical steam turbine takes hours to spin up and sync to the grid, not days. I am not aware of any steam turbines that require burning two day's worth of fuel just to connect to the grid.
>Several dams in my state barely reach full capacity and are not running 24/7.
Again, how is this relevant in a broader context? Lots of places have dams running dry just as equally many have dams overflowing.
>The co-op that supplies my power offers a flex plan that gives you free power from 11pm-6am.
Ok? This is not par for the course so i'm nor sure why you're mentioning it.
>You’re supposed to schedule your car to charge during that time.
And if you can't? What if offices suddenly have 50 EVs charging at the same time? Will this not lead to a capacity constraint? If you broaden the horizon, can this not lead to a localised collapse of the grid due to a demand surge?
>If half the cars in my county did this nightly it would be easier to predict the nightly loads.
It would also lead to a huge surge in power at the times when solar output is literally zero.
This is obviously going to be region-dependent, but I don't think upgrading the grid is avoidable, even if we kept using ICE cars. Energy usage _is_ going to continue increasing as population grows and HVAC requirements increase.
Also, I'd like to add some perspective with actual numbers: My whole house consumed 109.49 kWh yesterday, of which only about 12kWh was my car charging (I have a PHEV). That number wouldn't change significantly if I had an EV, as my driving habits would probably stay roughly the same.
So car charging accounts for ~10% of my electricity consumption these days. Yes, that's not insignificant, but it's also not 50-75%. It's probably less impact on the grid than adding a central heat pump to a house, and I don't see people spreading FUD about heat pumps.
For the grid itself, time-of-use is _the_ important factor, as the size of wires, equipment, etc. is what determines the peak load the grid can handle. If the actual demand exceeds that for even only one minute in the year, it means we need to upgrade the grid. Operators are very interested in flattening that peak, so will offer incentives to move electricity consumption to off-peak times. Here in Québec, the evening / night is peak time in winter, as we all have electric heating, and night-time is colder. If vehicle charging is a significant burden, they'll offer incentives to move that to daytime.
>What if offices suddenly have 50 EVs charging at the same time?
They do. No problems so far.
>Will this not lead to a capacity constraint?
No. Batteries and rooftop solar.
>If you broaden the horizon, can this not lead to a localised collapse of the grid due to a demand surge?
Not with batteries and rooftop solar, no. We good?
>It would also lead to a huge surge in power
It's not a "huge surge." Many other appliances are off at night. The grid has excess power at night. The larger problems are in the day, caused by things like massive use of AC, which problems will be ameliorated by installation of more rooftop solar. Win win.
>at the times when solar output is literally zero.
Batteries.
And, sometimes, when needed, you also, wait for it… use fossil fuels or other non-solar sources! We can cut fossil fuel use drastically and cut carbon emissions, but still use them when needed, and everything will be OK!
> We can cut fossil fuel use drastically and cut carbon emissions, but still use them when needed, and everything will be OK!
Not to detract from your core point as there are ways to solve this[1], but long term the necessary level of solution is 99.9% reduction over all sources of CO2, so no, it isn’t OK unless it’s no more than 8 hours per year[1].
[0] hydrogen, synthetic hydrocarbons, even burning wood instead of coal works in this situation, so long as the costs are lower than for fossil carbon
[1] on average, with a margin of error of way more than 100% depending on all the carbon sources that aren’t electrical
> The co-op that supplies my power offers a flex plan that gives you free power from 11pm-6am. You’re supposed to schedule your car to charge during that time.
One thing I’ve realised recently, is that while this is an improvement compared to the status quo, at some point we’re likely to be running houses off car batteries in these hours and charging the cars off PV during the day. By the continuum hypothesis, at some point the net average power transfer into/out of cars/any given car is going to be zero, and I wonder what that will look like economically?
You won’t be doing that unless it’s a blackout.
The amount of extra cycles that would put on your battery wouldn’t be worth it. Considering the vast majority of the cost of a EV is your battery, the cost savings just don’t add up when you add the damage to your vehicles range and worth.
I’m not sure what average domestic nighttime electrical power use currently is, but 500 watts seems plausible, and cycling 6 kWh per day of an EV’s battery doesn’t seem like it will add much wear compared to normal use, unless you have better figures to estimate from?
Also, a major source of electrical demand is being systematically diminished: lighting. Energy-efficient lighting is more and more prevalent with each passing day.
Last year, my town's side streets were equipped with LED streetlights, which meant replacing 175W mercury-arc and 150W HPS fixtures with 40W LEDs that are noticeably brighter and produce a far more pleasant light.
Two of the biggest crypto miners are in Texas, where (until very recently) power was essentially free at night. The winter storm this past February changed the economics a little bit, but for the most part, the Texas Power Grid creates more power than can be consumed at night. And supposedly if the power grid (ERCOT) has enough consumption, they will pay the miners to shut down.
It probably will still provide some "payment" back, as long as some control is allowed for the grid operator.
At the moment they're happy with "charge any time between these hours", once everyone starts doing it it'll probably be "free if you leave your car plugged in and let us decide when and how much to charge".
Shifting demand has a price and they'll pay that to the users that help them achieve it in some form or other.
After all, if you don't charge it then, they'll need to provide the power later when it could cost them many multiples, it's just good business sense.
I think the biggest issue in the UK is how to get cars parked on street to be charged overnight. In towns and cities (and elsewhere too) it is very common to park your car on the street, potentially a good distance away from your home. Where exactly to build the charging infrastructure for people parking like this.
I guess if you only need to charge at the same frequency as you would refuel you could have chargers in super market parking lots. Cars charge whilst you do your weekly shop. Or at work, if they have space.
I also heard about one idea to have chargers at every street light, as they already have an input from the grid in them.
Hopefully this is an issue that can be solved without having to re-wire huge portions of the urban areas.
I'm in a city in Norway. It is very common to park cars on the street: The city is expanding their curbside charging stations because of it, and spaces are reserved for electric cars. Most parking garages I've seen here have some charging stations and a number of workplaces do as well, though admittedly that infrastructure might be, in part, due to engine warmer things. Not all workplaces provide parking, but as mentioned, the spaces you pay for often have this as an option. A good number of gas stations and shopping centers have charging as well. I've only lived here 8 years and it has grown exponentially.
I'm pretty sure, however, that the government has taken some initiative to expand the infrastructure. After all, electric cars are pretty well subsidized and you get everyday driving perks (like being able to drive in the bus lane). When folks are buying new here, they are buying electric and they aren't worried about meeting their "all-electric" goals. And realistically, government involvement, organization, and action are going to be the things that make it more difficult in, say, the US or the UK (From what I can tell: I'm American and am not as intimate with UK politics)
> Where exactly to build the charging infrastructure for people parking like this.
On the sidewalks. There are a number of chargers like that in the city I live that are more or less subscription based.
> Cars charge whilst you do your weekly shop. Or at work, if they have space.
That's one excellent solution. You can take the money that won't be spent with healthcare (because air pollution) and channel it as incentives to build out that infrastructure.
> I also heard about one idea to have chargers at every street light, as they already have an input from the grid in them.
That's another interesting idea, if the power is available. With the move to efficient lighting, the need for power has been reduced and I assume the cable gauges have reflected that.
> Hopefully this is an issue that can be solved without having to re-wire huge portions of the urban areas.
If the cars can recharge for hours, the need to rewire for higher currents is smaller.
I guess street chargers work great if you park on the street and can leave your EV in the same spot even after it is charged, like a regular parking spot. Otherwise it's just a chore to charge and move your EV to a non-charging spot when it's finished (of which there would be fewer)?
> If we tried to swich half the cars to electric overnight
Let's not, then. The logistics of swapping 150 million cars in a single night would be a nightmare. Charging wouldn't be my first concern.
On a more realistic note, even if we outlawed the sale of ICE cars today and mandated that every single new car hitting the road was electric, the required growth of the grid would be feasible. It would take quite a few years to replace the entire fleet.
If we were to try to switch all vehicles to EV “overnight”, we’d almost certainly also mandate PV on all car rooftops.
I have observed that discussion of this solution normally divides into these two groups:
(1) “That’s ridiculous, a car covered in with PV will only make enough electricity to go 10-30 miles per day!”
(2) “That’s a great idea, a car covered with PV will make enough electricity to go 10-30 miles power day, and most people only go 12-35 miles per day!”
When there is serious concern if the grid can cope, we should take all the mitigations we can get. Reducing aggregate charging demand by 90% is good even when every single driver still needs to use an external charger.
Just because you can't think of a solution does not mean others have not already thought of solutions. Rooftop solar for example. Doesn't work for all buildings, but we are talking about the aggregate effect on the grid, which it does work for in most locales. In other locales, there are other solutions. You mention hydrogen. Pretty corrosive and explosive stuff, but maybe.
Transformer is a patterning probabilistic machine for a sequence of identities[1]. These identities are fed to the transformer in lanes. The transformer is conditioned to shift lanes one position to the left until they make it to the output, and make a prediction in the right-most lane that got freed up. Attention adds an exponential amount of layer interconnectivity, when we compare it with a simple densely connected layers. The attention mask serves as a high-dimensional dropout, without which it would be extremely easy for the Transformer to simply repeat the inputs (and then fail to generalize when making the prediction). Each layer up until the vertical middle of the Transformer works with a higher contextual representation than the previous one, and this is again being unwound back to lower contexts from the middle layer back to the original identities (integers) on the outputs. This means that you have raw identities on the input and output which span a certain width/window of the input sequence, but in comparison the middle-most layer has a sequence of high level contexts spanning extreme lengths of the original input sequence, knowledge-wise. [1]It's important to know that modification (learning) by the Transformer, of the vector embeddings which represent the input/output identities/integers that the Transformer works with, constitute big portion of the Transformer's power, and the practical implication of that is that it's impractical to try to tell the Transformer that e.g. some of our identities are similar or there's some logical system in their similarity, because all the Transformer really cares about is the occurrence of these identities in the sequence we train the Transformer on, and the Transformer will figure out the similarities or any kind of logic in the sequence by itself.