I am running Sway on a Gentoo on a 4 years old X1 carbon.
> you will get similar usable battery life of around 6-8 hours
My macbook M3 gives me way more than 6-8 hours, it's simply insane. It literallly lasts for multiple days.
> Generally though, battery life isn't an issue anymore considering fast charging is everywhere.
Not an issue indeed, I got used to always charging my X1 carbon. But then I got an M3 for work, and... well it feels like I don't have to charge it ever :-).
As I said: very much a Linux person, but the M3 battery life is absolutely insane.
People need to get away from this idea of Key/Query/Value as being special.
Whereas a standard deep layer in a network is matrix * input, where each row of the matrix is the weights of the particular neuron in the next layer, a transformer is basically input* MatrixA, input*MatrixB, input*MatrixC (where vector*matrix is a matrix), then the output is C*MatrixA*MatrixB*MatrixC. Just simply more dimensions in a layer.
And consequently, you can represent the entire transformer architecture with a set of deep layers as you unroll the matricies, with a lot of zeros for the multiplication pieces that are not needed.
I might be completely off road, but I can't help thinking of convolutions as my mental model for the K Q V mechanism. Attention has the same property of a convolution kernel of being trained independently of position; it learns how to translate a large, rolling portion of an input to a new "digested" value; and you can train multiple ones in parallel so that they learn to focus on different aspects of the input ("kernels" in the case of convolution, "heads" in the case of attention).
I think there are two key differences though: 1) Attention doesn't doesn't use fixed distance-dependent weight for the aggregation but instead the weight becomes "semantically-dependent", based on association between q/k. 2) A single convolution step is a local operation (only pulling from nearby pixels), whereas attention is a "global" operation, pulling from the hidden states of all previous tokens. (Maybe sliding window attention schemes muddy this distinction, but in general the degree of connectivity seems far higher).
There might be some unifying way to look at things though, maybe GNNs. I found this talk [1] and at 4:17 it shows how convolution and attention would be modeled in a GNN formalism
>A single convolution step is a local operation (only pulling from nearby pixels), whereas attention is a "global" operation.
In the same way where the learned weights to generate K,Q,V matricies may have zeros (or small values) for referencing certain tokens, convolution kernels just have defined zeros.
The whole reason for the first "AI Winter" was because people were trying to solve problems with smaller neural nets, and of course you run into problems during training, where you can't get things to converge.
Once compute became more available, and you had more neural nets, and thus more dimensionality (in the sense of layer sizes), during training, you had more directions for gradient descent, so things started happening with ML.
And all the architectures that you see today are basically simplifications of the fully connected layers with max dimensionality. Any operation like attention, self attention, or convolution can be unrolled into matrix multiples.
I wouldn't be surprised if Google TPUs basically do this. It seems to reason that they are the most efficient because they don't move memory around, which means that the matrix multiply circuitry is hard wired, which means that the compiler basically has to lay out the locations of the data in the spaces that are meant to be matrix multiplied together, so the compiler probably does that unrolling under the hood.
Germany has a big alt-rising in the form of AFD, and consequently, they do track social media heavily. There is also a non-insignificant fundamentalist Muslim population.
For things like troll posts or just general hate speech, most of the time the police visit your house and ask you questions and give you a stern warning. And remember, police in EU isn't like police in US - when you get visited by police in EU, you aren't afraid that you are going to get shot up or thrown on the ground and tazed if you did nothing wrong.
In extreme cases where you are calling for things like beheading, yea they def arrest for that.
Source: close friend that lives in Germanty works for a company that does business with German government. I don't know first hand but he is pretty aware of the policics in EU and I have no reason to believe he would be exaggerating.
On anther note, Germany policing is quite progressive actually. For example, if you run, you don't get a charge for evading/eluding - its actually legal to run from police because "desire for freedom is a human right".
In France, discriminatory identity checks are a striking illustration of this. Police disproportionately target certain citizens on the basis of their skin color or presumed origin, particularly young people perceived to be Black or Arab, including children. These abusive controls can often lead to more serious police violence, including with fatal outcomes.
> "big alt-rising in the form of AFD, and consequently, they do track social media heavily. There is also a non-insignificant fundamentalist Muslim population"
They are not though alt-right movements all work on shifting the blame. They always find a scapegoat (jewish people in WW2) for material conditions instead of attacking the root causes.
>Why are all European countries interested in surveillance all of a sudden in the last 3-4 years?
Because they look at what happened US, at the rise of popularity of fascism throughout the world (which is mostly perpetuated by key media players under the mask of being "anti-woke"), and basically decide that the people can no longer be trusted.
And they are fully correct.
Ive said this before on here, but the whole idea of privacy from surveillance is not applicable anymore in todays world.
The standard line of arguing is that people should be able to speak the truth free from government knowing about it and trying to silence them. The problem with this
* Most of the "truth" that these people refer to has been literally false propaganda, or ability to say slurs on social media.
* Despite that fact, not a single public media person speaking these lies has been silenced in any way by the government, despite things like patriot act existing in US or equivalent things in EU. The only time people have been silenced in places like UK is when posting extremely out of pocket stuff.
Furthermore people also say that you don't want to give the government the power to do this now because a government that you may not like will want to do this. Well, to no one surprise, the people with this libertarian mindset (and the so called "centrists") overwhelmingly vote right wing, and consequently, right wing runs on a platform of freedom, but when those people get in power, they not only actively tries to silence actual truth and free speech, but also they just don't give a fuck about the law and do what they want anyways.
So as unfortunate as it is, its a much better outcome for the current state of administration in EU to take a more invasive role in policing the populace, because economic growth and stability over long term is worth way more than some idealistic approach based on above. Historically this has shown to be true over and over again, while the latter has shown to result in economic decline. So its wortwhile to sacrifice some personal rights in return for a better future - we already do this to a large extent so this is nothing new.
In terms of applicability to the regular person, please understand that the privacy ship for you has already long sailed. You already can be tracked and analyzed in extreme detail, by really any person or company that is willing to buy advertising data and do correlation. There are companies that literally do this and contract out to the government. Also, you aren't that important enough to care about.
There's also this other thing he said: Only a virtuous people are capable of freedom. As nations become corrupt and vicious, they have more need of masters.
The transformation from latent space of thoughts in the form of brain signals is going to be much more difficult than building something that can interpret all the ways I can communicate with my body.
Some system that can track body movements, skin temp, pulse, respiration, facial expression, eye movements, and voice is more than sufficient to infer a lot of information, especially when its given training data on your own history.
We're at a funny stage where some careers are becoming "post-LLM". For example, SWE is either rapidly approaching, or surpassing the point where LLMs can do most of what we traditionally viewed as day to day SWE work. However this doesn't translate into "no more SWEs." I have no doubt that what it means to be a lawyer day to day will shift with LLM advancements.
Its not about need for a lot of people. Its about having the money to optimize, aspects of their life, which everyone does in different areas.
Trucks have the following advantages
* Can drive most anywhere due to high ground clearance and 4wd. This comes in handy quite a bit. Having 4wd + weight + all terrain or appropriate tires means being able to leave the place during a winter storm versus being snowed in.
* Can carry big things or dirty things. Motorcycles, mountain bikes, furniture, landscaping supplies, and so on, without being limited in where you can go or how fast you can go while towing a trailer like you would with an SUV.
* They are safer for the occupants. Can't control if other people drink and drive. Can control if you survive or not if you get hit by a drunk driver.
Or you know, PUT A FUCKING GAS GENERATOR IN A GIANT BED THAT YOU HAVE OUT BACK.
Im legit suprised this isn't a thing yet. I saw the Rivian gear tunnel when it got first announced, and I was almost sure that they are gonna offer a generator+fuel tank to fit into there for range extension.
You can do an efficient diesel or multi gas 1 Cyl engine, and you can make a system where you can put one or 2 of them in the bed along with any aftermarket gas tank, and now you have something that is "mission configurable".
Run a lightweight DE like i3wm with any modern thinkpad and you will get similar usable battery life of around 6-8 hours.
Generally though, battery life isn't an issue anymore considering fast charging is everywhere.
reply