> It uses TSMC’s 6 nm process, which won’t provide the performance and power savings that the cutting edge 5 nm process would. In prior generations, AMD fabbed x60 and x600 series cards on the same cutting edge node as their higher end counterparts, and used the same architecture too. However, they clearly felt a sharp need to save costs with this generation, compromising the RX 7600 a bit more than would be expected for a midrange GPU.
It's also so they only need to design one memory controller for 6nm. I believe I remember this corroborated from an AMD engineer interview around the 7900XTX launch. Memory controllers aren't just logic that can be "compiled" to whatever target node. They have specific electrical requirements that take substantial design work. For this generation AMD has a 6nm memory controller that they use both in this 6nm monolithic design and in their 6nm memory controller chiplets on the larger designs.
The 8GB VRAM size limitation is the biggest downside for cheap local AI. My feelings is that current LLMs start producing interesting results from 12GB and above.
For a company that's very affordable, for an individual we're not there yet but I've seen some impressive demos running on a M1/M2 laptop so no doubt time will bring down the hardware requirements even more (and hopefully the price of used 3090s as well).
Screw around on ebay, bidding for used cards from individual sellers, as opposed to 'top' sellers that get substantially more for their listings. They may or may not have been burning up the GPU for the last couple years overclocking. Eventually you can land a $600-ish 3090 [1]. Good luck.
I got burnt on a second hand card a few years ago, reckon it had been used for crypto mining, weird checker board pattern would appear every so often. That said I still bought a 3070 for around 300 off FB market place recently. Was in the guys house, what I saw reassured me he used it for games.
Here's where the unified memory architecture of M1 Macs, gaming consoles, APUs, and other custom SoCs can shine and shows the inefficiencies of the traditional ageing desktop PC architecture in the modern world where the graphics VRAM is completely separated from the system RAM.
I'd like to see PCs designed more like consoles or M1 Macs, with GDDR shared as unified RAM between GPU and CPU. I have an laptop with a last gen AMD APU that's no slouch, but the VRAM slice of total RAM is still fixed as configured in BIOS between 512MB and 4GB, instead of fully unified and dynamically shred by the OS, which seems highly ineficient and wasteful in the modern age.
It's why PS5 and Xbox with their 16GB of fully unified VRAM can compete with gaming PCS witch need 16GB of system RAM and over 8GB of VRAM. Why have two separate memory zones where one sists empty most of the time and when needed is actually too small while the other is half empty, when you can unify them and make use of the whole pie as needed?
The problem is memory bandwidth. The M2 has a bandwidth of 100 GB/s, whereas this RX 7600 has 288 GB/s, and top of the line RX 7900 and RTX 4090 reach almost 1 TB/s. Reaching such speeds with standard DDR5 would require a massive number of channels that is impractical for desktop PCs.
The M1, M2, PS5 and Xbox get away with it by being closed systems without any possibility for expansion, but gaming PCs are expected to be expandable.
Depends on the machine. If you pair a RTX 4090 with a potato CPU then sure, if you go for something more reasonable, not at all. At least you need to consider the CPU, motherboard, and RAM.
Certainly PC gaming has terrible ergonomics and is very annoying. Sadly, some things you can't play on a console.
Cloud gaming is not even close to being an option, the latency is disastrous. Even streaming locally on gigabit network has poor latency, let alone over the internet.
The main argument I can think of for separating their memory is that, of course, GDDR can be optimized for bandwidth and regular ram can be optimized for latency or somewhere in between.
But, memory latency continues to make poor progress compared to CPU speeds. So, since we’re already going to need a complicated system of caches on the CPU side, maybe it is not such a big deal if CPU memory acts more like GDDR.
It doesn't explain why GPUs that are integrated into a CPU (like AMD's ones) and use the same memory, cannot use it all.
If they could use all memory, you could cheaply run neural networks with 64Gb of RAM without buying a professional GPU. No wonder manufacturers don't want you to be able to do that.
Both discrete and integrated GPUs can use as much system RAM as you allow them in your BIOS or operating system.
The problem is that system RAM is slow compared to onboard VRAM on discrete GPUs. Size isn't everything, speed and latency are also factors that have to be considered.
I imagine if that ever happens it will only be used to nickel and dime consumers and you'll end up paying even more through forced upselling than with the inefficient VRAM we currently have.
I can’t imagine the SOCs are too far away for PCs. It does go against the upgradeability philosophy but I’m not sure how many users really care about that
If you put the memory on the same package as the CPU it can have higher bandwidth without needing an unreasonably high number of pins in the CPU socket.
But there is no reason you can't do both. Intel is already by putting HBM on the Xeon Max. It still has separate memory slots, because they're for two different things. HBM is fast and expensive. DDR costs less so you can have more of it for the same amount of money and the large majority of workloads aren't memory-bound.
You could then have systems with DIMM slots that aren't populated because the CPU has enough HBM to satisfy you, others that cost less with no HBM and the same amount of DDR, and still others that have say 16GB of HBM and 128GB of DDR. Both of which are still upgradable, but if you want more HBM it comes with a new CPU.
As exemplified in my post above, AMD integrated GPUs don't have unified memory, at least my 5000 series I bought in 2022. Maybe the new 7000 series changes this.
Not exactly. While that may have been the case way back in the past when AMD APUs were designed as a homonegnous CPU-GPU unit from the start, I can tell you that's not the case with the relatively modern 5000 series.
I need to go in the Bios and specify explicitly how much of the system RAM I want allocated exclusively of the integrated GPU and the rest stays available for system RAM.
The reason is because Ryzen 5000 APUs seem to be a job rushed out the door so they're just a Zen 3 CPU with a separate Vega GPU glued together on the same die, but they're not a homogenous design, designed to work as one unit, like the APU of the PS5, so memory wise they're unaware of each other, even though AMD calls them APUs, they're not really, but more like separate CPU and GPU on the same die.
I wish I knew about this limitation at the time, as Intel chips with integrated graphics have unified memory.
"Unified" doesn't mean the CPU and GPU can store different data in the same memory at the same time. The BIOS setting is just assigning how much are visible for each to use in the absence of OS support for doing it dynamically. Clearly if you can change the amount it isn't a hardware limitation, and I'm not even sure if it needs some special support. The iGPU surely supports DMA, so if it does a load from memory "assigned" to the CPU does it perform any differently than from memory "assigned" to the GPU?
Sorry but this sounds like jumping to conclusions, going off only the fact that you can fence off some memory in the BIOS. See eg this windows gaming site review where they couldn't find a difference in the performance between allocating 64MB or 2GB save for satisfying some games that need to detect a certain amount of GPU memory to start: https://www.techspot.com/article/1578-amd-raven-ridge-reserv...
But I don't doubt there still exist software (Firmware, OS, driver, graphics API) problems where unified memory ends up being used badly. There doesn't seem to be evidence that much if any of it are because of hw limitations.
I wonder if a clever enough driver+firmware could make that hot-adjustable (or truly unified, rather) with nothing but a software patch. It sounds like the pieces are all there.
My understanding is that the Steam Deck, which uses a Zen 2 APU, dynamically allocates system RAM to VRAM. It only works in SteamOS (Arch Linux) though. Folk that run Windows on their Steam Decks need to go into the BIOS and bump the static mapping to 4GB in order to get good performance in many games.
A mid-lifetime upgrade where you pop in a new GPU, fill up any unpopulated ram slots, or add in a new drive, can be nice. CPUs last a really long time nowadays.
I would hope that this is more common than GP thinks, and this is also something I look for when I'm buying laptops- if they don't have an okay amount of extra PCIe lanes laying around, I don't want it- because they can become really fun projects (ie strap a GPU to it and get a few more years of Minecraft out of it, or use it as a network appliance with a built-in UPS)
That’s a cool idea. How do you get to the PCIe lanes?
I’ve been hoping thunderbolt would make GPU enclosures more of a thing, but they seem to still be bottlenecked by even that pretty good interface (last I checked, at least).
In the past, the thing to look for was laptops with multiple mini PCIe connectors (generally for WiFi cards and sometimes cellular) but now the thing to look for is M.2 slots with PCIe on them. Generally the easiest thing to do is to yank out the wifi card and add your own thing there, and then for networking you can just use either onboard or a decent type-c gigabit dongle. Some laptops have spots for multiple SSD's and you can just use one of those instead.
Keep in mind that even a basic PCIe card can pull up to 75 watts and that you'll probably need some sort of external power supply. There are two main routes you can go- use a second external laptop power brick, or what I did- slap an entire computer power supply in there.
I bought an SFX power supply from EVGA, who has an awesome B-stock program where you can get returned / refurbished stuff for cheaper than normal and it still comes with a warranty. If you do decide to use a computer PSU, I'd highly recommend one that's modular, because you won't need a bunch of the wires and they take up a lot of space quickly.
To tell the power supply to turn on, you can either use the 'paper clip trick' to jumper it on, or get an "ADD2PSU" board (basically a relay on a PCB) and use a USB cable to power the relay and you should be good to go.
If you use a laptop power supply, you'll either need something that's 12V or get step down boards or a Pico PSU or something like that.
The thing to search up is "M.2 NGFF to PCIe" or something like that. I'd recommend getting ones with some kind of cable and an open back of slot design so that you don't wind up needing two adapters for one thing.
Adapters and cables are available from the usual suspects- Amazon, eBay, and Aliexpress will all have what you want.
This is a lot of information, but this is legitimately pretty easy to do as long as you make sure you check all the boxes- PCI lanes, external power, and something to keep the new stuff you're strapping on safe.
Right. At which point, why pay the cost of the rest of the PC being modular? Both the $800 and the $200 would be smaller amounts in the case of an integrated system.
Sure. But if the entire integrated system can cost a similar amount to the most expensive component you might upgrade in a modular system, there’s little practical difference.
Depends how good the original was if it’s worthwhile. GPUs don’t need that much from the rest of the machine as long as it’s somewhat sensible for gaming. Especially at 1440p or 4k the GPU is a huge portion of the performance
My current PC is using an old 4th gen i7 with an RTX 3060Ti and it keeps trucking along just fine, even with my 144Hz display. It used to have a GTX 960 but its 2GB of VRAM were anemic for some ML stuff I wanted to run + the newer Nvenc is a massive upgrade in quality. I don't see any reason to upgrade anything else.
Amd socket AM4 supports 5 generations of Ryzen processors. AM5 came out with 7000 series and it will be supported by 8000 series as well. Upgrades are easy and give a significant performance boost.
Power standards aren't changing that much. Yes, there's 12VHPWR now, but that's only on the top end gpus, and adapters exist. ATX 12VO might become a thing, but feels like OEM standardization.
Before that, the last big change was Haswell low power states ten years ago. If your old PSU didn't meet that standard, you would probably be ok if you just disable the lower power sleep states.
I'm suddenly curious if this is a fairly static limit: 8GB for LLMs. Is there any paper or posting somewhere describing how this will move up or down with technology changes? Does it depend entirely on the "thickness" or density of the data being analyzed to produce a model? I'm imagining a spreadsheet or database table with 100 rows vs one with 1000. I guess it depends entirely on how thoroughly data was collected? Not all attributes are dependent, etc.
Almost twice the bandwidth of the rx580, from April 2017. About the same msrp.
It'll be interesting to see how this holds up versus Intel, whose been doing pretty ok at this price point. Intel's initial launch was pretty rocky but the drivers have gotten much much faster already.
As an RX580 user, I was hoping for a bit more memory - as it stands, this doesn't feel compelling for me to upgrade. Maybe there'll be a 7600xt or something, but I'll be holding out another year or three (and turning graphics settings down to medium)
I have an RX 6700XT that I bought used that's been flawless since I got it, even though I've been doing cursed stuff with it. Is this specifically a 7000 series gpu problem?
It's also so they only need to design one memory controller for 6nm. I believe I remember this corroborated from an AMD engineer interview around the 7900XTX launch. Memory controllers aren't just logic that can be "compiled" to whatever target node. They have specific electrical requirements that take substantial design work. For this generation AMD has a 6nm memory controller that they use both in this 6nm monolithic design and in their 6nm memory controller chiplets on the larger designs.