large language models are large and must be loaded into memory to train or to use for inference if we want to keep them fast. older models like gpt3 have around 175 billion parameters. at float32s that comes out to something like 700GB of memory. newer models are even larger. and openai wants to run them as consumer web services.
Suppose some some parallelized, distributed task requires 700GB of memory (I don't know if it does or does not) per node to accomplish, and that speed is a concern.
A singular pile of memory that is 700GB is insufficient not because it lacks capacity, but instead because it lacks scalability. That pile is only enough for 1 node.
If more nodes were added to increase speed but they all used that same single 700GB pile, then RAM bandwidth (and latency) gets in the way.
This "memory shortage" is not about AI companies needing main memory (which you plug into mainboards), but manufacturers are shifting their production capacities to other types of memory that will go onto GPUs. That brings supply for other memory products down, increasing their market price.
Maritime shipping software | Full-stack software engineers | UX Designers | Implementation Managers & Customer Success execs | Data analysts | Global REMOTE | UCT-8 to UCT+4
Would you like to help tackle one of the most important sectors for the global economy - maritime shipping?
We're hiring for multiple roles: software engineers, data analysts, UI/UX designers, implementation managers.
Nearly every person is affected by stuff shipping on the seas - whether its energy, ore, or wind turbines, but the industry itself is antiquated and archaic, and hampered by paperwork and bureaucracy. Our customers are crying out for better solutions. We're looking for people to help us build those solutions.
If you're the sort of person who is:
- intellectually curious about new industries
- wanting talking to and getting close to end users
- keen to use LLMs and other AI tools
Then please get in touch.
--
For our SWE roles we have budgeted ~ 90k USD.
Our tech stack for new projects is largely typescript/vue/mysql (we have several fans of fp-ts and EffectT with us and are investing in using more functional programming). But like the industry at large we also have significant legacy code in javascript. If you're the sort of person who likes refactoring complex projects to make them more workable, then we also want to hear from you.
--
For our Data Analyst or Implementation Manager roles:
You'll be helping our customers in maritime shipping get the most from our software. Initially, a lot of that work will be in helping our customers get their data in and out of it. Lot's of MySQL, Metabase, AWS Glue, DBT, or other ETL pipelines.
--
For our UI/UX designer roles we have budgeted ~ 40k USD.
You'll be helping us on three different axes: (1) designing the user interface for new features, (2) fleshing out the user journeys and mockups for our roadmap, and (3) helping us craft a consistent and learnable user experience across all our software.
--
We hire remotely and globally with offices in Houston, London, Singapore, and Joinville (Brasil). You should be fluent in English. The current team is >20 people. You should be in a timezone within 3 hours of one of those offices.
Interested? You can contact me directly from my profile. I'm the CTO.
Maritime shipping software | Multiple roles | Global REMOTE | UCT-8 to UCT+4
Would you like to help tackle one of the most important sectors for the global economy - maritime shipping?
We're hiring for multiple roles: software engineers, data analysts, UI/UX designers, implementation managers.
Nearly every person is affected by stuff shipping on the seas - whether its energy, ore, or wind turbines, but the industry itself is antiquated and archaic, and hampered by paperwork and bureaucracy. Our customers are crying out for better solutions. We're looking for people to help us build those solutions.
If you're the sort of person who is:
- intellectually curious about new industries
- wanting talking to and getting close to end users
- keen to use LLMs and other AI tools
Then please get in touch.
For our SWE roles we have budgeted ~ 90k USD.
Our tech stack for new projects is largely typescript/vue/mysql (we have several fans of fp-ts and EffectT with us and would like some more). But like the industry at large we also have significant legacy code in javascript. If you're the sort of person who likes refactoring complex projects to make them more workable, then we also want to hear from you.
For our Data Analyst or Implementation Manager roles... You'll be helping our customers in maritime shipping get the most from our software. Initially, a lot of that work will be in helping our customers get their data in and out of it. Lot's of MySQL, Metabase, AWS Glue, DBT, or other ETL pipelines.
For our UI/UX designer roles... You'll be helping us on three different axes: (1) designing the user interface for new features, (2) fleshing out the user journeys and mockups for our roadmap, and (3) helping us craft a consistent and learnable user experience across all our software.
We hire remotely and globally with offices in Houston, London, Singapore, and Joinville (Brasil). You should be fluent in English. The current team is >20 people. You should be in a timezone within 3 hours of one of those offices.
Interested? You can contact me directly from my profile. I'm the CTO.
Hi there. I have about a year's worth of experience working with Java, Spring Boot, Python and Node.js. Would love to have a chat regarding this job role. Here's my linkedin: https://www.linkedin.com/in/sherwyn-a-fernandes-/
I would treat them as totally different things. There is a Category of Sets, but a Set of Categories would be a bit harder to define. So axomatic set theory could be a specific case of category theory, I suppose. But you can probably do a Class of all categories. (A Class is sort of a set-theoretic way to get around Russel's paradox, incidentally, you usually use a Class to define categories, so...) Though that's actually quite an irrelevant point. It's a completely different language for describing mathematics. I think describing category theory as an alternative foundation for mathematics (you really mean topos theory here) is a bit of an exaggeration. it's technically true, but most mathematicians I know are using it as a powerful device to prove things in algebraic topology or geometry, etc.
> In the case of GIMP, the very name itself means it cannot be used in commercial enterprises.
I have no idea where you got this info from but it is 100% wrong. Of course GIMP can be used for commercial purposes. Or any purposes you want, really. The GNU GPL does not prevent commercial use.
Nobody bats an eye when we say we edited their pictures with GIMP. Even our British customers, who are more likely to understand what "gimp" means, I guess.
I mean, they already often use mac OS after all, which means pimp, in French. And who cares.
> - can you allocate memory for the whole system? - can you make types homogenous so they can fit in tight arrays (unions are common for nodes) - can you batch similar types - specially for auto diff/math can you represent operations as a stack instead of a tree?
these are good questions, thank you! I'm "learning" c++ in a completely different way - looking at books rather than looking at existing code. Appreciate this comment!
Some of these questions I had thought about, but was learning more towards "describe the function in some higher level representation and then 'compile' it down to something computable and autodiff-able". This is exactly where my mind goes when I think about allocating memory for the whole system. that felt more like a racket/lisp way of looking at the problem.
Lean definitely intends to be usable as a general purpose language someday. but I think the bulk of the people involved are more focused on automated theorem proving. The Lean FRO [0] has funds to guide development of the language and they are planning to carve out a niche for stuff that requires formal verification. I'd say in terms of general purpose programming it fits into the category of being "relatively far from haskell in terms of maturity".
much of your experience with this depends on who you are and how society perceives you. I do not drink alcohol, but I am also a 6ft tall competitive weightlifter. People usually don't question my "no, thank you". And while I have had the annoying person persist and try to get me to drink, they're usually gone after I've reached my 2nd stage of "did you not hear me say no?". I suspect the experience of a less tall and muscular woman might be quite different when it comes to people trying to undermine her choices.
society has a lot of "defaults" that we're suppose to accept. drinking is one. cars is another. eating meat is often one. Some people see it as an attack on their identity when you reject choices made by their society.
> Some people see it as an attack on their identity when you reject choices made by their society.
Spot on. When I was a freshman in college, my then-girlfriend and I decided to become vegetarian. My parents responded with some puzzlement but were ultimately supportive. She, on the other hand, spent the evening crying with her parents, who saw it as a rejection of their lifestyle. I remember her father asking me with a mix of anger and frustration “but what do you EAT?!” and I pointed out that all the things on the plate that were not meat were, in fact, edible. In retrospect, this was the wrong approach (I was reflexively sarcastic in my youth), and I would now approach the question with a bit more compassion. But the experience helped me see why small things (eg, do you drink or not, do you own a gun or not, do you exercise or not, etc) can foment deep divisions among people.
