GPT is basically a machine that builds machines.
We understand how the machine builder works. We don’t understand how the output of the machine works. Like mechanically we understand each individual sub-component works e.g. a single matrix multiplication, but we don’t understand why specific weights are the value they are, or the broader macrostructure.
I don’t think you realize that acting like “No we actually do understand how it works” just reveals that you don’t understand the question, it does not show that the question has an answer.



I was never that into the open source community, but I did admire it from afar.
I don’t see how anyone would ever contribute to open source now, knowing it will just be thrown into the meat processor.