You need to be more specific. If your point is that you literally cannot backprop while doing a forward pass, then sure. But you can certainly do inference then backprop on the outcome…
I fail to see how that is substantially different from a human doing something then reflecting and learning.
You can critique LLMs and transformers generally, but to say it’s somehow a problem with backprop is a bit of a stretch.
I think AI can be great for this. It can make so many thing accessible to people that could not otherwise. Your example here is good and there are some legitimately fun AI songs.
The key is being able to discriminate something you’ve done for fun and being a serious artist. The problem becomes when people conflate something they’ve done as a fun hobby as being worthy of sitting next to human created art.
I’ve used AI to code lots of things for personal use that I wouldn’t have had the time to complete otherwise. The difference is, I use them personally and don’t release them as a serious software product.
Yes but that’s related to LLMs specifically, not backpropagation. There’s plenty of ML paradigms that use backprop and have continual learning setups.
Backprop is the process of how the weights are updated.