Given how murky the internal activities of AI labs are and how new the AI industry is, it is unclear whether or to what extent AI labs are currently engaging in RKA. However, the labs have a strong incentive to do so. New data, better models, and new markets will be critical to maintain the stratospheric growth that the market valuations of labs imply. Recent macroeconomic work on AI bubble dynamics helps formalize this pressure: High valuations can be sustained only with high growth and eventually high profits. Given that AI is a “general purpose technology,” there has already been a natural incentive for AI providers to integrate their models into a wide range of industries. The next step is for those providers to expand their market vertically to downstream uses, owning those uses, not merely servicing them.
The industries most vulnerable to RKA are those in which information shared with AI providers directly implicates the industries’ products or services—for instance, software or legal. In such industries, AI usage data from customers could most plausibly be employed to develop competing products or services. For example, employees of an enterprise software company might use AI models to write code or analyze data, disclosing critical information about enterprise software to AI labs. Similarly, in the legal industry, lawyers might employ AI models to execute legal workflows, revealing proprietary legal strategies. And in the financial services industry, AI models might reveal sensitive information about investment strategies. Moreover, if many employees within the same enterprise are using the same AI models simultaneously, the available information, once linked and synthesized, could reveal much more about the enterprise than any single employee has access to.
To make matters worse, there is no reason to think that the laptop-text-based interface for AI will remain predominant. AI startups are already building ambient note-taking devices, integrating agents directly into workflows, and incorporating visual AI into augmented reality products. Further in the future, embedded AI will escalate this effect—for example, deployed industrial robots can be used as data collection devices, extending the threat of RKA to industries like manufacturing, where incumbents may license robots only to have their industrial information appropriated by them. As the possible use-cases for AI expand, so too will the set of information available for acquisition and, thus, so too will the possible targets of vampiric inference. In other words, there may one day be no industry safe from the threat of RKA.
I don’t agree with the almost implicit assumption that AI works well and can actually replace a lot of industries and workers… but the broader point of this article is definitely worth taking seriously, the clear incentive structure here is for AI companies to try to escape their collapsing business model by attempting to steal the business model of other companies by vacuuming up all of their data in a business to business contract for AI and then turning it against them.
This argument hinges on the idea that the most important part of a business is its ‘data’, far more important than anything like its assets or its workers or its long-standing client relationships, which is… dubious.
If you presuppose that all the value of an enterprise is confined entirely to the part that’s computable, then you’ll be shocked to discover that a computer can do all of it. But a computer is not doing all of it, and if a real existing business today cannot replace itself with a computer, why would some vastly overcapitalised slop foundry be able to?
I think we have already seen what direct replacement by AI companies looks like, and it’s not pretty. Want an image for an ad but you’re too cheap for a designer? There’s a chatbot for that. Want to represent yourself in court? There’s a chatbot for that. Slopplications of that sort did capture some demand for services, hardest felt by writers/graphic designers because you can more easily get away with a steep drop in quality there (for a little while).
I don’t believe a takeover of that sort can happen in more sophisticated cases. Critical knowledge in SW (speaking from my wheelhouse) isn’t some guarded secret-burger-sauce-recipe kind of knowledge. It’s vast amounts of knowledge, organized and made actionable through effort and built-up skills. It’s not knowledge they would have to replicate, it’s labour. And I don’t think they can privacy-violation their way to replicate a domain-specific knowledge worker from a chat history.
I don’t disagree with the thrust of what you are saying, but you are also misunderstanding something here, **we aren’t talking about replacing a human employee with an “AI” for a task like creating a marketing image, we are talking about convincing large corporations and entities to feed all of their data into an “AI Lab” and then that “AI Lab” basically does traditional industrial espionage and sabotage to replicate the business model of the company/industry they were supposed to be assisting.
This would likely be done with a lot of automation, but it is CRUCIAL to understand this strategy can be done without AI at all, what matters is convincing organizations to hoover all of their sensitive data over to you and then leveraging it in ways that are opaque enough that blowback either doesn’t happen or is long delayed.
I just don’t think they’re smart enough to make a dent in an actually competitive industry, no matter how much espionage they got.
They may throw some cash around and buy their way into a niche field that actually has a business model, but I don’t see how the espionage potential would help them much. To break into a difficult domain based on that, you would need the smart humans who can tell you what you are looking at.
Maybe it would be possible if you assume competent shady management, that is willing to invest heavily in human capital and systematically violate every privacy policy in the world. I think that’s just not how they operate.
I just don’t think they’re smart enough to make a dent in an actually competitive industry, no matter how much espionage they got.
Which is exactly why they buy up all the enterprise data streams and machinery around it relevant to the industry before they try it, so they don’t have to be competent…