AI will not wait for companies to get their policies in order

In the first part, Eva Gostisa and Nejc Setnikar of the law firm Jadek & Pensa presented the legal framework of the AI Act, copyright, and personal data protection, while in the second part I spoke about what artificial intelligence already means for companies’ online and information security. 

We discussed the AI Act, copyright, data protection, and how artificial intelligence is already changing how companies operate and digital security. 

No futurism. No panic. And no promises that AI will miraculously solve all business problems. 

How companies govern the use of AI today 

Today, we all talk about AI, but this event was deliberately much more grounded. No promises of miraculous productivity and no futuristic scenarios where artificial intelligence solves every business problem. And also no panic that it will take over companies tomorrow. 

The truth is, as usual, less cinematic, but considerably more uncomfortable. 

AI is already here. Companies are already using it, often much more than they think. Someone uses it to help write proposals. Someone uses it to summarize meetings. Someone pastes an internal document into it to draft a reply faster. Someone in marketing generates an image. Someone in development generates code. Someone in sales prepares a personalized email. 

And then a very basic question arises: does the company even know where all of this is happening? 

Very often the answer is: not entirely. 

That was also one of the key points of the first part of the event. The AI Act does not replace the GDPR. The rules do not replace one another; they stack on top of each other. If something is compliant with the AI Act, that does not mean it is also compliant with the GDPR. And if a company uses an AI tool, that does not mean it understands which data go into it, who is responsible for them, and what happens to them afterwards. 

Generative AI raises new questions 

With generative artificial intelligence, things get complicated quickly. The model does not operate like a spreadsheet in Excel, where you correct or delete a row. Information can surface as part of generated content, as an approximation, as an inference, or as a pattern. 

The event also mentioned the GEMA v. OpenAI case in Germany, where a Munich court ruled that OpenAI, through the training and operation of ChatGPT, infringed copyright in protected song lyrics. The court also treated as significant memorization, i.e., the possibility that protected texts are reproduced from the model in generated content. 

This is a good example because it very clearly shows the difference between how we would like the technology to work and how it actually works. 

Companies want clarity. Who owns what? Who is liable? Where is the data? What can we use? What can we delete? 

AI often answers: it depends. 

And 'it depends' is not exactly the favorite phrase of lawyers, leadership, or security teams. 

What happens when someone outside the company starts using AI 

While the first part of the event was mainly about how companies govern AI use internally, in the second part I wanted to show the other side: what happens when someone outside the company starts using AI. Someone with no interest in being compliant. Someone who does not care about the policy. Someone looking for the cheapest path to your data, money, or access. 

This is where the story changes very quickly. 

In my view, artificial intelligence does not bring that many entirely new types of attacks. Rather, it makes existing attacks faster, cheaper, and more effective. That is a fairly important distinction. 

Phishing has been around for a long time. So has social engineering. So has the abuse of publicly available information. Fake CEO calls, fake invoices, fake domains, poorly protected email, and stolen passwords—none of this is new. 

What is new is that today someone can do all of this better, faster, and in language that no longer has obvious errors. 

AI phishing in flawless Slovenian 

For a long time, what saved us with phishing was mainly poor Slovenian. Odd sentences. Wrong cases. That feeling that the person on the other side did not really master Slovenian. 

That period is over. 

AI phishing in flawless Slovenian is no longer an advanced technique. It is a basic function. 

And when the grammatical errors disappear, one of the most convenient warning signs people have learned to rely on disappears. An attacker no longer needs perfect command of the language. They do not need a copywriter. They do not even need much time. They need some public information, a bit of context, and a tool that can turn that into a convincing message. 

Companies’ public digital footprint is larger than we think 

There is more of this public information than companies are prepared to admit. 

In the presentation, I showed how an attacker can quickly assemble a picture of an organization based on a LinkedIn profile and company posts, job ads, event photos, and news. Who works in finance. Who is in HR. Who is new. Who has just been promoted. Who is a supplier. What software you use. Who could approve a payment. Who is probably the best target. 

This is no longer manual detective work. AI assembles a vulnerability profile from this data in a matter of minutes. 

If you state in a job ad that you are looking for an SAP administrator, you have said something about your infrastructure. If you announce that you have signed a contract with a new partner, you have given someone excellent context for a fake message. If a leadership change is posted on LinkedIn, it can be an ideal moment for an attack, because people are still getting used to the new hierarchy. 

The biggest problem is often not advanced attacks 

What is even more concerning is that many companies do not face their greatest risks with advanced attacks, but with basic security practices. 

We are not talking about quantum cryptography. We are talking about 2FA. About password managers. About not sending passwords over email or personal channels. About backups following the 3-2-1 rule. About a properly configured DMARC. About not leaving the domain completely open. And about the company even knowing who owns the domain and who has access to key systems. 

This is IT, too 

One of my slides had the sentence: 'This is IT, too.' 

Because in practice, IT is not just infrastructure or user support. IT is often everything. The website. Email. Access. Domains. Computers. Phones. The bank. CRM. ERP. Backups. Passwords. Employees. Suppliers. 

In many small and mid-sized companies, all of this often depends on one person. Without clear processes. Without enough time. Often also without the proper mandate to put things in order in time, before an incident occurs. 

As long as everything works, it is invisible. 

When something goes down, it becomes very visible. 

Today, companies need less excitement and more order 

And this is exactly where AI and security meet the same problem as AI and regulation: companies do not need more excitement. They need more order. 

Not in the sense of bureaucracy, but in the sense of basic hygiene. 

They need to know which AI tools they use. Which data go into them. Who is allowed to use what. Where the risks are. How suppliers are vetted. How payments are approved. How employees recognize an attack that is no longer broken and obvious, but well written, personalized, and very well timed. 

AI will not wait for companies to get their policies in order. Neither will attackers. 

Today, AI is above all a matter of responsibility 

That is why the most important conclusion seems quite simple to me: artificial intelligence is no longer just a topic of innovation. It is a topic of responsibility. 

Legal. Organizational. Security. 

And if companies are already using it—which they almost certainly are—now is the right moment for less excitement and more stocktaking. 

Because the biggest risk is not that companies will not use AI. The biggest risk is that they will use it faster than they understand it, while attackers will understand it well enough to use it against them. 

The technology will not slow down. The only question is how quickly companies will learn to live with it.