April 24, 2024

The mountain of shit theory

Uriel Fanelli's blog in English

Fediverse

And punctual, the accher arrive.

And punctual, the accher arrive.

The problem of hacker attacks in the Lazio region lies not so much in the fact that someone has placed ransomware inside the systems, but in the fact that institutional communication betrays a catastrophic sloppiness, which is characteristic of tenders in the Italian PA.

If you tell me that your system has been attacked, in fact, you are telling me something that does not surprise me. If the actor attacking you is sophisticated enough, keeping a system safe is very difficult, and even very sophisticated measures may prove insufficient. The problem of computer security as a matter, in fact, is that it is the antithesis of sloppiness.

You can only be relatively confident if you have done EVERYTHING possible.

Having said that, however, at some point some genius decided to reveal some details.

The first is that an "administrative" computer left on has escalated the privileges of the domain controller, which in turn has spread the infection to the servers where the reservations were made.

As an architect, I see here a catastrophe of incompetence.

  • apparently, production systems accessible from the internet were in the same domain (I suppose windows) as a system that should be in an “office” type network.
  • apparently, the production systems accessible from the internet are in the same network, and are also accessible from systems in the office network.
  • perimeter security was at least questionable. apparently the intruders figured out a single user's VPN username and password. Two-factor authentication? None.
  • the single user, it seems, was one of the system administrators. And here, I stop for pity.

In practice, if I read a communication matrix of that network I would find that all the machines (domain controllers, production systems, office network computers) are on the same segment, without restrictions. As if that weren't enough, the perimeter of the systems (the VPN) was covered only by ONE factor authentication. No bastion hosts between networks, no jump boxes, no segregation.

Interesting, but this is a portrait of a home wifi.

The first point is that in a large organization rarely do you put domain controllers in the same computer segment as employees (at most you put secondaries or satellites, depending on the type of controller). If this is done, then the segment must be segregated from the rest, as often happens for the operations departments, which usually enter the production networks via jumpbox. The second point is that the production machines are not placed in the same "office" network, even less if they are exposed to the internet. Nor in the same microsoft domain. Otherwise you are building a security bridge.

I don't want to know how and why an employee's computer was attacked from the internet: I expect an office network to not be exposed to the internet. Otherwise we would have to talk about an attack on layer 8 (social engineering on the employee using that computer.). I learn that the attackers would have "learned the password of the vpn", which makes the arms fall even more '.

And then when I hear that the ransomware has taken control of the domain controller, my response is on average "plan everything and do it from scratch". It is not at all obvious that the sole purpose of that software was to ask for a ransom, and if it infected the controller it could have installed software anywhere, on any computer in the domain. And the hostile software may even have copied itself into some silly firmware. Moral? change everything. But absolutely everything, because even a USB stick forgotten somewhere can be infected. (it was the solution chosen by the German parliament when they discovered they had been infected by the Russians, to say: all hardware removed and replaced).

But the problem is not even the "what to do", in Italy there are professionals who can advise. The problem is the sloppiness this system demonstrates.

In many cyber attacks the technique used is sophisticated. It is as sophisticated as the defenses are. But in this case, the very idea that there was almost no segregation between production systems and office systems speaks very clearly. Sloppy, right from the initial design.

Here it was a stupid layer 8. But really stupid. Sciatteria at its finest.

And here's the thing: cyber security and sloppiness are sworn enemies. It is like water and fire: where there is fire there is no water, and where there is water there is no fire. More sloppiness, less security.

The digitization of the PA, requiring security, places the need for the state to free itself from the sloppiness that characterizes public procurement.

Because a system built with sloppiness is, from the point of view of an attacker, open like a mussel in Taranto. The correct answer to the question “which are the safest systems in the world” is very simple: those well thought out, well built, well maintained.

Solutions?

The problem with sloppiness is that it stratifies. If you go to that infrastructure and try to fix the problem, you will typically find that you don't have enough licenses for that domain or multiple domains. Then you make them buy, and you find that the networks are not segmented. Try making VLANs and find that half of the switches are not capable. Then you replace half of the switches, and you find that they consume too much energy and the electrical system does not hold them. Then get the electrical system fixed and find that the building in general does not have enough power. And if you try to bring more power they will tell you that the building lacks the right system … .. and when you are done, you will find that the floors do not hold the weight of the racks and that the air conditioning is insufficient and that, and that, and that …

If you try to act only on the servers, you will discover that the servers do not have the license for management, that the BMCs were not in an offband network, that to tell the truth there is no offband network (as in this case), you will discover that the domain does not control what is installed on individual computers, and so on, layer by layer by layer….

And all these layers are due to the fact that at some point there was a "political" meeting where it was decided "we should do this but there is no time", "we should do this but there is no budget ", or" we managers have screwed up the project times "and" we managers have screwed up the budget ". The result of these "political" decisions is that they are sloppy decisions, which start from the idea that technicians are "theoretical" (yes, in theory we would do this, but in practice …) while the "practical" ones would be the manager.

And technicians are normally treated as theoretical characters who ask for the dream book, while it is up to the "politicians", that is, the gentlemen of sloppiness, to be "practical" and make ends meet.

Ramsonware attacks happen every day all over the world. But when you are called in, all you find below is BAD IT management. More sloppiness, less security. And this is true all over the world.

But in this case, if only half of what the newspapers wrote is true, the attack was relatively straightforward. A sign that the design was already bad.

What is going wrong then?

Until now, the IT systems of the public administration were small islands, closed boxes that did not undergo any control and any competition. The sloppiness therefore settled without anyone noticing it, and only the older employees remembered what the fuck is that damned token ring cable that has been there for decades but you can't touch hindsight it all comes down. Nobody controlled, and that empowered the old guy who knew the story of the 1988 token ring cable. "We put this cable on when I still had a lot of hair … snif, snif, what a time."

Now the rules have changed. Because if that fucking token ring cable unfortunately exposes something to the network, some hackers will find it. They have all the time they want to attack. And then it would be better for it to be documented, and for someone to take care to get rid of obsolete technologies.

The DPCs of the Italian PA are facing a new challenge with digitalization: the one in which to check that everything is done WELL are not "ministry inspectors", but hostile entities that if you make a mistake open you up like a mussel. And they have no mercy. And they don't give you three months to put everything back together. And they don't hear blablabla.

But still: solutions? Since sloppiness is layered, it is not possible to get hands on with these systems. You will find undocumented stuff that "can't be touched", you will find absurd situations, impromptu remedies taken in a hurry, impossible and fortuitous workarounds, stuff that only works with version A but not with version B, in a spaghetti layering of " non possimus ". Forget it.

The only thinkable thing is the segregation of the PA from the global internet. Which does not solve the problem but MITIGATES it.

It means that the whole PA ends up in a network segment that is not visible to the rest of the world. But from Italy (and only from the wholesale network on cable, not from servers in Italian datacenters: only the residential and mobile user, that is, identifiable by the police) can enter this network through highly observable proxies or routers. But these routers must be reachable only by Italian ASs dedicated to the access network.

Of course, this keeps open the possibility for the malicious attacker to buy a line in Italy and attack from there, but to buy the line he must give his details. This does not produce security in itself (except perhaps against some DDOS) but at least it produces traffic observability and especially control: undoubtedly a malware could infect home computers and launch an attack from there.

To mitigate even these attacks (which in any case are more observable) it would be to decouple the systems and put an API gateway in front of us (so we are talking about a gigantic gateway): after having put the entire PA offband, what must be done (an office at a time) is to make them usable through an API gateway that works in front of the systems, without anyone being able to reach the systems themselves.

It involves building a gigantic API gateway, building it well and securely, and then integrating the PA systems one by one, taking care to make them unreachable from the internet immediately after integration. Similar designs have been adopted in both France and Germany and are mandatory for the PA in Switzerland – go for an API gateway anyway, even when reading instructions on what looks like a website.

Does this make you immune? Absolutely not, since at that point the game is protecting the API gateway. But it makes you "reasonably" safe. You could go even further, and make sure that the data in your databases cannot be changed.

If you use a database where you can only write the data but then you will no longer be able to delete or modify it (not necessarily a blockchain, almost all Hi-End databases have offered this feature for at least 30 years), the action of the ransomware that does cryptolocking it is difficult: it cannot encrypt the data because after writing they are immutable . However, this collides with the work you have to do, and you will have to do data engineering heavily. On all country data. Congratulations.

But here we go back to the starting point: the system had servers exposed to the internet exposed to the same domain controller as the machines in the office network. The DB data was not immutable, otherwise it would have been impossible to encrypt it. The offended systems were exposed to the internet without a GW API or queue system, an enterprise bus, in between. The systems of the Lazio region were exposed to the whole world, when they are usable (at least for that purpose) only by people who live in Lazio, or mostly in Italy. And so on.

All of these design deficiencies are not obvious in the short term.

And it is for this reason that the CEDs of the PA, from the point of view of the attacker, are simply fat defenseless prey.

Now you have to choose: either safety, or sloppiness.

Tertium non datur.

Leave a Reply

Your email address will not be published. Required fields are marked *